Page updated Jan 16, 2024

Identify entities from images

The following APIs will enable you to identify entities (faces and/or celebrities) from images.

For identifying entities on iOS we use both AWS backend services as well as Apple's on-device Core ML Vision Framework to provide you with the most accurate results. If your device is offline, we will return results only from Core ML. On the other hand, if you are able to connect to AWS Services, we will return a unioned result from both the service and Core ML. Switching between backend services and Core ML is done automatically without any additional configuration required.

Set up your backend

If you haven't already done so, run amplify init inside your project and then amplify add auth (we recommend selecting the default configuration).

Run amplify add predictions, then use the following answers:

1? Please select from one of the categories below (Use arrow keys)
2❯ Identify
3 Convert
4 Interpret
5 Infer
6 Learn More
7
8? What would you like to identify?
9 Identify Text
10❯ Identify Entities
11 Identify Labels
12
13? Provide a friendly name for your resource
14 <Enter a friendly name here>
15
16? Would you like use the default configuration? (Use arrow keys)
17❯ Default Configuration
18 Advanced Configuration
19
20? Who should have access?
21 Auth users only
22❯ Auth and Guest users

Run amplify push to create the resources in the cloud

Working with the API

In order to match entities from a pre-created Amazon Rekognition Collection, ensure that both collectionId and maxEntities are set in your amplifyconfiguration.json file. The value of collectionId should be the name of your collection that you created either with the CLI or the SDK. The value of maxEntities should be a number greater than 0 or less than 51 (50 is the max number of entities Rekognition can detect from a collection). If both collectionId and maxEntities do not have valid values in the amplifyconfiguration.json file, then this call will just detect entities in general with facial features, landmarks, etc. Bounding boxes for entities are returned as ratios so make sure if you would like to place the bounding box of your entity on an image that you multiple the x by the width of the image, the y by the height of the image, and both height and width ratios by the image's respective height and width.

You can identify entity matches from your Rekognition Collection in your app using the following code sample:

1func detectEntities(_ image: URL) async throws -> [Predictions.Entity] {
2 do {
3 let result = try await Amplify.Predictions.identify(.entities, in: image)
4 print("Identified entities: \(result.entities)")
5 return result.entities
6 } catch let error as PredictionsError {
7 print("Error identifying entities: \(error)")
8 throw error
9 } catch {
10 print("Unexpected error: \(error)")
11 throw error
12 }
13 }
14}
1func detectEntities(_ image: URL) -> AnyCancellable {
2 Amplify.Publisher.create {
3 try await Amplify.Predictions.identify(.entities, in: image)
4 }
5 .sink(receiveCompletion: { completion in
6 if case let .failure(error) = completion {
7 print("Error identifying entities: \(error)")
8 }
9 }, receiveValue: { value in
10 print("Identified entities: \(value.entities)")
11 })
12}

Detecting Celebrities

To detect celebrities you can pass in .detectCelebrity in the type: field. Results are mapped to IdentifyCelebritiesResult. For example:

1func detectCelebrities(_ image: URL) async throws -> [Predictions.Celebrity] {
2 do {
3 let result = try await Amplify.Predictions.identify(.celebrities, in: image)
4 let celebrities = result.celebrities
5 let celebritiesNames = celebrities.map(\.metadata.name)
6 print("Identified celebrities with names: \(celebritiesNames)")
7 return celebrities
8 } catch let error as PredictionsError {
9 print("Error identifying celebrities: \(error)")
10 throw error
11 } catch {
12 print("Unexpected error: \(error)")
13 throw error
14 }
15}
1func detectCelebrities(_ image: URL) -> AnyCancellable {
2 Amplify.Publisher.create {
3 try await Amplify.Predictions.identify(.celebrities, in: image)
4 }
5 .sink(receiveCompletion: { completion in
6 if case let .failure(error) = completion {
7 print("Error identifying celebrities: \(error)")
8 }
9 }, receiveValue: { value in
10 print("Identified celebrities with names: \(value.celebrities.map(\.metadata.name))")
11 })
12}