Identify entities from images

The following APIs will enable you to identify entities (faces and/or celebrities) from images.

Set up your backend

Run amplify add predictions, then use the following answers:

1? Please select from one of the categories below (Use arrow keys)
2  `Identify`
3? What would you like to identify? (Use arrow keys)
4  `Identify Entities`
5? Provide a friendly name for your resource
6  `identifyEntities`
7? Would you like use the default configuration? (Use arrow keys)
8  `Default Configuration`
9? Who should have access? (Use arrow keys)
10  `Auth and Guest users`

Run amplify push to create the resources in the cloud.

To detect general entities like facial features, landmarks etc, default configuration from CLI workflow will suffice (i.e. celebrity detection enabled & entity identification from collection disabled).

Amplify will now detect general entity features when IdentifyActionType.DETECT_ENTITIES is passed in. Results are mapped to IdentifyEntitiesResult. For example:

1public void detectEntities(Bitmap image) {
2  Amplify.Predictions.identify(
3    IdentifyActionType.DETECT_ENTITIES,
4    image,
5    result -> {
6      IdentifyEntitiesResult identifyResult = (IdentifyEntitiesResult) result;
7      EntityDetails metadata = identifyResult.getEntities().get(0);
8      Log.i("MyAmplifyApp", metadata.getBox().toShortString());
9    },
10    error -> Log.e("MyAmplifyApp", "Entity detection failed", error)
11  );
12}

1fun detectEntities(image: Bitmap) {
2  Amplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image,
3    { result ->
4      val identifyResult = result as IdentifyEntitiesResult
5      val metadata = identifyResult.entities.firstOrNull()
6      Log.i("MyAmplifyApp", "${metadata?.box?.toShortString()}")
7    },
8    { Log.e("MyAmplifyApp", "Entity detection failed", it) }
9  )
10}

1suspend fun detectEntities(image: Bitmap) {
2  try {
3    val result = Amplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image)
4    val identifyResult = result as IdentifyEntitiesResult
5    val value = identifyResult.entities.firstOrNull()?.box?.toShortString()
6    Log.i("MyAmplifyApp", "$value")
7  } catch (error: PredictionsException) {
8    Log.e("MyAmplifyApp", "Entity detection failed", error)
9  }
10}

1public void detectEntities(Bitmap image) {
2  RxAmplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image)
3    .subscribe(
4      result -> {
5        IdentifyEntitiesResult identifyResult = (IdentifyEntitiesResult) result;
6        EntityDetails metadata = identifyResult.getEntities().get(0);
7        Log.i("MyAmplifyApp", metadata.getBox().toShortString());
8      },
9      error -> Log.e("MyAmplifyApp", "Entity detection failed", error)
10    );
11}

As a result of passing in an image, the bounding box (android.graphics.RectF) that captures detected entity will be printed to the console.

Detect pre-determined entities in an image

You can also match entities from a pre-created Amazon Rekognition Collection in Amplify. To access this feature, you must used advanced configuration in Amplify CLI:

Run amplify add predictions, then use the following answers:

1? Please select from one of the categories below (Use arrow keys)
2  `Identify`
3? What would you like to identify? (Use arrow keys)
4  `Identify Entities`
5? Provide a friendly name for your resource
6  `identifyEntities`
7? Would you like use the default configuration? (Use arrow keys)
8  `Advanced Configuration`
9? Would you like to enable celebrity detection? (Y/n)
10  `Y`
11? Would you like to identify entities from a collection of images? (y/N)
12  `Y`
13? How many entities would you like to identify? (50)
14  `10`
15? Would you like to allow users to add images to this collection? (Use arrow keys)
16  `Yes`
17? Who should have access? (Use arrow keys)
18  `Auth and Guest users`

Run amplify push to create the resources in the cloud

Note: If entity detection was already configured, run amplify update predictions to reconfigure as necessary.

The value of collectionId is the name of your collection, which can be created directly via CLI. The value of maxEntities must be a number greater than 0 or less than 51 (50 is the max number of entities Rekognition can detect from a collection). If either value of collectionId or maxEntities is invalid, then identify will just detect entities in general with facial features, landmarks, etc.

If both collectionId and maxEntities are properly configured, then Amplify will detect entity matches from the Rekognition Collection in your app. Results are mapped to IdentifyEntityMatchesResult. For example:

1public void detectEntities(Bitmap image) {
2  Amplify.Predictions.identify(
3    IdentifyActionType.DETECT_ENTITIES,
4    image,
5    result -> {
6      IdentifyEntityMatchesResult identifyResult = (IdentifyEntityMatchesResult) result;
7      EntityMatch match = identifyResult.getEntityMatches().get(0);
8      Log.i("AmplifyQuickstart", match.getExternalImageId());
9    },
10    error -> Log.e("AmplifyQuickstart", "Identify failed", error)
11  );
12}

1fun detectEntities(image: Bitmap) {
2  Amplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image,
3    { result ->
4      val identifyResult = result as IdentifyEntityMatchesResult
5      val match = identifyResult.entityMatches.firstOrNull()
6      Log.i("AmplifyQuickstart", "${match?.externalImageId}");
7    },
8    { Log.e("AmplifyQuickstart", "Identify failed", it) }
9  )
10}

1suspend fun detectEntities(image: Bitmap) {
2  try {
3    val result = Amplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image)
4    val identifyResult = result as IdentifyEntityMatchesResult
5    val imageId = identifyResult.entityMatches.firstOrNull()?.externalImageId
6    Log.i("MyAmplifyApp", "$imageId")
7  } catch (error: PredictionsException) {
8    Log.e("MyAmplifyApp", "Identify failed", error)
9  }
10}

1public void detectEntities(Bitmap image) {
2  RxAmplify.Predictions.identify(IdentifyActionType.DETECT_ENTITIES, image)
3    .subscribe(
4      result -> {
5        IdentifyEntityMatchesResult identifyResult = (IdentifyEntityMatchesResult) result;
6        EntityMatch match = identifyResult.getEntityMatches().get(0);
7        Log.i("AmplifyQuickstart", match.getExternalImageId());
8      },
9      error -> Log.e("AmplifyQuickstart", "Identify failed", error)
10    );
11}

Detecting Celebrities

To detect celebrities you can pass in IdentifyActionType.DETECT_CELEBRITIES. Results are mapped to IdentifyCelebritiesResult. For example:

1public void detectCelebs(Bitmap image) {
2  Amplify.Predictions.identify(
3    IdentifyActionType.DETECT_CELEBRITIES,
4    image,
5    result -> {
6      IdentifyCelebritiesResult identifyResult = (IdentifyCelebritiesResult) result;
7      CelebrityDetails metadata = identifyResult.getCelebrities().get(0);
8      Log.i("MyAmplifyApp", metadata.getCelebrity().getName());
9    },
10    error -> Log.e("MyAmplifyApp", "Identify failed", error)
11  );
12}

1fun detectCelebs(image: Bitmap) {
2  Amplify.Predictions.identify(IdentifyActionType.DETECT_CELEBRITIES, image,
3    { result ->
4      val identifyResult = result as IdentifyCelebritiesResult
5      val metadata = identifyResult.celebrities.firstOrNull()
6      Log.i("MyAmplifyApp", "${metadata?.celebrity?.name}")
7    },
8    { Log.e("MyAmplifyApp", "Identify failed", it) }
9  )
10}

1suspend fun detectCelebs(image: Bitmap) {
2  try {
3    val result = Amplify.Predictions.identify(IdentifyActionType.DETECT_CELEBRITIES, image)
4    val identifyResult = result as IdentifyCelebritiesResult
5    val celebrityName = identifyResult.celebrities.firstOrNull()?.celebrity?.name
6    Log.i("MyAmplifyApp", "$celebrityName")
7  } catch (error: PredictionsException) {
8    Log.e("MyAmplifyApp", "Identify failed", error)
9  }
10}

1public void detectCelebs(Bitmap image) {
2  RxAmplify.Predictions.identify(IdentifyActionType.DETECT_CELEBRITIES, image)
3    .subscribe(
4      result -> {
5        IdentifyCelebritiesResult identifyResult = (IdentifyCelebritiesResult) result;
6        CelebrityDetails metadata = identifyResult.getCelebrities().get(0);
7        Log.i("MyAmplifyApp", metadata.getCelebrity().getName());
8      },
9      error -> Log.e("MyAmplifyApp", "Identify failed", error)
10    );
11}

As a result of passing in an image, identify will return the name of a detected celebrity.

Identify text

Label objects in an image

Identify entities from images

Set up your backend

Working with the API

Detect entities in an image

Detect pre-determined entities in an image

Detecting Celebrities