Label objects in an image

The following APIs will enable you identify real world objects (chairs, desks, etc) in images. These objects are referred to as "labels" from images.

Set up your backend

Run amplify add predictions, then use the following answers:

1? Please select from one of the categories below (Use arrow keys)
2  `Identify`
3? What would you like to identify? (Use arrow keys)
4  `Identify Labels`
5? Provide a friendly name for your resource
6  `labelObjects`
7? Would you like use the default configuration? (Use arrow keys)
8  `Default Configuration`
9? Who should have access? (Use arrow keys)
10  `Auth and Guest users`

The Advanced Configuration will allow you to select moderation for unsafe content or all of the identified labels. Default uses both.

Run amplify push to create the resources in the cloud.

Working with the API

Label objects in an image

You can identify real world objects such as chairs, desks, etc. which are referred to as “labels” by passing in LabelType.LABELS as the identify action. For example:

1public void detectLabels(Bitmap image) {
2  Amplify.Predictions.identify(
3    LabelType.LABELS,
4    image,
5    result -> {
6      IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
7      Label label = identifyResult.getLabels().get(0);
8      Log.i("MyAmplifyApp", label.getName());
9    },
10    error -> Log.e("MyAmplifyApp", "Label detection failed", error)
11  );
12}

1fun detectLabels(image: Bitmap) {
2  Amplify.Predictions.identify(LabelType.LABELS, image,
3    { result ->
4      val identifyResult = result as IdentifyLabelsResult
5      val label = identifyResult.labels.firstOrNull()
6      Log.i("MyAmplifyApp", "${label?.name}")
7    },
8    { Log.e("MyAmplifyApp", "Label detection failed", it) }
9  )
10}

1suspend fun detectLabels(image: Bitmap) {
2  try {
3    val result = Amplify.Predictions.identify(LABELS, image)
4    val identifyResult = result as IdentifyLabelsResult
5    Log.i("MyAmplifyApp", "${identifyResult.labels[0].name}")
6  } catch (error: PredictionsException) {
7    Log.e("MyAmplifyApp", "Label detection failed", error)
8  }
9}

1public void detectLabels(Bitmap image) {
2  RxAmplify.Predictions.identify(LabelType.LABELS, image)
3    .subscribe(
4      result -> {
5        IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
6        Label label = identifyResult.getLabels().get(0);
7        Log.i("MyAmplifyApp", label.getName());
8      },
9      error -> Log.e("MyAmplifyApp", "Label detection failed", error)
10    );
11}

Label moderation tag in an image

You can also detect whether identified content inside the image is safe by enabling moderation by passing in LabelType.MODERATION_LABELS.

1public void detectLabels(Bitmap image) {
2  Amplify.Predictions.identify(
3    LabelType.MODERATION_LABELS,
4    image,
5    result -> {
6      IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
7      Log.i("MyAmplifyApp", Boolean.toString(identifyResult.isUnsafeContent()));
8    },
9    error -> Log.e("MyAmplifyApp", "Identify failed", error)
10  );
11}

1fun detectLabels(image: Bitmap) {
2  Amplify.Predictions.identify(LabelType.MODERATION_LABELS, image,
3    { result ->
4      val identifyResult = result as IdentifyLabelsResult
5      Log.i("MyAmplifyApp", "${identifyResult.isUnsafeContent}")
6    },
7    { Log.e("MyAmplifyApp", "Identify failed", it) }
8  )
9}

1suspend fun detectLabels(image: Bitmap) {
2  try {
3    val result = Amplify.Predictions.identify(MODERATION_LABELS, image)
4    val identifyResult = result as IdentifyLabelsResult
5    Log.i("MyAmplifyApp", identifyResult.isUnsafeContent)
6  } catch (error: PredictionsException) {
7    Log.e("MyAmplifyApp", "Identify failed", error)
8  }
9}

Identify entities from images

Interpret sentiment