Label objects in an image

Amplify Android v1 is now in Maintenance Mode until May 31st, 2024. This means that we will continue to include updates to ensure compatibility with backend services and security. No new features will be introduced in v1.

Please use the latest version (v2) of Amplify Library for Android to get started.

If you are currently using v1, follow these instructions to upgrade to v2.

Amplify libraries should be used for all new cloud connected applications. If you are currently using the AWS Mobile SDK for Android, you can access the documentation here.

The following APIs will enable you identify real world objects (chairs, desks, etc) in images. These objects are referred to as "labels" from images.

Set up your backend

Run amplify add predictions, then use the following answers:

1? Please select from one of the categories below (Use arrow keys)
2  `Identify`
3? What would you like to identify? (Use arrow keys)
4  `Identify Labels`
5? Provide a friendly name for your resource
6  `labelObjects`
7? Would you like use the default configuration? (Use arrow keys)
8  `Default Configuration`
9? Who should have access? (Use arrow keys)
10  `Auth and Guest users`

The Advanced Configuration will allow you to select moderation for unsafe content or all of the identified labels. Default uses both.

Run amplify push to create the resources in the cloud.

Working with the API

Label objects in an image

You can identify real world objects such as chairs, desks, etc. which are referred to as “labels” by passing in LabelType.LABELS as the identify action. For example:

1public void detectLabels(Bitmap image) {
2    Amplify.Predictions.identify(
3            LabelType.LABELS,
4            image,
5            result -> {
6                IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
7                Label label = identifyResult.getLabels().get(0);
8                Log.i("MyAmplifyApp", label.getName());
9            },
10            error -> Log.e("MyAmplifyApp", "Label detection failed", error)
11    );
12}

1fun detectLabels(image: Bitmap) {
2    Amplify.Predictions.identify(LabelType.LABELS, image,
3        { result ->
4            val identifyResult = result as IdentifyLabelsResult
5            val label = identifyResult.labels.firstOrNull()
6            Log.i("MyAmplifyApp", "${label?.name}")
7        },
8        { Log.e("MyAmplifyApp", "Label detection failed", it) }
9    )
10}

1suspend fun detectLabels(image: Bitmap) {
2    try {
3        val result = Amplify.Predictions.identify(LABELS, image)
4        val identifyResult = result as IdentifyLabelsResult
5        Log.i("MyAmplifyApp", "${identifyResult.labels[0].name}")
6    } catch (error: PredictionsException) {
7        Log.e("MyAmplifyApp", "Label detection failed", error)
8    }
9}

1public void detectLabels(Bitmap image) {
2    RxAmplify.Predictions.identify(LabelType.LABELS, image)
3            .subscribe(
4                  result -> {
5                      IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
6                      Label label = identifyResult.getLabels().get(0);
7                      Log.i("MyAmplifyApp", label.getName());
8                  },
9                  error -> Log.e("MyAmplifyApp", "Label detection failed", error)
10            );
11}

Label moderation tag in an image

You can also detect whether identified content inside the image is safe by enabling moderation by passing in LabelType.MODERATION_LABELS.

1public void detectLabels(Bitmap image) {
2    Amplify.Predictions.identify(
3            LabelType.MODERATION_LABELS,
4            image,
5            result -> {
6                IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
7                Log.i("MyAmplifyApp", Boolean.toString(identifyResult.isUnsafeContent()));
8            },
9            error -> Log.e("MyAmplifyApp", "Identify failed", error)
10    );
11}

1fun detectLabels(image: Bitmap) {
2    Amplify.Predictions.identify(LabelType.MODERATION_LABELS, image,
3        { result ->
4            val identifyResult = result as IdentifyLabelsResult
5            Log.i("MyAmplifyApp", "${identifyResult.isUnsafeContent}")
6        },
7        { Log.e("MyAmplifyApp", "Identify failed", it) }
8    )
9}

1suspend fun detectLabels(image: Bitmap) {
2    try {
3        val result = Amplify.Predictions.identify(MODERATION_LABELS, image)
4        val identifyResult = result as IdentifyLabelsResult
5        Log.i("MyAmplifyApp", identifyResult.isUnsafeContent)
6    } catch (error: PredictionsException) {
7        Log.e("MyAmplifyApp", "Identify failed", error)
8    }
9}

Identify entities from images

Interpret sentiment