Label objects in an image

Amplify Android v1 is deprecated as of June 1st, 2024. No new features or bug fixes will be added. Dependencies may become outdated and potentially introduce compatibility issues.

Please use the latest version (v2) of Amplify Library for Android to get started. Refer to the upgrade guide for instructions on upgrading your application to the latest version.

Amplify libraries should be used for all new cloud connected applications. If you are currently using the AWS Mobile SDK for Android, you can access the documentation here.

The following APIs will enable you identify real world objects (chairs, desks, etc) in images. These objects are referred to as "labels" from images.

Set up your backend

Run amplify add predictions, then use the following answers:

? Please select from one of the categories below (Use arrow keys)
  `Identify`
? What would you like to identify? (Use arrow keys)
  `Identify Labels`
? Provide a friendly name for your resource
  `labelObjects`
? Would you like use the default configuration? (Use arrow keys)
  `Default Configuration`
? Who should have access? (Use arrow keys)
  `Auth and Guest users`

The Advanced Configuration will allow you to select moderation for unsafe content or all of the identified labels. Default uses both.

Run amplify push to create the resources in the cloud.

Working with the API

Label objects in an image

You can identify real world objects such as chairs, desks, etc. which are referred to as “labels” by passing in LabelType.LABELS as the identify action. For example:

public void detectLabels(Bitmap image) {
    Amplify.Predictions.identify(
            LabelType.LABELS,
            image,
            result -> {
                IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
                Label label = identifyResult.getLabels().get(0);
                Log.i("MyAmplifyApp", label.getName());
            error -> Log.e("MyAmplifyApp", "Label detection failed", error)

fun detectLabels(image: Bitmap) {
    Amplify.Predictions.identify(LabelType.LABELS, image,
        { result ->
            val identifyResult = result as IdentifyLabelsResult
            val label = identifyResult.labels.firstOrNull()
            Log.i("MyAmplifyApp", "${label?.name}")
        { Log.e("MyAmplifyApp", "Label detection failed", it) }

suspend fun detectLabels(image: Bitmap) {
        val result = Amplify.Predictions.identify(LABELS, image)
        val identifyResult = result as IdentifyLabelsResult
        Log.i("MyAmplifyApp", "${identifyResult.labels[0].name}")
    } catch (error: PredictionsException) {
        Log.e("MyAmplifyApp", "Label detection failed", error)

public void detectLabels(Bitmap image) {
    RxAmplify.Predictions.identify(LabelType.LABELS, image)
            .subscribe(
                  result -> {
                      IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
                      Label label = identifyResult.getLabels().get(0);
                      Log.i("MyAmplifyApp", label.getName());
                  error -> Log.e("MyAmplifyApp", "Label detection failed", error)

Label moderation tag in an image

You can also detect whether identified content inside the image is safe by enabling moderation by passing in LabelType.MODERATION_LABELS.

public void detectLabels(Bitmap image) {
    Amplify.Predictions.identify(
            LabelType.MODERATION_LABELS,
            image,
            result -> {
                IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
                Log.i("MyAmplifyApp", Boolean.toString(identifyResult.isUnsafeContent()));
            error -> Log.e("MyAmplifyApp", "Identify failed", error)

fun detectLabels(image: Bitmap) {
    Amplify.Predictions.identify(LabelType.MODERATION_LABELS, image,
        { result ->
            val identifyResult = result as IdentifyLabelsResult
            Log.i("MyAmplifyApp", "${identifyResult.isUnsafeContent}")
        { Log.e("MyAmplifyApp", "Identify failed", it) }

suspend fun detectLabels(image: Bitmap) {
        val result = Amplify.Predictions.identify(MODERATION_LABELS, image)
        val identifyResult = result as IdentifyLabelsResult
        Log.i("MyAmplifyApp", identifyResult.isUnsafeContent)
    } catch (error: PredictionsException) {
        Log.e("MyAmplifyApp", "Identify failed", error)