Label objects in an image

Setup the backend

If you haven't already done so, run amplify init inside your project and then amplify add auth (we recommend selecting the default configuration).

Run amplify add predictions and select Identify. Then use the following answers:

? What would you like to identify? 
  Identify Text 
  Identify Entities 
❯ Identify Labels 
  Learn More 
? Would you like use the default configuration? (Use arrow keys)
❯ Default Configuration 
  Advanced Configuration 
? Who should have access? Auth and Guest users

The Advanced Configuration will allow you to select moderation for unsafe content or all of the identified labels. Default uses both.

Now run amplify push which will generate your aws-exports.js and create resources in the cloud. You can now either add this to your backend or skip and add more features to your app.

Services used: Amazon Rekognition

Working with the API

Detect labels, such if an image has a desk or a chair in it

Predictions.identify({
  labels: {
    source: {
      file,
    type: "LABELS"
.then(response => {
  const { labels } = response;
  labels.forEach(object => {
    const { name, boundingBoxes } = object
.catch(err => console.log({ err }));

Detect unsafe content in an image

Predictions.identify({
  labels: {
    source: {
      file,
    type: "UNSAFE"
.then(response => {
  const { unsafe } = response; // boolean 
.catch(err => console.log({ err }));

for both labels and unsafe content

Predictions.identify({
  labels: {
    source: {
      file,
    type: "ALL"
.then(response => {
  const { labels } = response;
  const { unsafe } = response; // boolean 
  labels.forEach(object => {
    const { name, boundingBoxes } = object
.catch(err => console.log({ err });