Set up Predictions

Prerequisite: Install and configure the Amplify CLI

Create new backend

Run the following command in your project's root folder:

1amplify add predictions

The CLI will prompt configuration options for the Predictions category such as what type of use case you have (identifying objects from an image, translating text, etc) and default or advanced settings.

The Predictions category utilizes the Auth category behind the scenes to authorize your app to perform AI/ML actions.

The add command automatically creates the backend configuration. Once all your configuration is complete run the following:

1amplify push

A configuration file called amplifyconfiguration.json will be copied to your configured source directory, for example ./src.

NOTE: if using Identify Labels or Identify Text, you will need to manually add configuration items to your amplify backend folder due to this known issue in identifyText and this known issue in identifyLabels.

Identify Text

amplify/backend/identify-text-resource/parameters.json

1"identifyDoc": true,
2"access": "auth | authAndGuest",
3"format": "PLAIN | FORM | TABLE | ALL"

Identify Labels

amplify/backend/identify-labels-resource/parameters.json

1"access": "auth | authAndGuest",
2"type": "LABELS | UNSAFE | ALL"

Install Amplify Libraries

To install the Amplify library to use predictions features, run the following commands in your project’s root folder:

1npm install aws-amplify @aws-amplify/predictions

Make sure that the @aws-amplify/predictions package has the same version number as the aws-amplify package in your package.json file.

Configure the frontend

Import and load the configuration file in your app. It's recommended you add the Amplify configuration step to your app's root entry point. For example index.js in React or main.ts in Angular.

1import { Amplify } from 'aws-amplify';
2import amplifyconfig from './amplifyconfiguration.json';
3
4Amplify.configure(amplifyconfig);

Make sure you call Amplify.configure as early as possible in your application’s life-cycle. A missing configuration or NoCredentials error is thrown if Amplify.configure has not been called before other Amplify JavaScript APIs. Review the Library Not Configured Troubleshooting guide for possible causes of this issue.

Import existing backend

The manual setup enables you to use your existing Amazon AI and ML resources in your app.

1import { Amplify } from 'aws-amplify';
2
3Amplify.configure({
4  // To get the AWS Credentials, you need to configure
5  // the Auth module with your Cognito Federated Identity Pool
6  Auth: {
7    identityPoolId: 'us-east-1:xxx-xxx-xxx-xxx-xxx',
8    region: 'us-east-1'
9  },
10  Predictions: {
11    convert: {
12      translateText: {
13        region: 'us-east-1',
14        proxy: false,
15        defaults: {
16          sourceLanguage: 'en',
17          targetLanguage: 'zh'
18        }
19      },
20      speechGenerator: {
21        region: 'us-east-1',
22        proxy: false,
23        defaults: {
24          VoiceId: 'Ivy',
25          LanguageCode: 'en-US'
26        }
27      },
28      transcription: {
29        region: 'us-east-1',
30        proxy: false,
31        defaults: {
32          language: 'en-US'
33        }
34      }
35    },
36    identify: {
37      identifyText: {
38        proxy: false,
39        region: 'us-east-1',
40        defaults: {
41          format: 'PLAIN'
42        }
43      },
44      identifyEntities: {
45        proxy: false,
46        region: 'us-east-1',
47        celebrityDetectionEnabled: true,
48        defaults: {
49          collectionId: 'identifyEntities8b89c648-test',
50          maxEntities: 50
51        }
52      },
53      identifyLabels: {
54        proxy: false,
55        region: 'us-east-1',
56        defaults: {
57          type: 'LABELS'
58        }
59      }
60    },
61    interpret: {
62      interpretText: {
63        region: 'us-east-1',
64        proxy: false,
65        defaults: {
66          type: 'ALL'
67        }
68      }
69    }
70  }
71});

IAM Policy

The Amplify CLI will set appropriate IAM policy for Roles in your Cognito Identity Pool in order to use an appropriate feature. If you are using the library manually you will need to configure this yourself. The below policy demonstrates setting policy for all services in the Predictions category:

1{
2  "Version": "2012-10-17",
3  "Statement": [
4    {
5      "Effect": "Allow",
6      "Action": [
7        "translate:TranslateText",
8        "polly:SynthesizeSpeech",
9        "transcribe:StartStreamTranscriptionWebSocket",
10        "comprehend:DetectSentiment",
11        "comprehend:DetectEntities",
12        "comprehend:DetectDominantLanguage",
13        "comprehend:DetectSyntax",
14        "comprehend:DetectKeyPhrases",
15        "rekognition:DetectFaces",
16        "rekognition:RecognizeCelebrities",
17        "rekognition:DetectLabels",
18        "rekognition:DetectModerationLabels",
19        "rekognition:DetectText",
20        "rekognition:DetectLabel",
21        "textract:AnalyzeDocument",
22        "textract:DetectDocumentText",
23        "textract:GetDocumentAnalysis",
24        "textract:StartDocumentAnalysis",
25        "textract:StartDocumentTextDetection",
26        "rekognition:SearchFacesByImage"
27      ],
28      "Resource": ["*"]
29    }
30  ]
31}

For rekognition:SearchFacesByImage you can scope the Resource down to an individual collection such as arn:aws:rekognition:<REGION>:<ACCOUNT_ID>:collection/<COLLECTION_ID>. Amplify CLI automatically does this.

Text-to-speech