Build a generative AI sandbox with Amplify and Amazon Bedrock

Overview

AWS Amplify helps you build and deploy generative AI applications by providing you with the tools needed to build a fullstack application that integrates with Amazon Bedrock. Amazon Bedrock is a fully managed service that provides access to Foundation Models (FMs) so you can build generative AI applications easily and not worry about managing AI infrastructure. Integrating generative AI capabilities into your application or building your own generative AI application requires more than just making calls to a foundation model. To build a fullstack generative AI application, in addition to Amazon Bedrock, you will need:

Authentication
API layer
Hosting
Accessible UI

With Amplify and Amazon Bedrock, you can create a generative AI application easily in under an hour. This guide will walk you through the steps of building a generative AI sandbox.

Project setup

For this we will use React and Next.js to take advantage of the full managed hosting of Next.js server-side rendering (SSR) capabilities in Amplify Hosting. The first step is to create a new Next.js project using yarn, npm, pnpm, or Bun.

1npx create-next-app@latest

1✔ Would you like to use TypeScript with this project? … No / Yes
2✔ Would you like to use ESLint with this project? … No / Yes
3✔ Would you like to use Tailwind CSS with this project? … No / Yes
4✔ Would you like to use `src/` directory with this project? … No / Yes
5✔ Would you like to use experimental `app/` directory with this project? … No / Yes
6✔ What import alias would you like configured? … @/*

For this project we will use the Next.js Pages router. Feel free to change the settings to your liking. Then go into your newly created Next.js app and add Amplify to your project with the Amplify CLI:

1amplify init

1? Enter a name for the project amplify-gen-ai
2The following configuration will be applied:
3
4Project information
5| Name: amplify-gen-ai
6| Environment: dev
7| Default editor: Visual Studio Code
8| App type: javascript
9| Javascript framework: react
10| Source Directory Path: src
11| Distribution Directory Path: build
12| Build Command: npm run-script build
13| Start Command: npm run-script start
14? Initialize the project with the above configuration? Yes

Add authentication

While adding authentication to your app is optional, allowing anyone to use an LLM through your app without authenticating is not recommended in practice. This is because requests to an LLM incur costs based on the amount of text sent to and received from the LLM. Most LLMs, like Amazon Bedrock, also enforce rate limits.

You can add authentication to your application with the Amplify CLI or Amplify Studio. For this, we will use the CLI:

1amplify add auth

The output from the CLI should look like this:

1Using service: Cognito, provided by: awscloudformation
2
3 The current configured provider is Amazon Cognito.
4
5 Do you want to use the default authentication and security configuration? _**Default configuration**_
6 Warning: you will not be able to edit these selections.
7 How do you want users to be able to sign in? _**Email**_
8 Do you want to configure advanced settings? _**No, I am done.**_

Then push your Amplify project to the cloud:

1amplify push

You should see:

1✔ Successfully pulled backend environment dev from the cloud.
2
3    Current Environment: dev
4
5┌──────────┬──────────────────────┬───────────┬───────────────────┐
6│ Category │ Resource name        │ Operation │ Provider plugin   │
7├──────────┼──────────────────────┼───────────┼───────────────────┤
8│ Auth     │ amplifyGenAi         │ Create    │ awscloudformation │
9└──────────┴──────────────────────┴───────────┴───────────────────┘

Configure Amplify

Add the frontend packages for Amplify and Amplify UI:

1npm i --save @aws-amplify/ui-react aws-amplify

Then add these imports to the top of the src/pages/_app.tsx :

1import '@aws-amplify/ui-react/styles.css';
2import { Amplify } from 'aws-amplify';
3import awsconfig from '../aws-exports';

And then after the imports, add this to configure Amplify:

1Amplify.configure({
2  ...awsconfig,
3  // this lets you run Amplify code on the server-side in Next.js
4  ssr: true
5});

Then wrap the <Component /> in JSX with the Authenticator component from '@aws-amplify/ui-react'. Your file should now look like this:

1import '@aws-amplify/ui-react/styles.css';
2import { Amplify } from 'aws-amplify';
3import awsconfig from '../aws-exports';
4import type { AppProps } from 'next/app';
5import { Authenticator } from '@aws-amplify/ui-react';
6
7Amplify.configure({
8  ...awsconfig,
9  // this lets you run Amplify code on the server-side in Next.js
10  ssr: true
11});
12
13export default function App({ Component, pageProps }: AppProps) {
14  return (
15    <Authenticator>
16      <Component {...pageProps} />
17    </Authenticator>
18  );
19}

If you start up your local dev server, you should now see this:

Amplify authenticator login form with email and password fields and sign in button.

Create an account for yourself for testing. Once you log in, you should see the default Next.js starter homepage.

Add an API layer

Add the Amazon Bedrock SDK to your dependencies:

1npm i --save @aws-sdk/client-bedrock-runtime

The default Next.js template with the Pages router should come with an example API route at src/pages/api/hello.ts. Let’s rename that file to chat.ts. The first thing we will need to do in this file is configure Amplify on the server-side. To do that, you will import Amplify and your Amplify config file that Amplify creates for you.

1import { Amplify } from 'aws-amplify';
2import awsconfig from '@/aws-exports';
3
4Amplify.configure({
5  ...awsconfig,
6  ssr: true
7});

Then, to get the Authentication credentials, import withSSRContext from aws-amplify. Change the default handler function to an async function.

1- export default function handler(
2+ export default async function handler(

Then, in the body of the function, create an SSR context:

1const SSR = withSSRContext({ req });

This will give you access to all the Amplify Library functionality on the server-side, like using the Auth module. To get the current credentials of the user, include the following:

1const credentials = await SSR.Auth.currentCredentials();

To test that this is working, you can output the credentials to the console and boot up your local server again. Let’s do that to make sure everything is working so far. In src/pages/index.tsx, create a function called callAPI that does a fetch to our API endpoint:

1const callAPI = () => {
2  fetch('/api/hello');
3};

Then in the JSX of the page, add a button that calls our callAPI function on click:

1<button onClick={callAPI}>Click me</button>

If you are logged in and click the button, your authentication credentials should print to the console in your terminal.

1fetch('/api/hello', {
2  method: 'POST',
3  body: JSON.stringify({ input: 'Who are you?' })
4});

Open src/pages/api/chat.ts and remove that console.log and initialize the Amazon Bedrock client with your user’s credentials:

1const bedrock = new BedrockRuntimeClient({
2  serviceId: 'bedrock',
3  region: 'us-east-1',
4  credentials
5});

Then you can send the Bedrock client the InvokeModel command. Here is the full chat.ts file with comments for what it is doing.

1import {
2  BedrockRuntimeClient,
3  InvokeModelCommand
4} from '@aws-sdk/client-bedrock-runtime';
5import { Amplify, withSSRContext } from 'aws-amplify';
6import type { NextApiRequest, NextApiResponse } from 'next';
7import awsExports from '@/aws-exports';
8
9Amplify.configure({
10  ...awsExports,
11  ssr: true
12});
13
14export default async function handler(
15  req: NextApiRequest,
16  res: NextApiResponse
17) {
18  const body = JSON.parse(req.body);
19  const SSR = withSSRContext({ req });
20  const credentials = await SSR.Auth.currentCredentials();
21  const bedrock = new BedrockRuntimeClient({
22    serviceId: 'bedrock',
23    region: 'us-east-1',
24    credentials
25  });
26
27  // Anthropic's Claude model expects a chat-like string
28  // of 'Human:' and 'Assistant:' responses separated by line breaks.
29  // You should always end your prompt with 'Assistant:' and Claude
30  // will respond. There are various prompt engineering techniques
31  // and frameworks like LangChain you can use here too.
32  const prompt = `Human:${body.input}\n\nAssistant:`;
33
34  const result = await bedrock.send(
35    new InvokeModelCommand({
36      modelId: 'anthropic.claude-v2',
37      contentType: 'application/json',
38      accept: '*/*',
39      body: JSON.stringify({
40        prompt,
41        // LLM costs are measured by Tokens, which are roughly equivalent
42        // to 1 word. This option allows you to set the maximum amount of
43        // tokens to return
44        max_tokens_to_sample: 2000,
45        // Temperature (1-0) is how 'creative' the LLM should be in its response
46        // 1: deterministic, prone to repeating
47        // 0: creative, prone to hallucinations
48        temperature: 1,
49        top_k: 250,
50        top_p: 0.99,
51        // This tells the model when to stop its response. LLMs
52        // generally have a chat-like string of Human and Assistant message
53        // This says stop when the Assistant (Claude) is done and expects
54        // the human to respond
55        stop_sequences: ['\n\nHuman:'],
56        anthropic_version: 'bedrock-2023-05-31'
57      })
58    })
59  );
60  // The response is a Uint8Array of a stringified JSON blob
61  // so you need to first decode the Uint8Array to a string
62  // then parse the string.
63  res.status(200).json(JSON.parse(new TextDecoder().decode(result.body)));
64}

Then update the callAPI function in your frontend to accept a JSON response:

1fetch('/api/hello', {
2  method: 'POST',
3  body: JSON.stringify({ input: 'Who are you?' })
4})
5  .then((res) => res.json())
6  .then((data) => {
7    console.log(data);
8  });

But if you click that button, you will see this in your terminal:

1⨯ AccessDeniedException: User: arn:aws:sts::553879240338:assumed-role/amplify-amplifyGenAi-dev-205330-authRole/CognitoIdentityCredentials is not authorized to perform: bedrock:InvokeModel on resource:

When you create an Authentication backend with Amplify, it sets up Amazon Cognito and creates two IAM roles, one for unauthenticated users and another for authenticated users. These IAM roles let AWS know what resources your authenticated and guest users can access. We didn’t give our authenticated users access to Amazon Bedrock yet. Let’s do that now.

You can update the IAM policies Amplify creates with an override. Run this command:

1amplify override project

Edit the override.ts file it creates and add a policy to allow the authenticated role access to Amazon Bedrock’s InvokeModel command like this:

1import {
2  AmplifyProjectInfo,
3  AmplifyRootStackTemplate
4} from '@aws-amplify/cli-extensibility-helper';
5
6export function override(
7  resources: AmplifyRootStackTemplate,
8  amplifyProjectInfo: AmplifyProjectInfo
9) {
10  const authRole = resources.authRole;
11
12  const basePolicies = Array.isArray(authRole.policies)
13    ? authRole.policies
14    : [authRole.policies];
15
16  authRole.policies = [
17    ...basePolicies,
18    {
19      policyName: '',
20      policyDocument: {
21        Version: '2012-10-17',
22        Statement: [
23          {
24            Effect: 'Allow',
25            Action: 'bedrock:InvokeModel',
26            Resource:
27              'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-v2'
28          }
29        ]
30      }
31    }
32  ];
33}

Run amplify push to sync your changes with the cloud. Now restart your local Next.js server and try to call Amazon Bedrock again through your API route.

Object with text completion: "I'm Claude, an AI assistant created by Anthropic"

Hosting

The last thing to do is to add hosting to our Next.js application. Amplify has you covered there too. Create a new Git repository in GitHub, Bitbucket, or GitLab. Commit your code and push it to the git provider of your choice. Let’s use GitHub as an example. If you have the GitHub CLI installed, you can run:

1gh repo create

1? What would you like to do? Push an existing local repository to GitHub
2? Path to local repository .
3? Repository name amplify-gen-ai
4? Repository owner danny
5? Description
6? Visibility Private
7✓ Created repository danny/amplify-gen-ai on GitHub
8? Add a remote? Yes
9? What should the new remote be called? origin
10✓ Added remote git@github.com:danny/amplify-gen-ai.git
11? Would you like to push commits from the current branch to "origin"? Yes

Now we can add Amplify Hosting from the Amplify CLI:

1amplify add hosting

Amplify Build Settings form

1✔ Select the plugin module to execute · Hosting with Amplify Console (Managed hosting with custom domains, Continuous deployment)
2
3? Choose a type Continuous deployment (Git-based deployments)
4? Continuous deployment is configured in the Amplify Console. Please hit enter once you connect your repository
5Amplify hosting urls:
6┌──────────────┬───────────────────────────────────────────┐
7│ FrontEnd Env │ Domain                                    │
8├──────────────┼───────────────────────────────────────────┤
9│ main         │ https://main.dade6up5cdfhu.amplifyapp.com │

Now you have a hosted fullstack application using Amplify connected to Amazon Bedrock.

If you are using TypeScript, you will need to ignore the Amplify directory in the tsconfig file at the root of your project or you might get a type error when Amplify tries to build your Next.js application. Update the "exclude" array in your tsconfig to have "amplify":

1"exclude": ["node_modules","amplify"]

Test the REST API

Use existing AWS resources