Generation
AI generation routes are a request-response API used to generate structured output from AI models. Examples of generation routes include:
- generated structured data from unstructured input
- summarization
Under the hood, a generation route is an AWS AppSync query that ensures the AI model responds with the response type defined for the route.
Generate Typed Objects
const schema = a.schema({ generateRecipe: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'You are a helpful assistant that generates recipes.', }) .arguments({ description: a.string() }) .returns( a.customType({ name: a.string(), ingredients: a.string().array(), instructions: a.string(), }) ) .authorization((allow) => allow.authenticated())});Generate Scalar Types
const schema = ({ summarize: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'Provide an accurate, clear, and concise summary of the input provided' }) .arguments({ input: a.string() }) .returns(a.string()) .authorization((allow) => allow.guest()),});const { data: summary, errors } = await client.generations .summarize({ input })Setting Inference Parameters
You can influence response generation by setting inference parameters for the AI model. This ability to control the randomness and diversity of responses is useful for generating responses that are tailored to your needs.
More information about inference parameters.
const schema = a.schema({ generateHaiku: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'You are a helpful assistant that generates haikus.', inferenceConfiguration: { maxTokens: 1000, temperature: 0.5, topP: 0.9, } }),});Limitations
1. Generation routes do not support referencing models
For example, the following schema defines a Recipe model, but this model cannot be used as the return type of a generation route.
const schema = a.schema({ Recipe: a.model({ name: a.string(), ingredients: a.string().array(), instructions: a.string(), }), generateRecipe: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'You are a helpful assistant that generates recipes.', }) .arguments({ description: a.string() }) .returns(a.ref('Recipe')) // ❌ Invalid .authorization((allow) => allow.authenticated()),});You can, however, reference custom types. Here's an example of a custom type that can be used as the return type of a generation route.
const schema = a.schema({ Recipe: a.customType({ name: a.string(), ingredients: a.string().array(), instructions: a.string(), }), generateRecipe: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'You are a helpful assistant that generates recipes.', }) .arguments({ description: a.string() }) .returns(a.ref('Recipe')) // ✅ Valid .authorization((allow) => allow.authenticated()),});2. Generation routes do not support some required types.
The following AppSync scalar types are not supported as required fields in response types:
AWSEmailAWSDateAWSTimeAWSDateTimeAWSTimestampAWSPhoneAWSURLAWSIPAddress
const schema = a.schema({ generateUser: a.generation({ aiModel: a.ai.model('Claude 3 Haiku'), systemPrompt: 'You are a helpful assistant that generates users.', }) .arguments({ description: a.string() }) .returns( a.customType({ name: a.string(), email: a.email().required(), // ❌ Required field with unsupported type dateOfBirth: a.date().required(), // ❌ Required field with unsupported type }) ) .authorization((allow) => allow.authenticated()),});