This post shows how to deploy a multimodal WhatsApp ordering assistant built with Amazon Bedrock AgentCore and Amazon Nova 2. Many quick-service restaurants spread ordering across an app, a website, a phone line, and the counter. Each of those is a separate system to build and run. Each one also fragments the customer’s history, making the same person look like a stranger on every channel. Customers already live in their messaging apps. WhatsApp reaches more than two billion people. A customer who can text, send a voice note, or place a call from the same conversation does not need to install anything or sign in.
A single WhatsApp Business number hosts the assistant. A customer can text the restaurant, send a voice note, or place a voice call. An AI agent takes the order end to end, from greeting to confirmation. All three channels share one backend and one cross-channel memory. A customer who texts today and calls tomorrow is recognized as the same person.
The solution uses the Meta WhatsApp Business Platform as the customer front door. Amazon Bedrock AgentCore hosts the agents. Amazon Nova 2 Lite handles text through the Amazon Bedrock Converse API, and Amazon Nova 2 Sonic handles real-time speech on voice notes and calls. The agents reach the restaurant backend through the Model Context Protocol (MCP). You deploy the whole system with the AWS Cloud Development Kit (AWS CDK). The channel and the ordering logic stay separate, so the backend doesn’t change when you add or remove a channel.
Solution overview
The design keeps three things apart: (1) the WhatsApp layer handles the conversation, (2) three agent runtimes run the conversations for their channels, and (3) the backend holds the menu, carts, orders, and locations. Inbound traffic arrives on a single HTTPS webhook, is acknowledged with a 200 immediately, and then processed asynchronously so no request blocks the response. This separation keeps each layer independently deployable and straightforward to reason about.
The customer front door is the Meta WhatsApp Business Platform. It exposes the Cloud API webhook, Messages API, Media API, and Calling API. Meta manages this service. You set it up and connect it as a prerequisite, not something this solution deploys. The AWS CDK provisions everything on the AWS side. It emits the webhook URL you register with Meta.
You use the AWS CDK to deploy the following AWS resources as a set of stacks in dependency order, grouped here by function.
- Amazon API Gateway provides two REST APIs. The first is a regional HTTPS webhook on an AWS managed certificate. It’s the only public endpoint. The second is an AWS Identity and Access Management (IAM)-authorized backend API in front of the ordering logic.
- AWS Lambda runs the webhook ingest, the webhook worker, the message sender, and the ordering business logic.
- Amazon Simple Queue Service (Amazon SQS) provides the inbound queue, with a dead-letter queue, that decouples the quick acknowledgment from the rest of the processing.
- AgentCore runtime, a capability of Amazon Bedrock AgentCore, hosts the three agents. Each conversation runs in its own microVM. Sessions stay isolated.
- Amazon Nova 2 Lite (text through the Converse API) and Amazon Nova 2 Sonic (speech-to-speech for voice) are invoked through Amazon Bedrock.
- AgentCore Gateway, a capability of Amazon Bedrock AgentCore, is a managed MCP server that exposes the backend REST API as MCP tools the agents call by name.
- AgentCore memory, a capability of Amazon Bedrock AgentCore, is one shared, cross-channel record keyed by a hashed customer ID.
- Amazon DynamoDB stores customer profiles, orders, menu items, carts, and locations. It also holds a last inbound window table for the WhatsApp layer.
- Amazon Location Service handles geocoding and nearest-location lookups.
- Amazon Kinesis Video Streams (Amazon KVS) provides a signaling channel that the voice-call runtime uses to mint credentials for the managed Traversal Using Relays around NAT (TURN) relay that carries call media.
- Amazon Virtual Private Cloud (Amazon VPC) with a single network address translation (NAT) gateway is the outbound path for the voice-call runtime. This is the only runtime that requires a VPC.
- AWS Secrets Manager holds the Meta Access Token, App Secret, and Verify Token, created as empty containers you populate out-of-band. AWS Systems Manager Parameter Store holds the customer-id pepper.
- Amazon Elastic Container Registry (Amazon ECR), AWS CodeBuild, and Amazon Simple Storage Service (Amazon S3) build and store the ARM64 agent container images.
- Amazon CloudWatch captures logs and metrics, and AWS Key Management Service (AWS KMS) encrypts data at rest.
Architecture diagram
Figure 1 shows the full architecture. The diagram organizes the solution into labeled groups A through G that carry the request path. Two supporting groups sit outside that path, handling a build pipeline that runs once at deploy time to build the agent images and security and monitoring services that back every channel at runtime.
Figure 1: Multimodal WhatsApp ordering architecture on AWS
- A. WhatsApp ingress and delivery: The webhook API Gateway, the ingest, worker, and sender Lambdas, and the Amazon SQS queue. The ingest verifies the Meta signature and enqueues the message. The worker handles the remaining processing. The Sender Lambda delivers replies.
- B. Agent runtimes: Three agents on AgentCore runtime, each an ARM64 container, one per channel: chat (Amazon Nova 2 Lite), voice note (Amazon Nova 2 Sonic), and voice call (Amazon Nova 2 Sonic over Web Real-Time Communication, or WebRTC). Only the voice-call runtime runs in a VPC.
- C. Artificial intelligence and machine learning (AI/ML): Amazon Nova 2 Lite and Amazon Nova 2 Sonic invoked through Amazon Bedrock, plus the shared AgentCore memory keyed by a hashed
customer_idfor cross-channel continuity. - D. MCP server (managed): AgentCore Gateway exposes the backend REST API as discoverable MCP tools (
GetMenu,AddToCart,PlaceOrder, and more) that every runtime calls by name. - E. API and compute: The IAM-authorized backend API Gateway and the ordering Lambdas that hold the business logic.
- F. Data storage: Amazon DynamoDB stores customer profiles, orders, menu items, carts, and locations.
- G. Networking and location: Amazon Location Service for geocoding and nearest-location lookups, the Amazon KVS managed TURN relay for voice-call media, and the VPC with a NAT gateway as the outbound path for the voice-call runtime.
Two supporting groups sit outside the request path. The build pipeline (AWS CDK, AWS CodeBuild, Amazon ECR, and Amazon S3) runs once at deploy time to build and store the ARM64 agent images. It is not in the path of any request. Security and monitoring (AWS Secrets Manager, AWS Systems Manager Parameter Store, Amazon CloudWatch, and AWS KMS) back every channel at runtime.
The following steps trace a single request end to end through the architecture:
- Meta delivers an inbound webhook (text, voice note, or call event) to the Webhook API Gateway and the Webhook Ingest Lambda.
- The ingest verifies the Meta signature, enqueues to Amazon SQS, and returns
200within Meta’s window. - The Webhook Worker derives a pseudonymous
customer_idusing the pepper in AWS Systems Manager Parameter Store. - The worker fetches media from the Meta Media API and invokes the matching agent on Amazon Bedrock AgentCore runtime (chat, voice note, or voice call) with
session_id = customer_id. - The runtime reads the customer’s long-term insights from AgentCore memory at session start.
- It runs the conversation with Amazon Nova 2 Lite (text) or Amazon Nova 2 Sonic (voice) through Amazon Bedrock.
- AgentCore Gateway is the managed MCP server. It exposes the backend REST API as MCP tools the agent calls by name.
- Tools route through the backend API Gateway to AWS Lambda, Amazon DynamoDB, and Amazon Location Service. Voice-call media uses the Amazon KVS TURN relay (runtime in a VPC).
- Replies go out through the Sender Lambda (text) or the worker (voice). Events are written back to memory at session end.
- AWS CDK builds ARM64 images through AWS CodeBuild into Amazon ECR. Amazon CloudWatch logs components and AWS KMS encrypts data at rest.
In short, every request flows from Meta’s webhook through the ingest, queue, worker, and agent runtime to the backend tools, then back to the customer on WhatsApp.
Channel flows
All three channels share the same front door, backend tools, and memory. What differs is the media on the wire and the runtime that handles it.
Text message: A text message arrives on the webhook. The worker derives customer_id and invokes the chat runtime, which reads memory, streams Amazon Nova 2 Lite through the Converse API, and calls backend tools through the MCP gateway as needed. Replies go through the Sender Lambda, and events are written to memory at session end.
Figure 2 shows the text flow, from the inbound webhook through Amazon Nova 2 Lite on the Converse API to the reply delivered by the Sender Lambda.
Figure 2: Text flow with Amazon Nova 2 Lite
Voice note (speech-to-speech): A voice note arrives as an audio message. The worker downloads the OGG Opus bytes and invokes the voice-note runtime. After reading memory, the audio is decoded to 16 kHz pulse-code modulation (PCM) and fed into a bounded Amazon Nova 2 Sonic speech-to-speech session. Tools are available through the same gateway. The spoken reply is returned as a WhatsApp voice message. There’s no transcription service in the path. It’s true voice-in, voice-out.
Figure 3 shows the voice-note flow, a bounded Amazon Nova 2 Sonic speech-to-speech session that returns a spoken reply with no transcription in the path.
Figure 3: Voice note flow with Amazon Nova 2 Sonic
Voice call (WebRTC): The customer selects Call, and Meta’s Calling API delivers a connect webhook with the WebRTC Session Description Protocol (SDP) offer. The worker relays it to the voice-call runtime in turnOnly mode, because it has no public IP. TURN credentials come from Amazon KVS. Meta provides no trickle Interactive Connectivity Establishment (ICE) path. The aiortc answerer waits for ICE gathering and returns a single-shot SDP answer. The worker delivers that answer to Meta. Media then flows over Datagram Transport Layer Security and Secure Real-time Transport Protocol (DTLS/SRTP) through the KVS managed TURN relay. Amazon Nova 2 Sonic drives the conversation.
Figure 4 shows the voice-call flow, where WebRTC media is relayed through the Amazon KVS managed TURN relay and Amazon Nova 2 Sonic drives the conversation.
Figure 4: Voice call flow with Amazon Nova 2 Sonic
Prerequisites
This solution has prerequisites in two areas: your AWS account and your Meta WhatsApp setup. Complete both prior to running the deployment, because it asks you for specific WhatsApp values and the agent cannot reply until they are in place.
AWS prerequisites
- You need an active AWS account with Amazon Bedrock model access enabled for Amazon Nova 2 Lite (
amazon.nova-2-lite-v1:0) and Amazon Nova 2 Sonic (amazon.nova-2-sonic-v1:0) in your deployment Region. Your IAM user or role must have permissions to deploy AWS CDK stacks and create the resources this solution uses, including AgentCore runtime, Gateway, and memory. On your local machine, installNode.js24.x or later, the AWS CLI 2.x configured with credentials, and git. Finally, bootstrap AWS CDK in your target account and Region (npx cdk bootstrap aws://<ACCOUNT_ID>/<REGION>).
The agent containers build inside AWS CodeBuild on ARM64, so you do not need Python, Docker, or audio toolchains locally. Deploy in an AWS Region where Amazon Nova 2 Lite, Amazon Nova 2 Sonic, and AgentCore runtime, Gateway, and memory are all available. The US East (N. Virginia) Region (us-east-1) is a good place to start.
For model availability by Region, refer to Supported models by AWS Region in Amazon Bedrock.
WhatsApp Business Platform (Meta) prerequisites
The WhatsApp side is set up once in the Meta consoles, as a prerequisite rather than a step in this walkthrough. No AWS API can create a Meta app for you, so complete it first and have the values on hand before you deploy. For a demo, the Meta sandbox test number, which is available at no additional cost, is enough. You do not need business verification or a production number. The full procedure lives in Meta’s documentation, linked in the following section.
Have these items ready before you deploy.
- A Meta Developer App with the WhatsApp product added, linked to a Business portfolio. Adding the product provisions a WhatsApp Business Account (WABA) and a sandbox test number at no additional cost. Refer to Get started with the Cloud API.
- The App ID and App Secret. The App Secret is the key Meta uses to sign each webhook, and the webhook Lambda recomputes that signature.
- An Access Token. The temporary token works for a quick test and expires in about 24 hours. For something longer-lived, create a System User token with the
whatsapp_business_messagingandwhatsapp_business_managementscopes. - The Business portfolio ID (the setup CLI can auto-discover the Phone Number ID and WABA ID).
- A Verify Token that you invent. It is a hard-to-guess string Meta echoes back during the one-time webhook verification handshake.
- For voice calls, the WhatsApp Calling API enabled on the number.
Store none of these in source control. The Access Token, App Secret, and Verify Token are secrets and go into AWS Secrets Manager during deployment, not into the CDK templates.
Deploy the solution with AWS CDK
The full solution is in the sample repository on GitHub. The repository contains