Armakuni Contact us
← All case studies

A new oven reached customers.
Its recipes hadn't.

A U.S. smart kitchen brand launched a countertop oven with no guided recipes in its app, and thousands of cooking videos it had no fast way to use. We built an AI agent that turns a video link into a structured recipe for the content team to review, and showed that a well-prompted model could predict when dinner will be done, without training a custom model.

Before

The library couldn't keep up with the product line.

The recipes already existed, on video.

After
Minutesper recipe draft, where it took hours, in the prototype
  • A finish-time prediction without a custom model, from a proof of concept of about two weeks.

The library couldn't keep up with the product line

The brand sells connected appliances, from smart thermometers to multicookers, and its mobile app is part of the product: guided recipes, video walkthroughs and instructions for the equipment in your kitchen.

Then it launched a countertop oven, and the app had no recipes for it. Every recipe for the new category had to be made from scratch. Someone watched the source material, wrote down the ingredients, wrote out the method and formatted it for the app, and then did it again for dietary preferences, different portions and other equipment.

The recipes already existed, on video

The frustrating part was that the content was out there. The team had access to cooking videos on YouTube and Instagram covering every technique the oven needed. Turning them into usable recipes was still a manual job, one video at a time, and weeks of editorial effort for each batch.

And the thermometer guessed

The brand had a second problem in the same app. Its smart thermometers record ambient temperature, the food's internal temperature and elapsed time every 15 seconds, but the finish-time estimate didn't adapt as conditions changed. Customers wanted to know when dinner would be ready, and the app was often wrong. The company had looked at training a custom machine-learning model and decided the cost and the development resources made it impractical.

How we built it

5 decisions, in the order we made them. Each one says what we did and why, with what it did drawn beside it.

  1. Three tools behind one AI agent
  2. From a video link to a recipe the app can use
  3. Changes that remember the last change
  4. A prediction from prompts, not from a trained model
  5. How early can the estimate be trusted?

Three tools behind one AI agent

Built withStreamlitBedrockAgentCoreAPI GatewayLambdaTranscribe
One AI agent Video toolTranscript toolRecipe tool
The engineering

The quick build is one big prompt that takes a video and returns a recipe. We split the work into separate tools instead, because fetching a video, reading a transcript and rewriting a recipe fail in different ways and need to be tested separately.

The content team submits requests through Streamlit. We built the AI agent on Amazon Bedrock AgentCore and LangGraph, with separate tools for video processing, recipe extraction and recipe customization:

  • Amazon API Gateway is the request entry point for invoking the AI agent.
  • AgentCore Runtime hosts the AI agent, with LangGraph coordinating tool selection, execution and response generation.
  • AgentCore Gateway exposes the three tools through Lambda-backed implementations.
  • The tools use Amazon Bedrock, Amazon Transcribe, Amazon SQS and Amazon S3 for model calls, transcription, queued processing and storage. Intermediate audio, images and documents are deleted from S3 after 24 hours, so storage cost follows the work being done instead of piling up.
  • AWS Secrets Manager handles secrets, with Amazon CloudWatch and AWS CloudTrail in the monitoring and audit design.

With each tool behind AgentCore Gateway, the AI agent can only do what a tool allows, and each tool can change without touching the others.

From a video link to a recipe the app can use

Built withTranscribeSQSBedrock
A video linkTranscribeBedrock extracts the recipeschema.org Recipe
The engineering

The app needs ingredients, ordered instructions, cooking times and equipment compatibility in one consistent recipe structure, so the output had to fit that structure exactly:

  • A team member submits a YouTube or Instagram URL. The video-processing tool retrieves the video and extracts its audio.
  • Amazon Transcribe turns the audio into a timestamped transcript.
  • Amazon SQS queues the long-running processing, so transcription never blocks the interface.
  • The recipe-extraction tool passes the transcript to a Bedrock foundation model, with prompts designed to pick out ingredients, instructions, cooking times and equipment compatibility.
  • The model returns the recipe in schema.org/Recipe format, ready for review and for the app's content pipeline.

The content team inspects the generated instructions and the compatibility information before anything reaches the consumer app. The team's job changes from writing recipes to reviewing them.

Changes that remember the last change

Each change keeps the lastA substitutionDifferent portionsA dietary versionA new method
The engineering

An editor rarely wants a recipe exactly as filmed. Through the same interface, without running extraction again, they can ask for:

  • An ingredient substitution.
  • A different number of portions.
  • A version for a dietary preference or an equipment constraint.
  • A change to the cooking method.

The customization tool keeps the recipe's conversational context, so each follow-up instruction builds on the earlier ones, and the model returns the updated recipe in the same structured format. When the team has an idea and no video, it can describe the recipe it wants, and the model drafts one for the same review.

A prediction from prompts, not from a trained model

Built withStreamlitBedrock
Readings every 30 secondsRead by a promptA finish time
The engineering

For the thermometer, the obvious answer was the one the company had already ruled out: train a custom prediction model, which it had judged too resource- and cost-intensive, with only limited sensor data from the appliances to train on. We tested a prompt-based approach instead: whether a foundation model, given the right context and the telemetry the devices already collected, could reason about finish time from partial readings.

A Streamlit prototype collected the readings and asked Amazon Bedrock for an updated prediction every 30 seconds. Each request carried the whole cook so far:

  • Ambient temperature and internal meat temperature, for the cooking conditions.
  • Elapsed time and the accumulated readings, for how fast the temperature was rising.
  • Meat type and target temperature, for what "done" means.

The returned finish-time estimate was recorded, then updated as the next batch of readings arrived. The approach used engineered prompts and Bedrock foundation models only, with no separate custom-model training pipeline to build or maintain.

How early can the estimate be trusted?

Every prediction recordedError measuredThe point it settles
The engineering

A prediction shown too early is worse than none. So we evaluated every cook from its first partial readings to its actual finish:

  • Every prediction was recorded.
  • Mean Absolute Error compared predicted completion times with the actual cooking time.
  • Plots showed how the estimates moved as more readings arrived.
  • The analysis found the stabilization point: how much temperature history the model needs before its estimate is reliable enough to show a customer.

Why: what the product needs to know is from which minute of the cook the app can show the number, and average accuracy doesn't answer that.

One recipe, end to end

  1. An editor pastes a cooking video's URL into the Streamlit interface.
  2. The AI agent calls the video-processing tool, which fetches the video and extracts the audio. SQS queues the long job.
  3. Transcribe returns a timestamped transcript.
  4. The recipe-extraction tool turns it into a schema.org/Recipe, with ingredients, steps, cooking times and which of the brand's appliances it suits.
  5. The editor asks for changes in plain words, for example half the portions or no dairy, and the customization tool rewrites the recipe, keeping every earlier instruction.
  6. The editor approves it, and it goes to the app's content pipeline.

What we'd tell another CTO

1Look for the content you already have. The recipes the oven needed were already on video; the work was in reading them reliably and landing them in schema.org/Recipe, the format the content pipeline already took.
2Try prompting before training. Given the readings and the right context, a foundation model made a custom ML cook-time model unnecessary, and the stabilization point showed when its estimate could be shown to a customer.

What else changed

  • A working recipe prototype with a documented architecture for video extraction, recipe generation and editor review, built in a fixed-scope proof of value of 6–7 weeks.
  • A finish-time prediction without a custom model, with the stabilization point identified, from a proof of concept of about two weeks.
  • The first engagement signed off within two weeks, as a zero-cost AWS Demo Squad proof of concept. The recipe work followed as a paid engagement.

What comes next

Production deployment of the recipe workflow, and the cook-time prediction built into the mobile app.

How we'd do it today

This project came before our current offers. Here's how we'd approach the same work now.

With AK AI Factory, the brand's experts would sign off what "compatible with this appliance" means before any tool used it, each AI agent would get its own identity in AgentCore Identity, with its rules, spend limits included, as Cedar policies checked on every tool call outside the AI, and Bedrock Guardrails would screen what the AI agent says, which matters when a recipe touches allergens. A golden set of real videos with the recipe a good editor would have written, plus a sealed set, would gate every prompt change, and for the thermometer the Mean Absolute Error and the stabilization point would become tests that any new prompt or model has to pass.

Modernize with AI AK AI Factory

More work like this

  1. AI80–90%A U.S. software provider for government agenciesof form-onboarding effort automated in validationA signed contract waited six months to go live. Most of that time went on retyping PDF forms.70%+ cut in professional-services cost, targeted6–7 months of onboarding before, targeted at weeksOn AWS Bedrock · S3 · OpenSearch · Nova Pro
  2. AI2 weeksA marketing platform for home-services businessesfrom the proof of concept to a production commitmentA hundred and fifty people listened to calls all day. The question was whether a smaller model could do the job.300–500 human-labeled calls in the test150 people reviewing calls every day, beforeOn AWS Nova Sonic · Lambda · S3 · Glue
  3. AI80%A U.S. learning-platform providercut in exam processing time, targetedEvery customer's exam arrived in its own format. An expert reformatted each one by hand.$0 to the customer for a $40,000 assessment, AWS-funded3–4 weeks to a roadmapOn AWS Bedrock · ECS · Redshift · Lambda

All case studies →

Have a problem like this one?

A free workshop: a few hours with the architects who'd do the work, on your own systems. No cost.

Book a free workshop