Armakuni Contact us
← All case studies

Six attribution models,
and no one in the shop to read them.

A Canadian eCommerce analytics platform had built deep attribution and machine-learning models, for customers who mostly had no analyst. In two proofs of value, we let a business owner ask the data a question in plain English and get a chart back, and built a pipeline that reads a web page, measures it and returns ranked recommendations for what to change. AWS funding meant the customer paid nothing for either.

Before

The buyers were shop owners, and the product was built for analysts.

Nobody told the customer what to change.

After
Plain Englishquestions answered with a chart
  • A webpage pipeline tested on 70–80 pages with their performance data, returning ranked recommendations on content, speed and layout.

The buyers were shop owners, and the product was built for analysts

The company's platform pulls in Shopify, WooCommerce and advertising data and runs it through six attribution models and ML pipelines, built on Aurora Serverless, Java and TensorFlow. It tracks purchase probability and customer lifetime value.

The people buying it were often the founder or the marketing lead of a smaller online shop. Asking a simple question, such as which channel had the best return last month, meant finding the right report and knowing how to read it. Each new layer of data was one more thing those customers couldn't use, which limited how many of them the platform could sell to.

Nobody told the customer what to change

The platform knew which pages converted and which didn't, and stopped there. Nobody looked at the page itself to say what to change, so a customer had to come to the platform, know what to look for and work out the fix alone.

Two questions, two proofs

We set out to test two things. The first was whether a non-technical owner could ask the data a question in their own words and get a clear, visual answer. The second was whether the platform could go and read a customer's page, measure it and come back with specific, ranked changes, bringing the value to the customer.

How we built it

2 decisions, in the order we made them. Each one says what we did and why, with what it did drawn beside it.

  1. Plain-English questions in the business's own vocabulary
  2. Read the page, measure it, then ask the model

Plain-English questions in the business's own vocabulary

Built withQuickSightAuroraMySQL
Campaign ROI, last month? QuickSight Q Topics a chart back
The engineering

Natural-language questions only work if the system knows what the business means by "ROI" or "lifetime value". The QuickSight proof of value began with a review of the Aurora Serverless/MySQL schema, the users' reporting needs and their common access patterns:

  • QuickSight connected to a production-representative sample of the data.
  • Two to three dashboards covered internal and customer-facing views of campaign performance, acquisition and purchase probability.
  • QuickSight Q Topics supplied the domain context, so a question about campaign ROI or customer lifetime value by channel lands on the right measures.
  • The engagement included training for the customer's team and a roadmap for embedding the capability into the product.

Why: topic configuration is where the answers become accurate, because without it a natural-language interface guesses at column names.

Read the page, measure it, then ask the model

Built withStreamlitFargateLambdaS3GlueDynamoDB
A page address comes inThe page is read and measuredMeasured chunks go to BedrockRanked fixes come back
The engineering

A model shown raw HTML gives vague advice. So the second proof of value followed a URL through collection, measurement and analysis, and gave the model structured context:

  • A Streamlit interface, hosted on Fargate, accepts the page address, and Lambda starts a stateless crawling task on Fargate.
  • Python tooling (BeautifulSoup and Selenium) extracts the page content, while containerized headless-browser audits capture performance measurements. HTML and audit JSON are stored in S3.
  • A scheduled Lambda running headless Chrome and Puppeteer generates the page thumbnails offline, so the interactive interface never waits on a screenshot.
  • Glue or Lambda prepares context-sized chunks containing the relevant HTML and its performance data, with S3 or DynamoDB holding intermediate data.
  • Bedrock analyzes each chunk, and multimodal processing uses the page thumbnails alongside the text, so the recommendations can address the visible layout as well as content and speed.
  • Aggregation ranks the outputs and notifies the interface when results are ready, through EventBridge or SQS, and the user gets the recommendations, the measurements and the thumbnails together.

Because each chunk carries its own measurements, the model can say "this hero image is slow" where it would otherwise say "consider optimizing images".

What we'd tell another CTO

1Give natural language the business's vocabulary through topic configuration, and push the insight to the customer, because a recommendation they didn't have to ask for is the value they notice.
2On the page side, never hand a model raw HTML. Chunk it, attach the measurements to each chunk and let the model see a thumbnail so layout advice is possible, and keep the interactive path light, with thumbnails on a schedule, crawls in their own tasks and results by notification.

What else changed

  • Two to three dashboards across internal and customer-facing views.
  • Working prototypes, documentation, a trained team and a production roadmap for both.
  • The QuickSight work took about four weeks and the page-optimization work about nine. AWS funding covered both, so the customer paid $0.

What comes next

Embedding both capabilities across the customer base, and measuring their effect on adoption, conversion and retention.

How we'd do it today

This project came before our current offers. Here's how we'd approach the same work now.

Axiom would give "conversion", "ROI" and "lifetime value" one meaning in a catalogue on Amazon DataZone, so every AI agent and every natural-language question reads them the same way, and AWS Lake Formation would keep each shop to its own rows. AK AI Factory would test the recommendations as well as the pipeline: a golden set of pages with the changes an experienced conversion specialist would make, plus a sealed set nobody tunes to, gating every prompt and model change.

A crawled page is untrusted input, because one hidden instruction on it can steer an AI agent, so the AI agent that reads crawled content couldn't also send data out. Across the customer base, AI Inference Engineering would send the routine pages to a model the company owns on Amazon SageMaker AI, for roughly a tenth of the frontier price, with the hard ones on a frontier model through Amazon Bedrock.

Modernize with AI AI Inference Engineering

More work like this

  1. DataAny hourA care-management software providerfor scheduled refreshes, where extracts had to fit 6 a.m. to 7 p.m.Reports could only refresh from 6 a.m. to 7 p.m. Outside those hours, everyone read yesterday.2 → 1 ETL vendors replaced by one pattern3 regions kept apartOn AWS Redshift · EC2 · RDS · DMS
  2. Data40%A U.S. outdoor and fitness brandhigher return on targeted campaigns, as reportedThe same customer bought the gear, ran the event and joined the club. The brand saw three strangers.2× loyalty participation5× engagement across channelsOn AWS DynamoDB · Redshift · Glue · CloudWatch
  3. DataWorkingA U.S. data intelligence companyclean-room prototype, with every security boundary testedIts value was in combining data with partners. Every partnership started from scratch.4–6 weeks to a working clean-room prototype2 machine-learning patterns inside the clean roomOn AWS Redshift · Glue · S3 · Athena

All case studies →

Have a problem like this one?

A free workshop: a few hours with the architects who'd do the work, on your own systems. No cost.

Book a free workshop