Retailers wanted to learn from their partners' data.
Nobody wanted to hand theirs over.

We built the collaboration layer so the protection doesn't depend on anyone being careful:

• Partners can now run joint analysis in the clean room.
• Secure data rooms separate each customer's datasets from the aggregated, anonymized data used for benchmarking, so a retailer sees how it compares with its peers without anyone seeing another retailer's data.
• The identity providers give partners and users one authentication model, so bringing on a new partner is a configuration change rather than a custom integration. Fine-grained IAM roles also keep each service and each operator down to least-privilege access.
• Keys managed through KMS protect data at rest in Amazon S3 and Amazon RDS, and encryption in transit is built into the design alongside it.

Access control sits in the data path, so it applies whoever runs the query.

Every change had gone through the same codebase, so a small change in one analytics flow could unsettle the others. We organized the application into Admin, E-commerce and Core modules, with defined API boundaries between them. Operational management, retailer-facing analytics and shared services can now change separately, and a product change involves far less unrelated code.

The same APIs give the team a defined route for new retailer and partner integrations, without rebuilding the whole analytics application.

Running at peak capacity all the time costs too much, but sizing for an average day doesn't work either, because month-end brings a reporting load that an average day can't absorb. The team moved application services onto Amazon EKS (Kubernetes) and set the infrastructure up to follow the load:

• Karpenter adds and removes nodes as workloads change, so the team doesn't pay for peak capacity during the quiet periods.
• NGINX Ingress handles routing, TLS and traffic policy in one layer, and that keeps response times steady during the heavy reporting windows.
• The deployment architecture can extend into additional AWS regions as customers and data-residency requirements grow, because adding a region fits the existing model instead of needing a custom build.
• Development workloads run on Spot capacity, cutting their cost without touching production's availability.

The old PostgreSQL setup slowed down at reporting peaks and needed manual upkeep. We moved PostgreSQL to Amazon RDS through AWS DMS, with zero data loss, so managed backups, replication and patching replaced part of the team's maintenance work.

These checks sit inside the release path itself. A check that someone could skip wouldn't actually keep the platform compliant, so it had to be placed where no release could get past it:

• The CI/CD pipelines run security scanning, policy checks and approval gates on every build, so every release carries the same controls without anyone having to apply them by hand.
• A self-hosted OpenVPN setup controls and logs engineering access to the deployment environments, in place of the ad-hoc credentials used before.
• Prometheus collects the metrics, Loki aggregates the logs, and Grafana brings service health, query behavior and access patterns together in one place, so an anomaly shows up as it happens instead of turning up later at audit time.

That way compliance evidence comes out of running the platform day to day, and there's no scramble to pull it together before each questionnaire.