Reference architecture · v3

One architecture. Every system your business runs on.

A modern data & AI architecture for the regulated enterprise — engineered end-to-end, from open lakehouse and ingest, through data products and AI, to decisioning, governance and audit.

Open formats Iceberg · Delta · Parquet Native execution your warehouse · your cloud Audit by default signed · replayable Compliance SR 11-7 · SS1/23 · DORA
The architecture, end-to-end

Seven planes, one coherent system.

The reference architecture spans every plane the regulated enterprise has to engineer — from the open lakehouse to the audit ledger. Click any plane or component to drill in.

reference architecture · v3 · click any block to expand blueprint live
sources
core ledgerCRM · ERPKafka streamSaaS APIswarehousefiles · unstructured3rd-party data
01 — Foundation Open lakehouse
02 — Ingest Streaming, batch & CDC
03 — Data products Domain-aligned mesh
04 — Governance Policy, lineage & classification
05 — AI / ML platform Features, models & evaluation
06 — Decisioning & agents Runtime & orchestration
07 — Observability & audit Drift, signed ledger & replay
consumers
channel appops & branchBI & analyticscopilotsaudit · riskregulator APIpartner systems
·

Click a component to drill in.

Includes
    Standards & SLOs
      Native integrations
        tip · click ▸ expand on any plane, or any component chip, to drill in
        Deployment
        Public, private or hybrid
        Data residency
        In-region only
        Tenancy
        Per-org isolation
        Audit trail
        Signed · replayable
        Engineering principles

        What makes the architecture work in production.

        The blueprint is opinionated where it has to be, and modular where it shouldn't be. Six principles run through every plane.

        01

        Open by default

        Iceberg, Delta, Parquet, Arrow. No proprietary lock-in at the storage or interchange layer — your data stays portable across engines and clouds.

        02

        Native execution

        Compute moves to where data lives. Snowflake, Databricks, BigQuery, Iceberg-on-S3 — models and decisions execute in-engine, not in a side car.

        03

        Contracts at every boundary

        Every ingest, every data product, every model output is a versioned contract — schema, semantics, SLOs and ownership all explicit.

        04

        Tenancy as first-class

        Shared-nothing isolation across data plane, control plane and audit log. Cryptographic separation per organisation, region or business line.

        05

        Audit by default

        Every decision and every data movement is signed, content-addressed and replayable. Evidence is a property of the architecture, not paperwork.

        06

        Composable, not monolithic

        Adopt one plane at a time. Each layer is independently deployable, with stable contracts to its neighbours — so you can modernise progressively.

        The seven planes

        Deeper into each layer.

        Each plane is independently deployable, but designed to compose. You can adopt the architecture progressively — one plane at a time, one decision at a time.

        01 — Foundation

        Open lakehouse, your cloud.

        An open lakehouse on Iceberg or Delta, deployed in your hyperscaler of choice. Storage, compute and catalog are decoupled, so you can run multiple engines on the same tables — Snowflake, Databricks, Trino, DuckDB — without copying data or fragmenting governance.

        Iceberg · Delta · Parquet · Arrow multi-engine: Snowflake · Databricks · Trino · BigQuery shared-nothing tenancy · in-region residency
        02 — Ingest

        Streaming, batch & CDC, contracted at the edge.

        Boundary contracts on every source — Kafka topics, batch loads, CDC from operational stores. Schemas are versioned, quality SLAs are testable, and breaking changes are caught before they reach a data product.

        Kafka · Kinesis · Pub/Sub · Debezium · Airbyte contract-tested with Great Expectations / Soda backfill & replay native
        03 — Data products

        Domain-aligned, contracted, discoverable.

        Domains own their data products. Each product has a contract — schema, semantics, SLOs and an owning team — and is published into a catalog the rest of the enterprise discovers and consumes from. The mesh isn't the destination; it's how the architecture stays coherent at scale.

        data products as first-class objects federated catalog · OpenLineage · OpenMetadata semantic layer for analytics & AI consumption
        04 — Governance

        Policy-as-code, lineage end-to-end.

        Approval policies, classification rules and residency constraints expressed as code, executed at every boundary. Lineage is captured automatically, end-to-end, from source through data product to model output and decision.

        OPA-compatible policy DSL OpenLineage end-to-end · Unity / Polaris compatible SR 11-7 · SS1/23 · EU AI Act · DORA mappings
        05 — AI / ML platform

        Features, models, evaluation — production-grade.

        A feature store on the same lakehouse as your data products. A model registry with stage gates and approvals. Training pipelines that are reproducible from contract to artifact. Evaluation harnesses for both ML and LLM use cases.

        feature store on lakehouse · point-in-time correct model registry · staging & approval gates reproducible training · LLM & ML eval harnesses
        06 — Decisioning & agents

        From model to decision, with the same governance.

        The runtime that turns models into decisions and agents into actions. Sub-150 ms p99 routing over your features and models, HITL queues for everything that needs a human, MCP-native tool calling for agentic workflows — all under one evidence chain.

        p99 142 ms · contract-versioned routing HITL queue · step-level approval MCP · tool-calling · agent observability
        07 — Observability & audit

        Drift, signed ledger, regulator-ready.

        Population, feature and outcome drift on every tenant, in real time. Every decision content-addressed and signed. Replay any decision, any model, any data product as it was on any given day. Six years of retention, regulator-API ready out of the box.

        drift ≤ 60 s detection · per-tenant baselines content-addressed audit ledger · 6 yr retention full replay · regulator API
        Engineering specifications

        What you can rely on, in production.

        CapabilityWhat it doesSLO / spec
        Open lakehouseIceberg or Delta on object storage in your cloud, with shared-nothing tenancy and in-region residency.Iceberg · Delta
        Multi-engine executionModels and decisions execute in-engine on Snowflake, Databricks, Trino or BigQuery — zero-copy.native · zero-copy
        Data productsDomain-aligned products with versioned contracts, SLOs, ownership and catalog publication.contracted · catalogued
        LineageEnd-to-end lineage from source through data product to model output and decision, OpenLineage compatible.100% coverage
        Policy-as-codeApproval, classification and residency policies expressed as code, executed at every boundary.OPA-compatible
        Feature storePoint-in-time correct features served from the same lakehouse, in-engine.in-engine · PIT
        Decision routingTenant-aware routing of decisions to the correct model version, with contract validation and signed output.p99 142 ms · 99.97% / mo
        Drift monitoringPopulation, feature and outcome drift detected per tenant in real time. Escalates via HITL queue.≤ 60 s detection
        Audit ledgerEvery decision and every data movement content-addressed, signed and replayable. Regulator-API ready.6 yr retention
        Compliance mappingsSR 11-7, PRA SS1/23, FCA Consumer Duty, EU AI Act, DORA — exported as evidence.five frameworks
        DeploymentRuns natively in your cloud — hyperscaler-managed, private VPC or hybrid — under your IAM. Helm or Terraform; 28-day provisioning average.28 d avg
        Working sessions, not sales calls

        Walk the architecture in your environment.

        Ninety minutes with our founding engineers. We bring the reference architecture, you bring one decision your business depends on. We map one to the other, in the room.