Integrating AI Into Business Systems: Architecture

Summary

An architecture for integrating AI into business systems through APIs, permissioned data, evals, human review, observability and controlled rollout.

Three things to remember

Start with one workflow and a defined output contract rather than connecting a model directly to every business data source and action.
Separate data, retrieval, model, tools, policy, human review and observability so each boundary can be tested and changed independently.
Roll out from observation to recommendation and only then bounded automation; repeat relevant evals whenever data, tools or models change.

Choose one workflow and define the output contract before the model

Integrating AI into business systems should begin at a specific handoff: route an email, extract contract fields, retrieve a sourced answer or prepare a draft CRM update. Define the input, output, required fields, refusal conditions and accountable owner. This becomes the contract for APIs and evals.

When a process is fully deterministic, rules-based automation may be cheaper and more reliable. AI fits unstructured input or bounded judgement; business rules, limits and consequential actions still belong in deterministic controls. Select the model after defining the accuracy, latency and cost the workflow requires.

One owner, one outcome and one representative dataset.
A defined output schema, validation and refusal reasons.
Separate AI judgement, deterministic rules and human decisions.
A baseline for time, errors, rework and current cost.

Good AI integration does not make a model know everything. It gives the model the right context, tools and permissions for one testable task.

A seven-layer architecture for changeable AI integration

A practical architecture separates the request channel, orchestration, data or retrieval, model, tools and APIs, policy and observability. A model should not hold credentials or call production directly; a service layer validates schema, permissions, limits and idempotency before execution. OpenAPI can describe tool contracts, while business validation remains on the server.

Separation lets teams change a model without rebuilding CRM integration, change a data source without breaking the interface and evaluate components when outcomes fail. For longer tasks, explicit queues and state are preferable to one open request. A correlation ID should connect the request, retrieved data, model output, tool call and final outcome.

An authenticated request channel.
An orchestrator for state, timeouts and retries.
Permissioned, versioned retrieval with source references.
A model gateway for models, prompts, limits and cost.
Tool adapters that validate schemas and business permissions.
Policy and human approval for consequential actions.
Logs, metrics, traces, feedback and alerts.

Manage data and access for every request

Retrieval must preserve the access boundary of the source system. If an employee cannot see another department's contract, AI should not retrieve it either. Store document ID, version, source and permission scope with each piece of knowledge so answers can be verified and stale information can be removed.

Send only the minimum data needed for the task. Secrets, credentials and sensitive fields should not enter prompts or logs unnecessarily. CISA and NCSC recommend secure-by-design practices across AI system design, development, deployment and operations; data and permission boundaries belong in the architecture before production connection.

Inherit source permissions instead of creating one open knowledge store.
Metadata for ownership, version, validity and permitted use.
Redact or remove unnecessary fields before model calls.
Defined retention, log access and data-revocation processes.

Build evals and roll out from observation to automation

A test set should include normal cases, missing data, conflicting documents, adversarial input and business exceptions. Evaluate retrieval, schema conformance, tool selection and the final action, not only answer wording. NIST calls for testing, evaluation, verification and validation before deployment and during operation; OpenAI similarly recommends criteria-led evals when models or prompts change.

Begin in shadow mode, where AI processes work without affecting the source system. Move to recommendations that reviewers compare, then automate only stable, low-risk categories. Keep mandatory approval for consequential communication, data changes, permissions, finance or difficult-to-reverse actions.

Shadow: measure without affecting the source system.
Assist: propose while a person edits and approves.
Bounded automation: execute stable categories within limits.
Re-evaluate after every model, prompt, data or tool change.

Production checklist and Friday Works handover

Production needs rate limits, controlled retries, idempotency, circuit breakers, queues, fallbacks and a kill switch. Dashboards track quality, latency, cost, retrieval failures, tool errors and human overrides. A runbook identifies alert ownership, shutdown steps and how incomplete tasks are handled when a model or API is unavailable.

Friday Works hands over the architecture, API contracts, data map, test set, eval report, environment configuration, dashboards and runbooks included in scope. A focused pilot is commonly planned over 3–6 weeks after 1–2 weeks of validation; systems with several sources or sensitive actions are staged. Expansion follows only when production evidence meets the agreed thresholds.

Data, permission, API and ownership diagrams.
Eval report, acceptance thresholds and known limitations.
Monitoring, alerts, budget limits, fallbacks and incident runbooks.
Handover documentation and a decision gate for the next stage.

FAQ

Frequently asked questions

Can AI connect directly to a CRM or ERP?

It can connect through APIs, but a service layer should validate identity, permissions, schemas, limits and business rules. Models should not hold credentials or broad direct access to production systems.

Should we use RAG, fine-tuning or prompting?

Prompts provide task instructions; RAG fits changing, sourceable knowledge; fine-tuning can fit stable behaviour or formats with quality data. Use eval evidence rather than trend-driven selection.

Does AI integration require synchronising all data?

No. Begin with the minimum source for one workflow, preserve source permissions and retrieve only the data needed for each request. Broad synchronisation increases cost, staleness and access risk.

How do we know AI is safe enough to execute automatically?

Use a representative test set, acceptance thresholds, logs, limits, fallbacks and evidence from shadow and assist stages. Automate only low-consequence, reversible categories first.

What does Friday Works hand over after a pilot?

Depending on scope, handover can include source code, architecture, API contracts, data maps, test sets, eval reports, configuration, dashboards, runbooks, operating documentation and an expand-or-stop recommendation.

References

Sources used in this guide

We prioritise official guidance and primary technical sources. Visit each source for full context and the latest updates.