Agent Architecture for SMBs: A 90-Day Path to Production

An enterprise agent should not be designed as “a chatbot plus a few APIs.” But a small or midsize business should not copy a global company's central agent platform, marketplace, and knowledge infrastructure either. The former can answer but cannot execute reliably; the latter exhausts budget before it creates business value.

An SMB needs a minimum production architecture: an agent that completes one bounded business loop—understanding an objective, acquiring context, using tools, verifying the result, and returning control to a person when uncertainty or risk is too high.

This article answers four practical questions: which use cases deserve an agent, how to structure the production system, how to deploy it in 90 days, and how to measure the outcome honestly.

Does the problem need an agent?

OpenAI describes an agent's foundation as model, tools, and instructions, and recommends use cases involving complex judgment, unmanageable rules, or substantial unstructured data. Anthropic reaches a similar conclusion from production work: begin with simple, composable patterns and add autonomy only when its value covers the extra latency, cost, and failure surface.

If a few deterministic branches solve the task reliably, do not build an agent yet.

Adapt enterprise layers to SMB constraints

Gateway, orchestration, tools, knowledge, policy, and evaluation remain useful ideas from enterprise architectures. The difference is that an SMB should combine them and split only under proven load.

An SMB may have only two to five people operating the entire system. Its architecture must remain simple, replaceable, observable, reversible, and economically tied to accepted results.

The minimum production architecture

The architecture separates model judgment from real execution. An LLM may propose a plan or tool call; a deterministic policy engine decides whether it is allowed. Tools expose narrow business operations—not database administration or an unrestricted shell.

1. Interaction and gateway

Requests can arrive through a web app, messaging platform, email, CRM button, API, or event. The gateway handles identity, tenancy, rate limits, and normalization into a task envelope containing task type, actor, input, risk, budget, and acceptance criteria. Authorization, cost, and evaluation then belong to a business action rather than an untraceable conversation.

2. Deterministic orchestration first

Code controls known flows: load a lead, enrich it, score it, draft a response, request approval, and update CRM. Let the agent choose a tool only when the next step genuinely depends on its findings.

Production states include queued, running, waiting_approval, succeeded, failed, and cancelled. Every step needs a timeout, retry ceiling, idempotency key, and checkpoint. Long work belongs on a queue, not inside one open HTTP request.

3. Agent runtime and model gateway

Agent = Instructions + Tools + Task State + Policies + Evaluation

Treat models as replaceable dependencies. Small models classify, extract, and route; stronger models plan and synthesize. Set token, tool-call, time, and cost ceilings per task. Validate output against a schema before it becomes an action. Failed validation receives a bounded repair attempt or human escalation.

Keep vendor SDK calls out of business logic. Model and prompt changes pass regression evaluations before gradual rollout.

4. Separate state, facts, knowledge, and preferences

• Task state: progress, tool results, and unresolved questions.
• Business facts: CRM, ERP, and order systems remain sources of truth.
• Knowledge: manuals, policies, and templates are retrieved with citations and access control.
• Long-term preferences: store only with a purpose, consent, expiry, and deletion path.

Webpages, emails, and documents are untrusted inputs. Their instructions cannot become system commands. Retrieval must enforce tenant isolation, data classification, source, version, and permission filters.

5. Narrow tools and least privilege

Avoid a universal “operate CRM” tool. Prefer get_customer, list_recent_orders, draft_followup, and update_lead_status, with typed parameters and business-rule validation.

The policy engine—not the model—validates identity, resource scope, data class, approval state, and financial thresholds. OWASP similarly recommends least-privilege tools, explicit authorization for sensitive actions, distrust of external input, and separation of reasoning from irreversible execution.

6. Evaluation and observability

Record task, prompt or skill, and model versions; cited context; summarized tool parameters; approvals; latency; tokens; cost; final state; and human corrections. Redact credentials and personal data.

Evaluate three levels:

1. Step: schema validity, field completeness, and tool choice.
2. Trajectory: unnecessary loops, privilege attempts, waste, and recovery behavior.
3. Outcome: acceptance, rework, cycle time, and cost per accepted result.

Start with 30–50 historical tasks covering ordinary, edge, and adversarial cases. Rerun them after changes to models, prompts, tools, knowledge, or permissions. NIST's Govern–Map–Measure–Manage structure provides a useful lightweight governance backbone.

Default to one agent

Begin with one orchestrator, one agent, and several narrow tools. Split only when evidence shows that unrelated contexts interfere, genuine parallelism saves meaningful time, an independent evaluator must not share the generator's path, or roles require different permissions or models.

Even then, use central orchestration and structured messages. Cap turns, total budget, delegation depth, and execution time. Agents holding a long “meeting” is not intelligence; it is uncontrolled cost.

First workflow: lead research and follow-up

Consider a 30–80 person B2B services company. Sales repeatedly inspects inbound forms, researches companies, assesses fit, drafts outreach, and updates CRM. The task is frequent, time-consuming, reviewable, and safe to begin in draft-only mode.

New lead event
  → validate and deduplicate
  → read CRM and approved sources
  → create an evidence-backed profile and score
  → validate fields, citations, and prohibited claims
  → draft follow-up and next action
  → sales approval
  → update CRM and turn human corrections into eval cases

Version one receives read and draft permissions only. After two stable weeks above the acceptance threshold, it may update reversible internal fields. External communication continues to require human confirmation.

A 90-day rollout

Weeks 1–2: scenario and baseline

• Select one workflow and one accountable business owner.
• Sample 50 historical tasks and measure handling time, waiting, first-pass acceptance, rework, and exceptions.
• Define forbidden outcomes: cross-tenant access, unsupported claims, and unapproved external action.
• Map sources of truth, data classes, and tool permissions.

Deliverables: use-case contract, baseline, 50-case evaluation set, risk register, and stop criteria.

Weeks 3–4: read-only MVP

• Implement the task envelope, state machine, model gateway, and three to five read tools.
• Produce structured recommendations and drafts without writing to business systems.
• Add traces, cost accounting, and error categories.
• Shadow-run with two or three real users.

Release gate: no authorization failure, at least 95% critical-field completeness, 100% traceable citations, and cost within budget.

Weeks 5–8: bounded writes and approval

• Add reversible writes, idempotency, and approval records.
• Convert human edits into failure categories and regression cases.
• Test prompt injection, sensitive-data leakage, excessive tool access, and unbounded loops.
• Add timeouts, circuit breakers, fallback, and a kill switch.

Release gate: agreed first-pass acceptance, stable P95 latency and accepted-result cost, and zero critical security failures.

Weeks 9–12: controlled trial

• Compare old and new processes for at least two full business cycles.
• Include model, infrastructure, review, maintenance, and failure costs.
• Reuse stable gateway, approval, evaluation, and connector components.
• Reduce scope or stop if the exit criteria are not met.

Deliverables: outcome dashboard, incident playbook, runbook, and evidence-based expand/stop decision.

Report outcomes honestly

Do not claim an “80% efficiency gain” before the pilot. Freeze the baseline first and fill in measured results afterward.

monthly net value
= hours saved × fully loaded labor cost
+ incremental business value
- model and infrastructure cost
- review and maintenance cost
- expected loss from failure

The useful unit is cost per accepted business result, not cost per model call.

Buy, assemble, or build?

1. Buy an agent embedded in existing SaaS when its permissions and audit are sufficient.
2. Assemble a managed model, workflow or agent SDK, and an owned policy layer when cross-system execution matters.
3. Build a general platform only when scale, regulation, or durable differentiation can fund the long-term team.

Whichever route you choose, own the task protocol, tool contracts, evaluation set, and audit data. These assets will outlast a particular framework.

Production checklist

• A business owner, system owner, and kill-switch owner are named.
• Every task has acceptance criteria, budget, timeout, and step limit.
• Business facts remain in a system of record.
• Tools are least-privilege; no unrestricted write or code execution exists.
• External, financial, destructive, and production actions require approval.
• Tenant, user, session, and long-term memory are isolated.
• Logs are redacted and runs are reconstructable from task IDs.
• Offline evals, shadow mode, gradual rollout, and rollback exist.
• Model, retrieval, and tool failures have degraded paths.
• Monthly review decides whether to expand, narrow, or retire the workflow.

Conclusion

An SMB does not win by building a more complicated platform than a large enterprise. It wins with a shorter loop from problem discovery to pilot, measurement, and correction.

Use deterministic workflows as the skeleton and an agent only for uncertainty that requires judgment. Grant read and recommendation access before execution authority. Establish evaluation and audit before pursuing autonomy and scale.

One agent that reliably completes a business loop, knows when to stop, and produces evidence is worth more than a department of “digital employees” no one can evaluate.

Further reading

• OpenAI: A Practical Guide to Building AI Agents
• Anthropic: Building Effective Agents
• Anthropic: Demystifying Evals for AI Agents
• OWASP: AI Agent Security Cheat Sheet
• NIST: AI Risk Management Framework