An architect reviews a pull request on a Monday morning. The code is competent, the tests pass, and it was written in eleven minutes by a system that has never met the customer whose invoice it will eventually calculate.

Nothing about that is alarming on its own. The problem arrives later, in a meeting where somebody asks who decided this was the right design — and the honest answer is that nobody did. The system proposed, the pipeline approved, and a person clicked merge while thinking about something else.

That is the question worth an executive's attention. Not whether AI can write enterprise software. It plainly can. What has to remain human for the result to be worth deploying.

How this article is sourced

JWilliams & Associates publishes this site, and this is a sponsored feature about the company's stated approach. That belongs at the top rather than the bottom.

So here is the separation held throughout. The evidence on human-governed AI comes from published frameworks and 2026 research, cited with links and with their limits named. The description of how JWA works is a statement of design philosophy supplied by the company, and it is labeled as such.

It is not an audit, a certification, or a customer result. Nothing here should be read as a claim that any particular system is secure, and no outcome is asserted for any client.

What NIST says about the human in the loop

NIST's AI Risk Management Framework, in its appendix on human-AI interaction, is unusually blunt for voluntary guidance. Human roles and responsibilities in making decisions and in overseeing AI systems, it states, “need to be clearly defined and differentiated.”

The same appendix refuses the comfortable version of that story. AI systems “can autonomously make decisions, defer decision making to a human expert, or be used by a human decision maker as an additional opinion” — three configurations with materially different risk profiles, routinely collapsed into one procurement conversation.

It then makes the point most human-in-the-loop programs would rather skip. Human-AI interaction results vary, and in some judgment tasks the AI component can amplify human bias, producing “more biased decisions than the AI or human alone.”

Adding a person to the loop is not automatically a control. Sometimes it is a rubber stamp with better production values.

Governance without guardrails is unenforceable

A paper published on Zenodo in March 2026, “Designing Accountability In,” supplies the distinction I find most useful operationally. It separates governance — organizational policy — from guardrails, meaning technical enforcement, and argues the split is consequential in both directions: governance without guardrails is unenforceable, while guardrails without governance are arbitrary.

Both failure modes are familiar. The policy document nobody can enforce. The technical control nobody can explain.

The paper maps specific mechanisms — domain bounding, human-in-the-loop interrupt architecture, circuit breakers, immutable audit trails, output guardrails — to the point at which each stops being optional. It draws on the NIST AI RMF, Article 14 of the EU AI Act on human oversight, and the Moffatt v. Air Canada (2024) liability ruling.

Independent work reaches a similar place from a different direction. A 2026 Enterprise AI Governance study models the risks of AI-driven workflows as a register that reads like a catalog of oversight gone nominal: excessive agency, unclear accountability, unmanaged third-party dependency, and automation bias — reviewers approving AI proposals by habit, citing research that predates the current AI wave by decades.

The author is candid about the study's status. It is a synthesis and design study, with empirical validation and legal review identified as necessary next steps. Treat the structure as useful and the completeness as unproven.

A control baseline, by risk tier

The most practical artifact in that study is a tiered baseline. Tier 1 covers internal, reversible uses with no sensitive personal data. Tier 2 covers customer, supplier or financial impact where actions are reversible or reviewed. Tier 3 covers legal or significant effects, irreversible actions, and regulated decisions.

ControlTier 1Tier 2Tier 3
Named owner in an AI inventoryRequiredRequiredRequired
Least-privilege authorizationRequiredRequiredRequired
Injection defenses and output validationRequiredRequiredRequired
Human review of defined exceptions—RequiredRequired
Human approval before consequential action—SampledRequired
Full audit record of inputs, outputs, decisions—RequiredRequired
Kill switch and tested fallback—RequiredRequired
Independent review before release——Required

Two rows deserve attention. Human approval before consequential action is mandatory only at Tier 3 and sampled at Tier 2 — a judgment about where a signature earns its cost. And the kill switch plus tested fallback is mandatory at Tier 2, not reserved for the most sensitive systems.

That second row contradicts common practice. Most organizations rehearse shutdown for their most critical systems and assume everything else can simply be switched off.

Where JWA's stated philosophy sits

Nothing above is JWA's argument. It is the published record, and JWA is one company working inside it.

What the firm describes is an approach built on architectural boundaries rather than feature lists. The persistence layer is treated as an enforcement point rather than a place to store rows: application components request operations through governed procedures instead of holding direct authority over tables. Identity and sessions are controlled centrally. Authorization is evaluated per operation against current identity, tenant boundary, granular permission and the specific request — rather than inherited from a login and carried forward for the life of a token.

That last distinction is not a new idea. It is what zero trust guidance means when it moves authorization from network location to application and service identities, and it is what the governance research means when it says AI components should act under the requesting user's identity or a minimal service identity, never a general account.

The company's stated position on it is already on the record in a companion perspective published under its CEO's name, arguing that authority belongs at the layer that owns the record rather than at the front door.

JWA also describes AI orchestration inside that frame — agents and assistants operating under scoped permissions, with consequential writes staged for approval rather than applied — and states a staffing philosophy explicitly: custom AI as a force multiplier for existing staff, not staff replacement as the default goal.

The reconstruction test

Every version of this argument reduces to a question a controller, an auditor or a customer can ask: explain this number.

A crosswalk published on Zenodo in June 2026 proposes operational structure for something the NIST framework itself flags as underdeveloped — the definition of human roles in AI-mediated decisions. Its central unit assigns each consequential AI-mediated decision to a specific individual at the moment it occurs, rather than attributing it retrospectively to a role or a team.

The gap between those two things is the gap between accountability and a signature block.

Two caveats belong with it. The paper is self-published by the institute that authored the framework, and it describes itself as a limited crosswalk rather than a substitute for NIST guidance. It has been submitted for consideration, not endorsed.

The idea still holds. A record that cannot answer which principal acted, in what capacity, against what prior state, and whether the request was anomalous for that principal is a log. Reconstruction is a higher bar than logging, and almost nothing in production clears it.

What AI genuinely accelerates, and what it cannot

The honest list is shorter than the slide deck. AI compresses work that was never the source of advantage: boilerplate, test scaffolding, the fourth version of a field mapping, extraction from supplier documents, a first pass at a migration script.

It does not decide which system owns a field. It does not accept the risk of an irreversible write. It does not notice that its confidence has no relationship to ground truth.

NIST's guidance on generative AI is sobering here. It notes that current pre-deployment testing may be inadequate or mismatched to deployment contexts, and that jailbreaking or prompt-engineering tests “may not systematically assess validity or reliability risks.” It also recommends change management records capturing the reason for each change, how it could affect each intended context of use, and step-by-step detail of how the change was planned, tested and deployed.

That is a documentation burden most AI programs have not funded. It is also the only thing standing between a fast improvement and an unexplained regression six months later.

An illustrative scenario, clearly labeled

Constructed to show the method. No real company, customer or result is described.

A distributor adds an AI assistant to its order desk. In the first build, the assistant calls the application's API, which holds a database connection with broad write rights. The assistant inherits everything the connection can do.

In week three it chains from a customer email — text an outsider can write — into a tool call that adjusts a promised date. It was authorized, because the connection could write. Whether it was correct depended on somebody reading the draft before it saved.

Rebuild it with a mediated persistence layer. The application holds no table grants. Every operation is a named call with the principal bound and the tenant asserted. The assistant's authority is scoped per task and expires with it. The date change is staged, with the source email attached, and a named person approves or rejects it.

The assistant still runs. It simply cannot do the thing nobody intended, because no path permits it.

What JWA does not claim

This is the section a vendor usually omits, so it is worth being explicit.

No certification is asserted here. No customer result, savings figure or incident record is claimed. No security guarantee is offered, and no architecture is universally sufficient — the frameworks cited in this article are voluntary guidance, and guidance cannot certify an implementation.

What can be claimed is narrower and more useful. The architectural properties described above are testable. Ask to see a revocation take effect. Ask which component holds table-level authority. Ask whether an agent can be distinguished from a human in the audit record. Those questions have answers, and the answers either survive inspection or they do not.

The multiplier and the multiplier's supervisor

The framing that survives contact with reality is unglamorous. AI is a multiplier. Multipliers do not decide what is worth multiplying.

Experienced architects still own the decisions that cannot be reversed cheaply: which system is authoritative for a field, where an approval gate sits, what happens when authority is withdrawn at 2 a.m. Security still owns the boundary. Quality assurance still owns the standard the output must meet.

What changes is throughput. What does not change is who signs.

The organizations that get this right will not be the ones with the most capable models. They will be the ones whose architecture can answer, in a single query, what happened, under whose authority, and whether anyone agreed to it beforehand.

Hashtags

#CustomSoftware #SystemsIntegration #EnterpriseAI #BusinessAutomation #AIGovernance #ZeroTrust #EnterpriseArchitecture #LeastPrivilege #DataGovernance #CyberResilience #AIAgents #ITLeadership #CISO #CIO #RiskManagement #DigitalTransformation #DataSecurity