AI Governance Auditing: Who Owns AI-Generated Code?

AI agents and coding tools now write a large share of production code. Human review still matters, and in regulated industries, every line must also answer to an auditor. A bank, an insurance company, or an air-traffic organization can keep the speed AI-assisted software development brings only if each change can answer three questions: where did it come from, who approved it, and who is accountable for it?

Or, to sum it up in just one question: what work should AI do, and what should humans own?

That is the second open question from our Tech Week conversation. In this article, we answer it by covering where human approval is required, how to trace the origin of AI-generated changes, and how to separate low-risk testing from changes that touch customer data or critical infrastructure. In short, it is a practical guide to AI governance auditing for engineering teams.

Key Takeaways

• AI agents write almost half of developers' code, so tracking where it comes from is crucial for delivery.
• AI coding tools can propose changes, but approval and accountability stay human.
• AI governance frameworks should be tiered by risk, and keep the heaviest approval for customer data, payments, access control, and critical infrastructure.
• Every AI-generated change should record its origin: who asked for it, using which tool and model, in what context, and who approved it.

AI Is Already Delivering Code in Regulated Engineering

Before we get into matters of control, it’s worth being clear about the state of AI-assisted software development and its implications.

Agents Now Write a Large Share of Production Code

In its Developer Ecosystem Survey 2026, JetBrains asked more than 15,000 professional developers how last month's work code was produced. On average, about 47% was written fully by AI agents, with a further share written with assistance from an AI tool.

Sonar's 2026 State of Code survey expects that figure to reach 65% by 2027. At the company level, Google said in April 2026 that 75% of its new code is AI-generated and approved by engineers. That is one company's self-reported figure, but it shows how widespread agentic AI software development has become and where it’s heading.

Where Regulated Teams Gain the Most

Many regulated organizations have large backlogs in software, including legacy core systems and outdated tools.

That is where AI coding tools help most. Legacy systems modernization, internal applications, and front-end rework are bounded, repeatable, and expensive to do by hand. On the flip side, AI-powered solutions allow regulated enterprises to accelerate modernization timelines by 40-50%, as reported by McKinsey.

What Puts That Speed at Risk

However, this speed gain is fragile. In the same Sonar survey, 96% of developers said they don't fully trust AI-generated code to be functionally correct, yet only 48% always verify it before committing.

In most industries, that gap shows up as bugs and engineering drift. In regulated industries, it shows up in an audit. For example, when a reviewer or supervisor asks who wrote a change to a payments service and who approved it, the answer can’t be vague. In some cases, the reaction is to restrict AI tool use, which makes the speed gain disappear with it.

The way to keep it is to make AI-generated work as traceable as human work, and that starts with ownership.

Who Owns AI-Generated Code?

In short, a person. An AI coding assistant can draft, refactor, and propose a change, but it can’t be accountable for any of it. Every AI change in production must be reviewed, accepted, and accounted for by a human being.

An Agent Can't Sign Off

The clearest public example comes from one of the most scrutinized codebases in the world. The Linux kernel's policy on AI coding assistants says AI agents must not add any “Signed-off-by” tags, because only a human can legally certify the source of a contribution. It’s the human submitter who reviews and takes full responsibility for all AI-generated code.

This policy requires contributors to reveal AI involvement with an "Assisted-by" tag, specifying the agent and model version. Disclosure and accountability remain separate: the tag indicates what helped, while the sign-off indicates who is responsible.

Regulated enterprises need the same split, enforced by their own tooling rather than by convention. Ownership here means accountability, not copyright. Who holds the rights to AI-generated code is a legal question that varies by jurisdiction. Who answers for it in production is an engineering question, and it has a clear answer.

Three Roles for Every Change

Ownership becomes clear when every change records three roles:

• Requester: the person who asked the agent for the change and set its scope.
• Approver: the person who reviewed the result and accepted it for merge or release.
• Owner: the team or person who answers for that code in production, usually through code ownership rules.

For low-risk work, one person can hold the first two roles. For high-risk changes, the approver must not be the requester. That split is what auditors look for, and it is where human approval becomes non-negotiable.

Where Human Approval Is Required

Human approval should scale with risk. If every AI-generated change gets the same heavy review, teams either stall or start skipping steps. If every change gets the same light review, the one that touches customer data slips through, as we covered in the first article of this series.

A simple three-tier model works for most engineering teams in highly regulated industries. The backbone of a practical AI governance framework for engineering is to decide in advance which changes need which controls, to write them down, and to let the pipeline enforce them.

Tier 1: Experimentation and Sandboxes

This is where teams should move fastest. Engineers and PMs test ideas in safe environments without real data or access to production. The control here is the boundary, not the review. Nothing reaches a production branch from Tier 1 without reclassification and passing the checks of a higher tier.

Tier 2: Standard Production Changes

Most day-to-day work lands here: a new screen, a change to an internal tool, or an update to a service that doesn’t touch sensitive data. Automated checks run first, and a human reviewer accepts the change and stores the approval with it.

Tier 3: Customer Data and Critical Infrastructure

This tier covers changes where a mistake is expensive or permanent: payments, login and access control, personal data, data migrations, core infrastructure, and safety-relevant systems in sectors such as aviation and defense.

Here, human approval is mandatory, and the approver must be someone other than the person who requested the change. That is not a new rule invented for AI. Under the EU Digital Operational Resilience Act, the technical standard on ICT change management requires financial entities to keep the functions that approve ICT changes separate and to plan, test, and document those changes. The Dutch supervisor AFM went further in its DORA pilot review: it expects a complete audit trail that shows separation of duties and documented approvals, including when changes move across several systems.

An AI agent is another system that changes pass through. If they don’t record its contribution, the audit trail has a gap exactly where supervisors look.

How to Trace the Origin of AI-Generated Changes

Provenance means that for any change in production, you can show where it came from, what made it, and who accepted it, using records an auditor can trust. In practice, that comes down to a short list of fields captured automatically with every change.

Record What Produced the Change

Start with the tool, the model version, and the request. Model versions matter because the same prompt can lead to different code from different models, so an auditor investigating an incident needs to know which one wrote it. The Linux kernel's “Assisted-by” format, which names the agent and the model version, is a simple public precedent teams can copy into their own commit rules.

The request matters as much as the model. A change asked for as “add a discount field” and a change asked for as “rework the pricing service” carry different risks, even if the diffs look similar.

Tie Every Change to an Identity and an Approval

Every AI-generated change should link to a real person through your identity provider. Shared service accounts and generic bot users break the chain because they record that an agent acted, but not on whose behalf.

Approvals belong with the change, not in a meeting note or a chat thread. If the approval lives in a separate system, the link between the two must be explicit and checkable.

Make the Record Tamper-Evident

A log that anyone can edit proves little. Hashed version history and signed provenance let an auditor trust the record without trusting the person who wrote it. The SLSA framework is a useful reference: its first build level requires provenance to exist, and its second requires provenance signed by the build platform, so it can't be altered after the fact.

Keep the Trail From Request to Release

Supply-chain standards already ask teams to keep provenance for the software they ship. NIST's Secure Software Development Framework sets that baseline. AI-generated code starts the trail at the initial request, then goes through review, build, and deployment without interruption.

That is the core of AI governance auditing. The record is not a separate compliance project, but is produced as a side effect of how the work gets done.

Build Governance Into Generation, Not Only Review

Approval rules and provenance records show what happened. However, they don’t change what the AI coding tool produced. If it generates code that ignores your architecture, the approver in Tier 3 spends the review fixing it, and the audit trail faithfully records a change that should never have been written that way.

That is why governance is cheapest at the point of generation. When AI works from an engineering context layer, it starts from the company's approved patterns, libraries, service boundaries, and conventions. The approver then reviews a change that is already aligned with the business and can focus their attention on business risk rather than style and structure.

DesignVerse builds around this. It generates code inside your existing architecture and standards, from the repos, documentation, design systems, and test definitions you already have. Every prompt is saved as a frozen, hashed version that is traceable to a named user and can be reverted. Access runs through your own identity provider with role-based permissions, and the platform runs inside your infrastructure: private cloud, on-prem, VPC, or air-gapped.

What DesignVerse doesn’t do is remove the human from the loop. As we’ve seen, backend changes require human review before going live, and guardrails are effective but don't replace approval.

Conclusion

AI agents can do a large share of the work in regulated engineering, but they can't own it. Ownership stays with people: the requester who scoped the change, the approver who accepted it, and the team that runs it in production.

A tiered AI governance framework for AI-assisted software development lets teams give agents more work because they can show exactly what the agents did. If you want to see where your current process loses that trail, start by calculating how much engineering drift is costing you, then request a demo to see how DesignVerse can help you set governance guardrails in place.

FAQs

Who owns AI-generated code?

In practice, the organization that deploys it owns the responsibility, and a named human approver is accountable for each change. Legal ownership depends on the level of human contribution in the code and the jurisdiction, so it is a matter for your legal team to discuss.

What is AI governance auditing in software engineering?

The ability to show, for any change, where it came from, which AI tool and context produced it, and who approved it and when. The records must be complete and tamper-evident so an auditor can trust them.

Which AI-generated changes need human approval?

Every production change needs at least one human reviewer. Changes involving customer data, payments, access control, data migrations, or critical infrastructure require approval from a designated person not requesting the change.

How do you trace the origin of AI-generated code?

Keep track of who requested it, the tool used, task details, sources, and approval for each change. Store it as hashed, versioned history that runs from the original request through to release.

Does AI development governance slow delivery down?

Not when risk tiers apply at generation. Low-risk work moves fast with automated checks, and human review effort goes to the changes where a mistake is expensive.