Skip to main content
Product note

Why "human in the loop" is the only viable model for AI in regulated clinical trials

Autonomy and compliance are not opposites, but getting both requires an architecture that most AI tools have not built. Here is what the compliant version looks like.

The phrase "human in the loop" has become the clinical AI industry's equivalent of "responsible AI": a phrase that sounds principled, is almost universally claimed, and is almost never defined with enough precision to be evaluated. Every AI platform targeting regulated industries says it keeps humans in the loop. Most of them mean that a human can, if they choose to, look at what the AI produced before it is committed. That is not the same thing.

The distinction matters because 21 CFR Part 11 and ICH E6(R3) both ask who is accountable for a record an AI helped create. Section 11.10(e) requires a secure audit trail of the actions that create or change an electronic record. Section 11.10(g) limits altering or signing a record to authorized individuals. E6(R3) section 3.9 keeps sponsor oversight on the process and the data, and section 4.2.2 describes the audit trail. "Can look at" is not the same as a documented review. The architecture that enforces the review is what separates a human-in-the-loop system from one that merely claims to be.

Two failure modes

There are two ways to get human-in-the-loop wrong in clinical AI, and they are equally bad.

The first is too much autonomy. The AI commits outputs to regulated records without documented human review. The audit trail shows what the AI produced but not whether a qualified person reviewed and accepted it. That misses §11.10(e) if the action that created the record is not in the trail, and §11.10(g) if no authorized person is attributable for the change. It misses E6(R3) section 3.9 if sponsor oversight of the data process is not documented, and section 4.2.2 if the audit trail cannot show who acted and when. An inspector asking "who approved this SDTM mapping?" gets back "the algorithm did", which is not an acceptable answer.

The second is human-as-rubber-stamp. The AI produces outputs at a volume and pace that makes genuine review impossible. The DM lead clicks Approve on 400 mapping proposals in an afternoon because the alternative is a bottleneck. The audit trail shows human approvals on everything, but the human approval is meaningless: it records presence, not review. This satisfies the letter of Part 11 while violating its intent, and in a genuine regulatory inquiry it does not hold up.

A genuine human-in-the-loop architecture avoids both failure modes. It structures the AI's output so that genuine expert review is possible and efficient, not so fast that it becomes rubber-stamping, not so slow that the DM lead becomes the bottleneck that defeats the purpose of automation.

What the architecture actually looks like

A compliant human-in-the-loop pipeline has four structural properties:

Proposals, not commitments. The AI produces a proposal. The proposal enters a Decision Queue. By default, nothing in the proposal is committed to any clinical artifact until a qualified person explicitly approves it. Auto-approval exists only as an explicit, tenant-configured policy, scoped by artifact type and confidence threshold, logged as a policy decision with its own audit trail, and revocable at any time. There is no silent "trust the AI" default. The queue is the gate, and the gate is the compliance boundary.

Confidence-stratified review. The volume problem is solved by routing, not by compression. High-confidence proposals with clear rationale are presented for confirmation: a qualified person reviews the reasoning and the code, confirms they are correct, and approves. Low-confidence proposals, flagged edge cases, novel domains, and derived variables are presented for substantive review: the qualified person reads the proposal, evaluates the reasoning, and either approves or revises. The DM lead's time goes to the cases that require their expertise. The mechanical confirmation work, though still requiring human touch, is structured so that it takes seconds per item rather than minutes.

Revision capability with attribution. When a DM lead disagrees with an AI proposal, they need a way to revise it, and the revision has to be logged with attribution. The audit trail shows what the AI proposed, what the person changed it to, who made the change, and when. A system where the person can only approve or reject forces a binary choice that does not reflect how expert review actually works. The revision is how section 4.2.4's attributed correction shows up in the record.

Immutable record of the exchange. The full exchange (AI proposal, confidence score, source attribution, human action, revised value if applicable, approval timestamp, approver identity) must be recorded immutably. Not as a log that can be updated, not as a soft delete that preserves the record but allows amendment. Immutable: append-only, with the full history of every decision point recoverable in sequence.

The regulatory consensus

The human-in-the-loop requirement is not a TrialNexus design choice. It is the converging position of the major regulatory frameworks that govern clinical trials.

21 CFR 11.10(e) requires a secure, computer-generated, time-stamped audit trail of the actions that create, modify, or delete an electronic record. Section 11.10(g) limits altering or signing a record to authorized individuals. ICH E6(R3) section 3.9 keeps the sponsor responsible for the quality of the processes and the data, and section 4.2.2 describes what the audit trail shows. The FDA's May 2023 discussion paper on AI/ML in drug development is not a guidance. It discusses human-led governance and accountability. It does not set a qualified-person rule for AI outputs.

None of these frameworks prohibit AI from participating in clinical data management. They establish that AI participation must be supervised, attributed, and documentable. The supervision is the human in the loop. The attribution is the DM lead's electronic signature on the approved output. The documentation is the audit trail that an inspector can follow without a guide.

Why this produces better outcomes than autonomy

The case for human-in-the-loop is not just regulatory. It is practical. An SDTM mapping that a DM lead reviewed and confirmed is a better mapping than one the AI produced autonomously, not because the AI was wrong, but because the DM lead caught the three cases where it was. The Decision Queue is not only a compliance mechanism. It is a quality control mechanism. The AI's proposal narrows the solution space dramatically. The DM lead's review catches the edge cases the AI missed. The combination produces a higher-quality output than either alone.

The study that runs in 12 days rather than 68 is not a study where the DM team did less work. It is a study where their work was concentrated on the decisions that required their expertise. The 80% of the build time that was translation work (protocol to CRF, CRF to SDTM, SDTM to Define-XML) ran as an AI-orchestrated pipeline. The 20% that required judgment, domain expertise, and professional accountability ran as it always has: with qualified people making decisions and signing off on them.

The DM lead who used to spend her day converting PDFs to Excel now spends it reviewing agent proposals, catching edge cases, and defending her decisions to stakeholders. She has not been replaced. She has been given back the part of her job that required her.

The question to ask

The practical test for any AI clinical trial platform: can the AI commit an output to a clinical data artifact without an accountable human decision behind it? If automated commitment exists, it must itself be a human decision: an explicit, documented policy choice by a qualified person, with scope, thresholds, and an audit trail showing who authorized it and when.

The AI proposes. A human approves: either each output directly, or the narrow, documented policy that commits routine outputs automatically. The record shows both. That is the architecture that survives a regulatory inquiry and delivers the value of AI automation without incurring the compliance risk of unaccountable clinical data management.

The gate is one piece of the system. What agentic AI means for a data management team is the workflow in front of it. What ICH E6(R3) requires is the oversight standard. The Part 11 checklist is the records test. SDTM mapping is where the review time actually goes.