Technology
Human-in-the-Loop
Review as a control, not a failure state: thresholds, queues, escalation and the feedback that improves the next run.
The concept
How review works inside the platform
A threshold that holds an item, a queue that owns it, a card that shows the evidence and a record that keeps the answer. The human-in-the-loop guide covers how to design them.
Every item in DocPro and FinSuite carries what is needed to decide whether a person should see it: a confidence score on each field, the result of each validation rule, the value band the invoice falls in, and any policy hold that applies to this vendor. Thresholds and rules read that information as the item moves. When one of them says stop, the item is held at that step and only the part that failed goes to a person. The rest of the record is not re-read or re-keyed.
The held item lands in a queue with an owner, a backup and a clock. Queues are separated by cause, because a misread field, a failed match and an amount above a band need different people. The reviewer gets a card with one thing to settle: the field and the value read, the alternatives considered, the crop of the page it came from and the check that failed. They can accept, correct or reject with a reason. They cannot change the threshold or the rule from the queue, because that is a separate change with its own approval.
Whatever the reviewer decides is kept. The correction is logged against the field, the document, the vendor and the reviewer, with the cause tagged. It completes the item, which carries on down the same pipeline an unattended item would. It also becomes evidence: a labelled example for the next model version, a candidate rule when the same correction keeps recurring, and a count that shows whether a threshold is set higher than it needs to be. How to choose triggers, size queues and measure review load is the subject of the guide. This page stays with the mechanism.
Where it sits
Review across the five stages
PTAS AI workflows run as Perception, Intelligence, Orchestration, Action and Verification. Review belongs to the last stage, but every earlier stage decides what reaches it.
In the products
What review looks like in DocPro and FinSuite
In DocPro a reviewer settles fields. In FinSuite a reviewer resolves exceptions, and an approver exercises authority. They are different jobs and the products keep them apart.
DocPro
Field review, not document review
A held invoice shows the reviewer row nine and the reason it stopped. The thirteen fields that cleared their thresholds are not put in front of anyone again.
DocPro
The image beside the value
The page region a value came from sits next to it, enlarged enough to read, so the reviewer rarely needs the original. One click opens it when they do.
DocPro
Type and boundary review
A page whose document type, or whose boundary inside a bundle, could not be settled is shown with the pages around it. The reviewer names it and extraction starts from there.
FinSuite
Exception lanes
Price variances, missing GRNs, GST mismatches and suspected duplicates each have a queue and an owner. The reviewer sees the documents side by side with the difference already computed.
FinSuite
Approval as a separate gate
An invoice above a band goes to the approver the policy names, with the checks it passed listed. Approval is authority, a different decision from correcting a field, and it is recorded as one.
FinSuite
Fix once, re-run many
When one cause holds twenty invoices, an inactive vendor GSTIN for instance, the reviewer resolves the cause once and the twenty are run through the checks again.
Worked example
Where held items go, and what a reviewer sees
One morning's held items sorted into queues by cause. Then the card for one of them, a handwritten quantity, from the moment it was held to the moment it was settled. Values are illustrative.
Where held items went this morning. Illustrative example
The person who can read a scan, the person who can call the supplier and the person with authority are rarely the same person. The queues follow that.
Held by cause
The threshold, rule or policy that stopped an item decides which queue it lands in.
Every queue has a name
An owner, a backup and a clock, so nothing waits in a shared inbox.
Sampled, not only flagged
Five items that passed cleanly were pulled for checking, which is how drift is caught.
Take one of the fourteen in document review. The Invoice Agent read row nine of a two-page invoice, found a handwritten figure written over the printed quantity, and scored the field below its threshold. The other thirteen fields on that invoice cleared theirs. This is what the reviewer was given.
The review card for one held field. Illustrative example
The reviewer settled one value with the evidence beside it. A screen that makes them re-read the whole invoice turns review back into data entry.
One thing to settle
The card shows one field and why it stopped. The rest of the invoice is not the reviewer's problem.
Evidence on the card
The crop, the alternatives and the PO line were there, so the PDF stayed closed.
The answer is kept
The correction, its cause and the reviewer went onto the record, and into the next model round.
Limits and controls
Where review stops being a control
Review works when the person has the evidence and the authority for the decision in front of them. It fails in predictable ways. A queue with no owner ages quietly. A threshold set too high sends people fields the model already had right, and they learn to accept without looking, which is worse than no review because it still looks like control. A reviewer asked to settle a field from an image that is genuinely unreadable can only guess, and the right decision there is to reject and ask for the document again.
We watch for these from the first week, with the process owner: how often reviewers change what was proposed, by field; how old the oldest item in each queue is; how long each decision takes. The guide explains how to read those numbers and what to change when they drift.
Thresholds per field and value
Set per field and document type, stricter where the amount is larger. Changing one is a versioned rule change made outside the queue.
Queues by cause, with owners
Reading, validation and approval holds go to separate queues, each with a named owner, a backup and a clock that escalates.
Evidence on the card
Value, alternatives, image crop, failed check and reference documents are shown together, so a decision never depends on hunting for the original.
Every decision recorded
Accept, correct, reject and escalate are events with a reviewer, a time and a cause, kept on the item and open to audit.
The thresholds and rules that hold items are covered under Business Rules. The audit of every hold, decision and override is under AI Security and Governance.
FAQ
Questions teams ask about review inside the platform
Does every document go to a person at first?
No. An item goes to a person when a threshold, a rule or a policy says it should. In the first weeks thresholds are usually set conservatively, so more items reach people while the scores are measured on your documents. They are moved on evidence, mostly how often reviewers change what the system proposed, and each move is a versioned rule change.
What does a reviewer actually see?
One card per held item. It shows the field and the value read, any alternatives the model considered, an enlarged crop of the page region the value came from, the check that failed or the threshold it missed, and any reference document that bears on it, such as the PO line. The reviewer can accept, correct or reject with a reason, or escalate. The original document is one click away, and most decisions do not need it.
Can a reviewer override a rule or approve above their authority?
No. A reviewer can change the data on an item and decide on the item. They cannot change the threshold or rule that held it, and they cannot approve an amount above the band their role allows. Those are separate permissions. Approval goes to whoever the policy names for the band, cover applies through delegation rules with a date range, and every decision records who acted under whose authority.
Do reviewer corrections change the model straight away?
No. A correction completes the item immediately. For the model, it is kept as a labelled example and used in the next training or calibration round, which is tested against the current version before release. Where the same correction keeps recurring for the same cause, it is proposed to the rule owner as a rule instead. Nothing changes the running system without going through that test.
Keep reading
Where review connects
Human-in-the-Loop AI guide
How to design triggers, thresholds, sampling and queues so review scales.
Read more →Business Rules
The thresholds and checks that decide when an item stops.
Read more →Machine Learning
How reviewer corrections become labelled examples for the next model version.
Read more →DocPro Invoice Agent
Row-level review on supplier invoices, with the field and the reason shown.
Read more →Next step
Size the review queue before go-live
Bring a week of one document type and your approval policy. We will show which items would stop, at which step, for which person, and what each reviewer would see on the card.