Skip to content
Jamie Blair

Illustrative design exercise · not a client case study

Design exercise: a trustworthy invoice-processing workflow

4 min read

An illustrative architecture exercise for invoice extraction and reconciliation, with human review, validation, observability, and production-readiness boundaries.

An abstract illustration of invoice documents moving through a structured validation workflow.

Note

This is an illustrative design exercise, not a client case study. The organisation, volumes, outcomes, costs, and performance figures used in earlier versions have been removed because they were not appropriate as public delivery evidence.

Invoice processing looks simple until formats, scan quality, tax rules, duplicate submissions, and incomplete purchase-order references meet in the same queue. The valuable engineering work is not merely extracting text. It is designing a workflow that makes uncertain data visible and keeps financial decisions reviewable.

This article describes how I would reason about that problem. It deliberately avoids claiming a real customer deployment or a universal accuracy figure.

1. Define the decision before the extraction

The first question is not “Which model should we use?” It is “What decision will this data support?” A useful discovery phase would identify:

  • the fields downstream systems genuinely require;
  • the tolerance rules around tax, totals, dates, and references;
  • which mismatches can be resolved automatically;
  • which cases require a human decision;
  • what evidence an auditor or operator needs later.

Those answers become an explicit schema and acceptance criteria. Without them, a pipeline can appear impressive while silently producing data nobody should trust.

2. Separate extraction from validation

A sensible workflow would use the least complex reliable method for each document:

  1. Extract text and layout from machine-readable PDFs.
  2. Apply known rules for stable fields and formats.
  3. Use OCR only where the source is an image or scan.
  4. Use a constrained model fallback for genuinely irregular content.
  5. Normalise every path into the same typed output schema.

The extraction method should never be treated as proof that a value is correct. Validation happens separately. Line items should reconcile to subtotals; tax and subtotal should reconcile to the total; currency and dates should be normalised; identifiers should be checked against known records where permitted.

3. Make uncertainty operational

A trustworthy system needs more than a confidence number. It needs a review policy. For example, a document could enter human review when:

  • a required field is missing;
  • totals do not reconcile;
  • the supplier or layout has not been seen before;
  • OCR quality falls below an agreed threshold;
  • a potential duplicate is detected;
  • a reference cannot be matched with sufficient confidence.

The review interface should show the source document, extracted values, validation failures, and provenance together. The goal is to help a reviewer decide quickly without hiding why the system was uncertain.

4. Design for traceability

Every processing attempt should have a correlation identifier and a structured record of:

  • the source and receipt time;
  • extraction method and version;
  • validation results;
  • manual changes and reviewer identity;
  • final disposition;
  • downstream delivery status.

Sensitive document content should not be copied into general application logs. Logs need enough context to investigate failures without becoming a second uncontrolled data store.

5. Prove the workflow before production

A prototype can demonstrate extraction and validation, but production readiness is a separate claim. Before live use, I would expect work across:

  • identity and least-privilege access;
  • retention and deletion policy;
  • encryption and secret management;
  • monitoring, alerting, and operational ownership;
  • regression datasets that represent difficult formats;
  • failure recovery, replay, and rollback;
  • data-processing and supplier governance;
  • an agreed human-review and exception process.

Parallel evaluation against representative historical documents can reveal where the design is genuinely useful. Results should be reported with the dataset, method, exclusions, and confidence intervals rather than as a context-free headline accuracy percentage.

The broader lesson

Document automation is primarily a data-quality and workflow problem. Models can help with irregular inputs, but confidence comes from validation, traceability, clear review boundaries, and honest production-readiness criteria.

Key Takeaways

  • An output schema and acceptance criteria come before model selection.
  • Extraction and validation are separate responsibilities.
  • Human review is a designed path, not an admission of failure.
  • Prototype evidence must not be presented as a production deployment.

If you want to discuss the engineering trade-offs in this design, use the private message form.

Related reading: