An LLM + RAG pipeline cut per-quote extraction time by ~97% (30–40 min → under 1 min) and held ~99.9% accuracy, without taking the human out of compliance.
Enterprise-tier quotes averaged 3.2 days to turn around, not because analysts were slow, but because every quote demanded reconciling unstructured vendor PDFs against a structured pricing matrix, then routing for compliance sign-off. Doubling throughput meant doubling headcount. Neither the unit economics nor the hiring pipeline supported that.
Leadership wanted "AI to fix this." The instinct most PMs would follow: ship a model that generates the quote end-to-end. In a GRC product, that instinct is wrong.
I spent two weeks embedded with the quote team across three offices. The pattern was consistent: analysts weren't slow on decisions, they were slow on extraction, manually pulling SKUs, pricing tiers, and contract terms out of 40-page PDFs so a human could apply judgment.
That reframe changed the scope. We weren't automating pricing. We were automating data preparation, with the analyst still making the call and signing the quote.
The architecture: a document-ingest pipeline parsed incoming PDFs, RAG retrieved the relevant pricing-matrix rows from our product catalog, and an LLM structured the extracted data into a validated table. The analyst saw the table pre-filled, with confidence scores, field-level provenance, and low-confidence rows flagged for manual review.
Engineering wanted to start with a fine-tuned model. I pushed back: we had zero labeled data and a 6-week pilot window. RAG first, fine-tune later if the baseline needed it. It didn't.
This extraction engine is one slice of a much larger legacy-modernization effort, Phoenix. It shipped first, standalone, because it was the highest-leverage, lowest-risk piece of that program.
The obvious AI build is usually wrong. End-to-end automation demos well. In a compliance-bound domain, it fails adoption. Find the unglamorous middle of the workflow and automate that.
Confidence scores are a UX decision, not a model one. What threshold flags a row for review? Who owns overrides? How do analysts report a model mistake? Those were PRD questions, not eng questions.
The best metric was analyst-reported. "I can breathe now" from one of the leads mattered more than the throughput number. Culture-of-adoption is the leading indicator; throughput is the lag.