← All work  ·  Case 04 of 08 · POC · Completed

Finding the studies is easy. Extracting the evidence is hard.

Researchers review dozens of clinical trials to support treatment decisions. The bottleneck isn't search; it's standardizing buried data like participant counts, study designs, and conflicts of interest. We built a POC to automate the extraction while enforcing strict human oversight.

Context & Stack
Claude on AWS BedrockClinical Evidence ToolingJSON / CSV Pipeline2026 · POC build
Impact · Evidence Extraction POC
~2 days
Review cycle time
vs. 1-2 weeks manually
JSON
Strict schema format
Converted to CSV for analysts
HITL
Side-by-side verification
PDF linked to extracted data
COI
Missed buried claims
Model struggled with citations
Chapter 01Problem space

Unstructured data in, critical product decisions out.

Healthcare providers often need to review massive volumes of medical literature and clinical trial data to support product and treatment decisions. The process is highly specialized, time-consuming, and difficult to scale.

The challenge isn't finding the studies. It's extracting and standardizing the information. A researcher might pull 50 to 100 abstracts. Inside each one, key data points - study design, participant count, level of evidence, outcomes, conflicts of interest, and trial status - are buried in different formats and locations. Manually parsing this into a structured table takes a researcher 1 to 2 weeks of grueling clerical work.

Chapter 02Architecture

Forcing strict JSON schemas for medical literature.

We built an extraction POC using Claude on AWS Bedrock. The pipeline ingested the raw PDFs/abstracts and was prompted to output a strictly enforced JSON schema corresponding exactly to the fields the researchers needed. Once the JSON was generated and validated, it was flattened into a CSV file - the exact format the clinical analysts were already accustomed to working in.

Clinical Trials

Raw PDFs

AWS Bedrock

Claude Extraction

Enforce Schema
Strict JSON

Flatten to CSV

Side-by-side UI

Verify vs PDF

Approved

Assessment

01

Ingestion

Raw medical literature and trial PDFs are fed into the Bedrock pipeline.

02

Strict JSON

Claude extracts participant counts and trial statuses into a locked JSON schema.

03

CSV Export

The validated JSON is compiled into a master CSV for the researcher to review.

Chapter 03Interface & Workflow

Trust, but verify with side-by-side UI.

In clinical evidence, you cannot blindly trust a language model's extraction. We designed a side-by-side Human-in-the-Loop (HITL) interface. The researcher views the original PDF on the left and the AI-extracted table on the right. This allows for rapid validation without context-switching between windows. The estimated time to review a batch of abstracts dropped from 1-2 weeks down to roughly 2 days during the POC phase.

Chapter 04The Honest Take

What the model struggled with.

The POC was highly successful for structural data like participant counts, but it wasn't flawless. We identified several key failure modes during testing:

  • Deeply Buried COI: Conflicts of Interest (COI) that were subtly phrased or buried in footnotes were occasionally missed by the model.
  • Complex Study Designs: The model got confused when parsing highly complex, multi-arm study designs, sometimes hallucinating the control group parameters.
  • Citations: Adding proper inline citations back to the source text proved mechanically difficult and brittle.
Chapter 05The Conclusion

The report-generation POC we stopped on purpose.

After successfully extracting the structured data, the natural next question was: Can we use this to auto-draft the final clinical assessment report?

We ran a feasibility test with a strict anti-fabrication rule (any unverifiable claim becomes a [VERIFY] placeholder). The result? The custom pipeline flagged 71 claims for verification, while a general-purpose AI flagged 26.

I stopped the POC there. "Longer and more flags" is a description, not a verdict. Without predefined clinical pass criteria to measure against, auto-generating the final report was crossing the line from controlled automation into unsafe territory. The recommendation stood as the handoff: stick to the extraction pipeline, and leave the judgment and drafting to the clinical analysts.

Chapter 06Lessons

Extraction is a UX problem, not just an ML problem.

The failure modes (like missing a COI) reinforced exactly why the side-by-side verification UI was necessary. If the UI didn't make it effortless to verify the model's work against the source document, the time saved in extraction would have been lost to the anxiety of manual auditing.

Next case

B2B Checkout

Cutting cart abandonment 42% through targeted interventions.

→

Let's build what actually ships.

krishnashemanth@gmail.com LinkedIn →