Manual-review reduction pipeline
Keyword rules on free-text notes flag cases for manual review. Many of those flags are false positives that cost reviewer time.
The pipeline finds those false positives and delivers them to operations and the automation team every day, so the cases are cleared without reaching a reviewer. I built all of it, including the matching engine at its core. Each new rule goes out through one tested, automated release path.
- Role
- End-to-end owner
- Context
- Decision automation
- Stack
- PythonSQLBigQueryAirflowKubernetesFastAPIGitHub Actions
- delivery of wrongly flagged cases to operations and the automation team
- Daily
- heavy matching, run as its own service so one failure cannot stop a run
- Isolated
- archival, correct even when a run is repeated
- Exactly once
- promotion from a Data Science rule to production, behind environment guards
- Automated
This is a public summary. Figures and internal specifics from CVS Health/Aetna are left out; I am glad to go into more depth in conversation.
The system
- PipelineBuilt and owned by me
Matching engine, run as an async service
The heavy note matching runs as an async Kubernetes service that Airflow starts and polls, with progress tracked stage by stage, so one long-running failure cannot take down the whole run.
- Pattern extraction, fuzzy entity matching with edit-distance limits, and a three-way answer
- I implemented the matching engine as well as the service around it
What I built
- Staging SQL
- Airflow workflow
- Matching engine
- Archival
- Daily delivery
The full pipeline, from note matching to daily delivery to the business and automation teams: staging SQL, an Airflow workflow, an async matching service on Kubernetes, exactly-once archival and file delivery.
I was the end-to-end owner and wrote most of the code in both of its repositories, the workflow and the service.
The matching engine
- Matching engine
The core of the pipeline is a text-matching engine that I implemented as its own service. It reads the free-text notes on each flagged case, pulls out the references in them with pattern extraction, and checks whether the entity a note is about is the entity on the record.
Notes are written by people, so the comparison is fuzzy, with a limit on edit distance for each token. Each comparison ends in one of three answers, yes, no or uncertain, which is a different question from whether a keyword appears. Only cases that can be safely released leave the queue.
The rules come from Data Science. I turned each one into code and a tested release, and reconciled production output against reference runs.
One failure should not take down a run
- Airflow workflow
- Matching engine
The heavy note matching runs as an async Kubernetes service that Airflow starts and polls, with progress tracked stage by stage. One long-running failure cannot take down the whole run.
Making change cheap
- Staging SQL
- Airflow workflow
- Promotion workflow
- Matching rules
Async stages and tuned SQL brought the runtime down.
An automated, environment-guarded promotion workflow shortened the path from a new Data Science rule to production. It became the team's reference pattern for new projects.
Working with the people who use it
- Matching rules
- Promotion workflow
- Business operations
- Automation team
I led the weekly sync with business operations, Data Science and the automation team, and turned each new Data Science rule into a tested production release.