The problem
Inbound leads arrive as call transcripts, SMS and form submissions, at a volume that makes manual review impractical. Each one needs three judgements: did a human answer, is this a genuine new-business enquiry, and is it spam.
The existing system did this inside the automation platform. It had grown to 297 modules and twelve levels of nesting, with 33 separate GPT modules — one for extracting a name, another for an address, another for job type, each with its own JSON parse, database update, audit insert and error commit. The exported blueprint was 9.3 MB.
Nobody could safely change it. Not because the logic was wrong, but because there was no way to see the whole thing at once, no way to test a change without running it against production data, and no way to know which of the 33 model calls a given lead would trigger. The cost of a small edit had become unpredictable, which is the point at which a system stops being maintainable regardless of whether it currently works.
What I built
A Flask service on a VPS, with the automation platform reduced to a dispatcher: pull a batch of 50 unanalysed transcripts from MySQL, POST each to the service, write the classification back.
The service is deliberately rules-first. Several hundred deterministic conditions handle the cases that can be decided from the transcript itself — Google LSA message formats, voicemail greetings, IVR menus, business-verification spam patterns, the structural difference between a transcript with speaker tags and an SMS without them. The language model is called only where the rules cannot reach a decision, and it answers a narrow question against a fixed schema rather than being handed the whole problem.
That ordering is the main design decision, and it runs against the obvious approach. Posting every transcript to a model would have been a fraction of the code. It would also have cost per lead at a volume of thousands per day, added a network round trip to every classification, made the same input capable of producing different answers on different days, and left no way to explain a decision to the operations team beyond quoting the model back at them. Rules are free, instant, deterministic and auditable. The model earns its place on the cases where a rule genuinely cannot be written.
| Before (V13) | After (V19) | |
|---|---|---|
| Modules in scenario | 297 | 40 |
| GPT modules on canvas | 33 | 0 |
| Blueprint size | 9.3 MB | 0.2 MB |
The classification logic did not disappear — it moved somewhere it can be read in a single file, version controlled, and run against a labelled dataset without touching production.
Results
- 94% agreement with manual review, measured across 1,000 randomly sampled production leads validated by hand.
- In production since 1 July 2026. 47,265 executions in the first 63 days, of which 47,264 succeeded — a single non-fatal warning across the whole period.
- Roughly 1,000 leads per weekday (median 1,006 executions, peak 1,682), falling to about 60 at weekends in line with inbound call patterns. Median classification latency 2.4 seconds, 95th percentile 6.3 seconds.
- Model calls reduced from 33 distinct invocation points to a fallback path used only where deterministic rules cannot decide.
- Classification logic became testable. The previous version could only be validated by running it.
What I would change
The service's own authentication is weaker than the boundary it sits behind. The API key is accepted from a query parameter as well as a header, which means it can end up written to access logs, proxy logs and anything else in the request path — a credential leaking into files nobody classifies as sensitive. The comparison is also a plain string equality rather than a constant-time one. Neither has caused a problem, and the service is not internet-facing in a way that makes the second interesting, but both are trivially fixable and there is no good argument for leaving them. Header-only, and secrets.compare_digest.
A testing default shipped. The rate limit delay is set to zero, with a comment noting the strict value is thirteen. That was the right setting for development and nobody changed it on the way out.
The 6% needs characterising. A random sample validated by hand is the right way to measure this, and 94% is a real number rather than a fitted one. But knowing the rate is not the same as knowing the shape: whether the misses cluster in one lead source, whether they fall on the rules or on the model fallback, and whether they skew towards false positives or false negatives. Those have very different costs — a genuine lead wrongly disqualified is lost revenue, while spam wrongly verified only wastes someone's time. The same 1,000 labelled examples would answer this without collecting anything new.
The retry loop is in the wrong place. Failed calls to the service retry through a chain of nested error handlers on the canvas, each one a sleep followed by a repeat of the same request — a retry loop with backoff, unrolled by hand into a diagram.
It works, and the logs show it working: the single non-successful execution in 47,265 consumed 24 operations against a median of 8 and ran for 506 seconds against a median of 2.4, which is the chain sleeping and retrying its way through an unresponsive service before degrading to a warning rather than dropping the lead. The design is sound and the outcome was correct.
The objection is to where it lives. It is three lines of Python in a client that already exists, and the visual version cannot have its backoff curve adjusted, its jitter tuned, or its behaviour unit tested without redrawing the diagram. This is a small illustration of the general point: the dispatcher belongs in a low-code tool, the logic does not, and retry handling sits on the wrong side of that line.
Several hundred hand-tuned string conditions are a maintenance liability. They are fast, free and explainable, which is why they are there. They are also brittle against lead sources that did not exist when they were written, and each new source risks quietly matching an existing rule rather than falling through to the model. A regular check of how often the fallback fires would show that drift early — a rising fallback rate means the rules are covering less than they used to, and a falling one means new inputs are being absorbed by rules that were never meant for them.