Highlights
Project snapshot
- Classifies 17,000+ cold-email replies into an 11-label taxonomy with Claude Haiku 4.5 at ~$0.20 per 1,000, with versioned, auditable prompts.
- Replaces three manual workflows — inbox triage, follow-up tracking, and lead sourcing — all driven from a spreadsheet-style NocoDB GUI with no engineer in the loop.
- Railway worker with a Postgres state-machine job queue (FOR UPDATE SKIP LOCKED), multi-provider enrichment, and automatic crash recovery.
Overview
Turning cold-email chaos into an auditable lead pipeline
Reply Intelligence is an end-to-end Python platform for a B2B cold-outbound agency that ingests every inbound email reply, classifies it with an LLM, and surfaces a single per-lead status to non-technical staff through a spreadsheet-style GUI (NocoDB).
Beyond classification, it replaces a hand-kept follow-up tracker with a self-maintaining database view and lets operators self-serve fresh, enriched leads through a job-queue worker — so the whole lead lifecycle runs without an engineer in day-to-day operation.
Problem
Manual triage, tracking, and sourcing didn't scale
- With 17,000+ replies accumulating, reps read every message by hand — hot leads slipped through and unsubscribe requests (a compliance risk) were missed.
- Follow-up history was a hand-maintained spreadsheet: tedious, error-prone, and always stale.
- Sourcing fresh decision-maker leads, enriching them, and loading them required a developer every time.
- There was no single, trustworthy view of where each lead actually stood.
Solution
One automated, auditable platform behind a familiar GUI
- Incremental, idempotent sync from Instantly into Postgres, then batch LLM classification into a fixed 11-label taxonomy with a plain-English rationale per reply.
- A self-maintaining follow-up tracker view plus a self-serve lead-sourcing job queue, all surfaced through NocoDB grids and forms.
- Human manual overrides are never touched by automation, and every prompt change is version-stamped for an auditable history.
Core Features
Four subsystems across the reply-to-lead lifecycle
Reply Classification Engine
- Cleaning + rule-based promo/auto-reply filtering before the model
- Batch classification by Claude Haiku 4.5 into 11 labels with rationale
- Priority-ranked per-lead rollup preserving manual overrides
Follow-up Tracker
- Outbound sync discriminating campaign vs. manual sends
- Hybrid long-form + wide-pivot materialized views (uncapped history)
- Auto-populated "last reply from Instantly" column
Lead Sourcing Automation
- State-machine job queue (pending → running → ready → moved)
- Two-stage rules + Haiku LLM filtering with a credit-budget cap
- Per-lead approval grids, mass-approve shortcut, and Resend notifications
Multi-Provider Enrichment
- Apollo firmographics and BetterContact phone enrichment
- SmartScout brand market data with RapidFuzz + Haiku resolution
- LLM company-name reconciliation across sources
Architecture
Sync, classify, and materialize — surfaced through NocoDB
Instantly API
→ instantly_sync.py (incremental, idempotent upserts)
→ replies (Supabase/Postgres)
→ classify.py (Claude Haiku 4.5, batched)
→ classifications
→ leads_status_update.py → leads + materialized views
→ NocoDB GUI (non-technical operator)
- A single orchestrator (run.py refresh) chains sync → refresh-status → classify → update-status and aborts on any failure.
- A separate Railway worker polls a Postgres job queue every 60s with FOR UPDATE SKIP LOCKED and crash recovery; direct psycopg2 handles ops PostgREST can't (REFRESH MATERIALIZED VIEW, DDL).
Technical Highlights
Versioned ML, resilient sync, and verified integrations
- Versioned, auditable classification: old and new prompt versions coexist so the newest wins while regressions stay diff-able, behind a stratified hand-review accuracy gate (>90% target).
- Idempotent and resilient: unique-key upserts, watermark-driven incremental sync, exponential backoff on rate limits, and retry-with-fresh-connection around flaky 230k-row scans.
- Empirically-verified integrations: undocumented Instantly and Prospeo API behaviors (ue_type semantics, the correct ?lead= param, filter shapes and silent caps) nailed down with dedicated probe scripts.
- Cost-efficient LLM use: ~$0.20 per 1,000 replies on Haiku with prompt caching, plus a Haiku-vs-Sonnet bake-off harness for quality benchmarking.
Business Value
Three manual workflows replaced by one operable system
- Every reply is now classified and rolled up to a single source-of-truth lead status, with full human override preserved.
- Follow-up history maintains itself, and non-technical staff source, review, and load enriched leads end-to-end.
- Processes 17,000+ replies at ~$0.20 per 1,000 with a versioned audit trail throughout.