ThreadVerdict
● LiveAn AI engine that reads Reddit and ranks the products people actually trust.
- Role
- Solo founder-engineer (at Revyve SEO)
- Period
- 2025 – present
Product recommendations on the open web are ad-poisoned; the honest opinions are buried in Reddit threads. ThreadVerdict reads those discussions at scale — every comment classified with full thread context — and ranks products with a scoring methodology rigorous enough that I published it. Live and monetized via Amazon Associates.
The pipeline
Six stages, each with a job to refuse bad data:
- Catalog — build the product universe from major stores across 100+ categories.
- Ingestion — pull the Reddit threads where each product is genuinely discussed (PRAW, scheduled).
- Classification — an LLM reads every comment with full thread context. Four labels (positive / negative / mixed / neutral); only genuine owner experiences count; name-drops without an opinion are excluded.
- Scoring — the Verdict Score (below).
- Ranking — products ranked within category; thin categories are flagged, not faked.
- Quality checks — duplicate authors collapse to one vote; sibling-model confusion (reviews of the wrong product generation) is caught and removed; refresh runs keep scores current.
The Verdict Score — ranking math you can defend
A 0–5 score built from two weighted factors: Approval (60%) — how positive the opinions are, with a small-sample adjustment so two glowing comments can't outrank two hundred mixed ones — and Acceptance (40%) — the share of mentions without complaints.
Both factors are deliberately capped (0.95 / 0.92): no real product is complaint-free, so a perfect 5.0 is unreachable by design. Products with fewer than 8 opinionated mentions show 'Limited data' instead of a score — refusing to rank is a feature.
Keeping the LLM honest
Classification quality is enforced three ways: structured outputs with schema validation (malformed responses are rejected and retried), cross-check heuristics before anything ranks (mention thresholds, dedup, sanity rules), and manual spot-checking of samples. There's no formal eval harness yet — that's the top of the roadmap: a labeled ground-truth set, classifier accuracy tracking, and drift monitoring as models change.
Published limitations
The methodology page documents what the system can't do: Reddit's demographic sample bias, low-volume categories, temporal drift (sentiment shifts after a firmware update or recall), and sarcasm/context edge cases. Publishing the failure modes was a product decision — trust is the product.