Autonomous AI Testing Agent
InternalAn agent that watches developer changes and writes the tests — now a team's standard workflow.
- Role
- Architect & builder (at Techloyce)
- Period
- 2024 – present
SubscriptionFlow is a SaaS billing platform serving hundreds of businesses, mid-migration from Laravel to Next.js — a huge, moving test surface no manual QA process could keep up with. I architected and built an agent that watches developer changes and generates the tests: PHP backend suites, Next.js frontend tests, and Playwright end-to-end scenarios, for new features and legacy code alike.
What it does
The agent triggers on developer changes (PRs/commits), analyzes the diff in the context of the codebase, and produces three layers of coverage: backend tests for the PHP/Laravel side, component and integration tests for the Next.js frontend, and Playwright E2E scenarios for user-visible flows. It backfills legacy code as well as covering new work — which mattered enormously during a platform migration, where regressions hide in the code nobody touched.
The trust model — how we know the tests are good
An agent that writes bad tests is worse than no agent: green suites that assert nothing create false confidence. Three gates keep it honest:
- CI-gated: every generated test must execute and run green before merge — broken or flaky generations surface immediately.
- Human-reviewed: generated tests go through code review like any other code; developers reject weak assertions.
- Failure-driven prompt iteration: every rejected or flaky generation feeds back into the prompt design. The generation quality today is the residue of every failure analyzed.
Adoption
The agent is the team's standard workflow — not a demo, not a pilot. That's the metric I care about: developers with deadlines choose to rely on it.
Also in production at Techloyce: MCP servers
Alongside the agent, I shipped Model Context Protocol servers for the platform's enterprise customers — including a ChatGPT plugin and an embeddable widget that lets end-customers manage their subscriptions, invoices, and profiles from an AI client after authentication. Same thesis as the agent: AI interfaces to production systems, with auth and correctness treated as first-class.
What I'd add next
Formal quality metrics for generated tests: mutation-testing scores to measure whether suites actually catch injected faults, flake-rate tracking per generation strategy, and a labeled corpus of rejected generations to eval new prompt versions against before rollout.