About CounterFact
CounterFact is a pre-A/B evaluation layer for recommendation and ranking teams. We use historical decision logs to estimate uplift offline and, just as importantly, tell you when the offline result is trustworthy.
Why we built it
Online experiments are the gold standard, but they are slow and expensive. Teams often spend weeks and valuable traffic testing candidates that were never likely to win, or that were not supported by the data. CounterFact helps you screen and prioritize candidates before they enter your experimentation queue.
How it works
You connect historical serving logs and a candidate policy (or generate one in the Playground). CounterFact runs off-policy evaluation and produces a clear recommendation for next action: proceed to an online test, pause and fix logging gaps, or collect targeted data to reduce uncertainty.
What you get
You get an offline estimate of expected impact, uncertainty measures, and practical diagnostics that explain whether the estimate is reliable. When available, CounterFact also runs robustness checks that stress assumptions and highlights where support or exploration is weak. Every run produces reproducible artifacts so results are easy to share and audit.
CounterFact works through a web interface and an API so you can start small without rebuilding your stack.
Try the Playground to run a readiness check, generate candidate files, and evaluate a policy end-to-end.