Frequentist vs. Bayesian

Frequentist testing to experimentation is the most commonly used hypothesis testing framework within industry, scientific and medical fields. Bayesian analysis requires a well-formed prior, which is information obtained from previous experiments.

Priors often require additional work from users who are often not trained to work with them. Or if you don’t have previous data, you can use your best guess, which is often inaccurate or biased. This is why the frequentist approach is usually preferred.

Another advantage of the frequentist system is that we can share our data clearly with customers to follow the work in a way that Bayesian analysis isn’t as easily replicated (especially at scale), and that we have been able to leverage improvements pioneered by industry leaders in product experimentation at companies like LinkedIn and Microsoft.