The Bayes Factor: How the Ratio of Marginal Likelihoods Quantifies Evidence for Competing Hypotheses

To quantify the exact weight of evidence for a scientific claim, a researcher must first mathematically define what that claim predicts before collecting a single data point. This is the binding constraint of the Bayes factor. If a hypothesis is so vague that it can accommodate any possible observation, its marginal likelihood collapses. The system only works when competing theories are forced to place bets on specific outcomes in advance.[8]

For most of the 20th century, statistical inference bypassed this constraint entirely. The dominant frequentist approach, anchored by the p-value, evaluates only a single hypothesis—typically the 'null' assumption that an effect does not exist. If the data looks sufficiently anomalous under the null, the null is rejected. The alternative hypothesis wins by default, without ever having to prove its own predictive accuracy.[4]

But as Alexander Etz notes in his analysis of statistical evidence, rejecting the null is not the same as proving the alternative. "The Bayes factor tells us how much our beliefs should change," Etz writes, distinguishing the weight of evidence from the final conclusion. A p-value cannot measure evidence for the null hypothesis; it can only fail to reject it, leaving researchers unable to distinguish between a genuinely zero effect and a study that was simply too small to find one.[6]

The Bayes factor solves this asymmetry by treating hypotheses as predictive models and comparing their track records. Mathematically, it is the ratio of two marginal likelihoods: the probability of the observed data given Hypothesis 1, divided by the probability of the data given Hypothesis 0. It asks a simple question: which theory did a better job of anticipating the data we actually collected?[3][5]

If the Bayes factor (denoted as BF10) is 5.0, the data is exactly five times more probable under the alternative hypothesis than under the null. If the ratio is 0.2, the data is five times more probable under the null. This symmetry allows researchers to actively prove the absence of an effect, a critical requirement in fields like medicine and psychology where knowing a treatment does not work is as valuable as knowing it does.[4][5]

Calculating a marginal likelihood requires integrating over a 'prior' distribution—a mathematical representation of the effect sizes expected under the hypothesis. In a 2012 paper in the Journal of Problem Solving, Jeffrey Rouder and colleagues demonstrated how to compute these factors for standard designs like t-tests, bridging the gap between abstract Bayesian theory and everyday laboratory practice.[7]

Rouder's team popularized the use of 'default' priors, such as the Jeffreys-Zellner-Siow (JZS) prior, which spreads the expected effect size over a Cauchy distribution. This allows researchers to calculate Bayes factors without having to subjectively specify the exact magnitude of the effect they expect to find, standardizing the metric for broad scientific publication.[7]

However, the choice of prior introduces a vulnerability known as Lindley's paradox. If a researcher specifies a prior that is too broad—claiming the alternative hypothesis could produce almost any effect size from zero to infinity—the probability of the specific data observed becomes diluted across that vast range of possibilities.[2]

As a result, the marginal likelihood of the alternative hypothesis drops, and the Bayes factor paradoxically throws its support behind the null hypothesis. The null, which predicts an effect size of exactly zero, concentrates all its predictive power on a single point. A theory that predicts everything predicts nothing, and the marginal likelihood ratio mathematically penalizes that vagueness.[2]

To interpret the resulting ratio, scientists rely on categorical scales. In 1939, physicist Harold Jeffreys proposed a foundational scale where a Bayes factor between 3.0 and 10.0 constitutes 'substantial' evidence, and anything above 10.0 is 'strong'.[2][3]

In 1995, statisticians Robert Kass and Adrian Raftery published a highly cited review adjusting these thresholds. They argued that a Bayes factor of 10 was too lenient for strong evidence, proposing a threshold of 20 (or 2 ln(BF) > 6) to account for the complexity of modern datasets and to provide a more robust guard against false positives.[2]

Beyond static thresholds, the Bayes factor enables a dynamic approach to data collection known as Bayes Factor Design Analysis (BFDA). A 2019 tutorial in PMC outlines how researchers can continuously monitor their data as it comes in, stopping the experiment the moment the Bayes factor crosses a pre-defined threshold of evidence, such as BF > 10 for the alternative or BF < 0.1 for the null.[1]

This continuous monitoring is strictly forbidden in traditional frequentist statistics, where checking the data early inflates the false positive rate. Because Bayes factors measure the relative likelihood of the data rather than long-run error rates, the evidence simply accumulates until a decisive ratio is reached, saving time and resources.[1][7]

The adoption of Bayes factors has accelerated in response to the replication crisis in the social sciences. By allowing researchers to distinguish between 'evidence of absence' (a Bayes factor strongly favoring the null) and 'absence of evidence' (a Bayes factor near 1.0), the metric provides a vocabulary for uncertainty that p-values lack.[4][6]

The transition requires a fundamental shift in how scientific claims are formulated. A theory can no longer just predict that two groups will be 'different.' To survive the ratio of marginal likelihoods, a theory must specify exactly how different they will be, and risk being penalized if it hedges its bets too broadly.[8]