Already-Treated Controls Subtract Evolving Treatment Effects: Why Two-Way Fixed Effects Assign Negative Weights in Staggered Rollouts

When an economist sits down to evaluate a policy that rolls out state by state, they must choose a mathematical engine to isolate the effect. The researcher decides which variables to hold constant, what timeline to measure, and how to separate the policy's impact from background noise.[6]

For twenty years, the default engine for this task was the two-way fixed effects regression. This statistical model became the undisputed standard in applied microeconomics, appearing in thousands of papers evaluating everything from minimum wage hikes to healthcare expansions.[1][3]

The model relies on a foundational logic called difference-in-differences. If California raises its minimum wage in 2018 and Nevada does not, the researcher compares the change in California's employment to the change in Nevada's employment over the exact same period.[6]

Nevada serves as the counterfactual, representing what would have happened in California had the policy never passed. By subtracting Nevada's baseline trend from California's observed outcome, the model isolates the specific effect of the wage hike.[1]

The Staggered Rollout Trap

This simple two-by-two comparison works flawlessly when a policy happens exactly once. But real-world legislation rarely follows such a clean schedule, and policies typically cascade across jurisdictions over several years.[6]

When states adopt a policy at different times, researchers face a staggered rollout. California might act in 2018, Oregon in 2019, and Washington in 2020, creating a complex matrix of treatment timing that a simple two-by-two grid cannot accommodate.[1][2]

To handle this complexity, researchers feed the entire dataset into a two-way fixed effects regression. The algorithm mathematically averages all the available comparisons into a single, neat coefficient that represents the policy's overall average effect.[3]

The model accomplishes this by controlling for fixed differences between states and fixed shocks across time. It assumes that once these baseline differences are stripped away, every remaining variation in the data represents the pure impact of the policy.[1][3]

But in 2021, econometricians discovered a fatal flaw in how the algorithm constructs that average. The model does not just compare treated states to states that never adopted the policy; it also compares newly treated states to states that adopted the policy years earlier.[1][4]

Unpacking the Black Box

Andrew Goodman-Bacon published a landmark theorem in the Journal of Econometrics that cracked open the algorithm's black box. He proved that the single coefficient is actually a weighted average of every possible two-by-two comparison in the dataset.[1]

"The TWFE estimator is a weighted average of all two-group/two-period DiD estimators in the data," Goodman-Bacon wrote. His decomposition revealed exactly which states the algorithm was pairing together behind the scenes.[1]

Some of these pairings make intuitive sense, like comparing a state treated in 2019 against a state that is never treated. The algorithm assigns heavy mathematical weights to these clean comparisons, anchoring the final estimate in solid counterfactuals.[1][6]

Other pairings compare early adopters to late adopters before the late adopters pass the policy. Because neither state has changed its status during that specific window, this comparison also yields a valid measure of the policy's impact.[1]

The critical failure occurs in the final type of pairing. The algorithm routinely uses early adopters as the control group for late adopters, comparing Washington's 2020 policy change against California's 2020 baseline.[1][2]

The Poisoned Control Group

Using California as a control group in 2020 requires a massive mathematical assumption. The algorithm must assume that California's 2018 policy has already reached its maximum effect and that its current trajectory perfectly mirrors a world without the policy.[3][4]

In reality, economic policies almost always have dynamic treatment effects that evolve over time. A new tax credit might take three years to reach peak enrollment, meaning the policy's impact grows steadily larger with each passing year.[4][6]

If California's policy effect is still growing in 2020, its baseline trend is fundamentally poisoned. The state is experiencing a delayed surge from its own 2018 legislation, making it a terrible proxy for what Washington would have looked like without the 2020 law.[1][4]

When the algorithm subtracts California's poisoned trend from Washington's outcome, it accidentally subtracts the evolving treatment effect. The model interprets California's delayed surge as a background economic shock, and penalizes Washington's estimate accordingly.[1][3]

Clément de Chaisemartin and Xavier D'Haultfœuille quantified this disaster in the American Economic Review in 2020. They demonstrated that this subtraction mechanism can mathematically assign a negative weight to the true treatment effect of the late-adopting state.[3]

The Mathematics of Negative Weights

A negative weight means the algorithm takes a genuinely positive policy outcome and flips its sign before adding it to the final average. If Washington's policy created 5,000 jobs, a negative weight forces the model to count those jobs as losses.[3][6]

"Under heterogeneous treatment effects, the TWFE estimator can be negative even if the treatment effect is positive for every unit at every time period," de Chaisemartin and D'Haultfœuille warned in their analysis of the estimator's mechanics.[3]

The researchers reviewed 116 highly cited papers published in top economics journals that relied on the standard algorithm. They found that 19 percent of the underlying comparisons in those papers were assigned negative weights by the mathematical engine.[3]

In a 2022 simulation published in the Journal of Financial Economics, researchers generated a fake dataset where a policy explicitly increased corporate outcomes by 18 percent. When they ran the standard algorithm, the negative weights overpowered the data, producing a final estimate of negative 5 percent.[5]

The severity of the distortion depends entirely on the variance in treatment timing. The more staggered the rollout, and the more the policy's effect changes over time, the more negative weights the algorithm generates.[1][5]

The Modern Solutions

The discovery of negative weights triggered an immediate methodological revolution in applied microeconomics. Researchers realized they could no longer trust the standard algorithm for any policy that rolled out in stages.[2][6]

Brantly Callaway and Pedro Sant'Anna developed a heterogeneity-robust estimator that completely bypasses the poisoned control groups. Their method manually constructs clean two-by-two comparisons, strictly forbidding the algorithm from ever using an already-treated state as a control.[2]

Liyang Sun and Sarah Abraham published a parallel solution for event studies in 2021. Their approach estimates a separate dynamic effect for each specific cohort of adopters, preventing the evolving effects from bleeding across the timeline.[4]

These modern estimators require researchers to explicitly define their counterfactuals rather than letting a black-box algorithm average them automatically. By forcing transparency, the new methods ensure that every mathematical weight remains strictly positive.[2][4][6]

The transition to these robust estimators is now nearly complete across top academic journals. When an economist sits down to evaluate a staggered policy today, the two-way fixed effects engine remains firmly turned off.[5][6]