NBER-Led Initiative Pushes Integration of Private Transaction Data Into Core US Economic Statistics

The U.S. economy generates terabytes of transaction data every second, yet the official statistics that guide monetary policy and federal budgets still rely heavily on monthly surveys. As survey response rates decline and the economy digitizes, the traditional architecture of national accounting is showing its age. When a sudden shock hits, policymakers are often forced to fly blind, waiting weeks for the Bureau of Economic Analysis (BEA) or the Census Bureau to release lagging indicators.[1]

To bridge this structural gap, the National Bureau of Economic Research (NBER), supported by a multiyear grant from the National Science Foundation, has launched a comprehensive initiative to integrate private-sector transaction data directly into core U.S. economic statistics. The newly established Economic Measurement Research Institute (EMRI) aims to fundamentally re-engineer how metrics like Gross Domestic Product (GDP) and the Consumer Price Index (CPI) are calculated.[3]

The initiative represents a paradigm shift from a survey-first methodology to one that systematically ingests credit card swipes, payroll processor records, and online prices in near real-time. By convening academic economists, government statisticians, and private data providers, the NBER is attempting to standardize the use of naturally occurring data to measure the 21st-century economy.

The primary claim driving this methodological shift is that private transaction data can accurately track economic turning points weeks before official surveys are tabulated. Personal Consumption Expenditures (PCE) account for roughly 68 percent of U.S. GDP, making consumer spending the most critical engine of the American economy. Traditional PCE data arrives with a significant lag and lacks subnational detail, making it difficult to analyze localized or short-lived economic shocks.[2]

The evidence for this claim is strong, particularly during periods of high volatility. During the COVID-19 pandemic, the BEA demonstrated the viability of alternative data by utilizing daily payment card transactions to monitor the collapse and subsequent recovery in consumer spending. This high-frequency barometer provided policymakers with crucial real-time insights when traditional surveys were disrupted.

Academic research analyzing these transaction datasets confirms their predictive power. A study benchmarking credit card spending against official statistics found that card data closely mirrored the official adjusted PCE. At the trough of the pandemic shock in April 2020, credit card spending dropped 25.7 percent, aligning tightly with the 22.7 percent decline recorded in official PCE figures. Furthermore, variation in credit card spending growth explained 92 percent of the variation in monthly adjusted PCE growth over the sample period.[2]

The Federal Reserve Bank of Chicago has also successfully operationalized this hybrid approach. Its Weekly Index of Retail Trade (CARTS) combines high-frequency data from private companies with the Census Bureau's Monthly Retail Trade Survey. CARTS demonstrates that private data can accurately explain high-frequency fluctuations in national retail spending while remaining mathematically anchored to official Census benchmarks.

Despite these successes, the evidence pack is explicit about the severe limitations of relying on private transaction records. The counter-claim is that private data is structurally biased and lacks the rigorous representativeness of government surveys. When the Bureau of Labor Statistics (BLS) evaluated consumer spending data from a prominent private aggregator, it found significant representational gaps. The evaluated dataset captured only about 10 percent of total U.S. credit and debit card spending, raising concerns about geographic and demographic blind spots.[1]

The mechanism of this bias is rooted in how the data is generated. Official statistics are meticulously designed to measure specific economic concepts and are weighted to be nationally representative. In contrast, credit card data inherently excludes cash transactions, introducing a structural demographic bias that underrepresents unbanked, elderly, and lower-income households. Private data reflects only the customer bases of specific financial institutions or merchants, not the American public at large.[1]

The BEA explicitly acknowledged these constraints when publishing its experimental payment card estimates. The agency noted that the data primarily covered brick-and-mortar merchants and was vulnerable to shifts in payment methods—such as the pandemic-accelerated move away from cash—which could artificially skew the perceived trajectory of spending. Consequently, the BEA warned that these estimates were a complement to, not a substitute for, official data grounded in proven methodologies.

Beyond demographic bias, the stability of private data pipelines remains a critical unknown. In May 2024, the BEA was forced to suspend its payment card transaction estimates entirely due to budget constraints, highlighting the fragility of relying on ad-hoc commercial data acquisitions.

Furthermore, private firms are under no obligation to maintain consistent data structures. They can alter their categorization algorithms, change their merchant codes, or simply refuse to share data. If a major payroll processor changes how it classifies gig workers, or a credit card network updates its industry codes, it could introduce artificial volatility into national statistics that algorithms might misinterpret as a macroeconomic shock.[3]

The NBER's EMRI initiative is designed to solve these exact methodological hurdles. Through a series of research conferences scheduled through 2027, the institute is establishing robust frameworks for data integration, focusing on how to value technologically driven quality changes and measure the gig economy. The ultimate goal is to build resilient hybrid models where high-frequency private data interpolates between the reliable, representative benchmarks provided by federal statistical agencies.[3]