Evidence Pack: The Accuracy and Trade-Offs of SMOTE Versus Class Weights in Imbalanced Data

Data scientists who favor algorithmic data generation argue that models need synthetic examples to learn complex decision boundaries in minority classes, while statistical purists argue that fabricating data distorts the true underlying probability distribution and prefer penalizing errors via class weights. When a dataset contains 99 percent normal transactions and 1 percent fraud, standard accuracy metrics fail entirely.

A model can achieve 99 percent accuracy simply by predicting that fraud does not exist. To force the algorithm to care about the rare event, practitioners must intervene. The debate over how to do so centers on two fundamentally different philosophies: changing the data before it reaches the model, or changing how the model learns from the data it already has.[4][5]

The Synthetic Minority Over-sampling Technique (SMOTE) takes the first approach. Rather than simply duplicating existing rare examples—which leads to severe overfitting—SMOTE fabricates entirely new data points. It identifies a minority class sample, finds its nearest neighbors in the feature space, and draws lines between them.

It then generates synthetic points along those lines. This provides the algorithm with a denser, more continuous region of the minority class to learn from. For weak learners or extremely small datasets, this synthetic scaffolding can be the difference between a model that finds a pattern and one that finds nothing.[3][4]

Class weighting takes the opposite approach: it leaves the dataset untouched and alters the algorithm's loss function. In a standard model, every misclassification carries the same penalty. Class weighting changes the math so that a mistake on the minority class costs significantly more. The scikit-learn documentation notes that the "balanced" heuristic automatically adjusts weights "inversely proportional to class frequencies in the input data."

If fraud represents 1 percent of the data, the algorithm is penalized 99 times more heavily for missing a fraudulent transaction than for misclassifying a normal one. The model is forced to pay attention to the rare class not because there are more examples of it, but because ignoring it destroys the model's overall score.[2]

Modern gradient boosting frameworks handle class weights natively through parameters that scale the gradient of the positive class. The XGBoost documentation explicitly advises users to "balance the positive and negative weights" using the scale_pos_weight parameter for imbalanced datasets. Because tree-based models partition the feature space using orthogonal splits, introducing synthetic SMOTE points can force the trees to create overly complex, fragmented branches to accommodate the artificial data. By using class weights instead, the trees maintain simpler, more generalized splits while still prioritizing the minority class errors during the boosting rounds.[6]

The tension between the two methods emerges most clearly in high-dimensional feature spaces. Because SMOTE interpolates blindly between points, it assumes that the space between two minority examples also belongs to the minority class. In complex datasets, this assumption frequently fails, generating synthetic noise that blurs the true decision boundary.

When evaluated on the Precision-Recall Area Under the Curve (PR-AUC)—the most rigorous metric for imbalanced data—SMOTE often degrades the model's underlying ranking ability. It artificially inflates recall at the cost of catastrophic precision, creating a model that flags too many false positives.[1][5]

A comprehensive 2024 study published on arXiv tested this dynamic at scale. Researchers ran 9,000 experiments across 15 different machine learning models and 30 imbalanced datasets, comparing SMOTE, class weights, and a baseline model. The results confirmed that both interventions substantially outperform doing nothing. However, the study revealed that class weights often match or exceed SMOTE's performance without the computational overhead of generating synthetic data or the risk of introducing artificial noise.[1]

The same benchmark highlighted a third, often-overlooked solution: decision threshold calibration. Both SMOTE and class weights attempt to fix the imbalance during training. Threshold calibration trains a standard model and then adjusts the probability cutoff required to predict the minority class.

Instead of requiring a 50 percent certainty to flag fraud, the model might flag it at 10 percent. The authors noted that while all three strategies beat the baseline, their results pointed to "Decision Threshold Calibration emerging as the most consistently effective technique." Across the 9,000 experiments, this post-training adjustment outperformed both data-level resampling and algorithm-level weighting.[1]

The consensus among modern practitioners is shifting toward a hierarchy of interventions. Because class weighting requires no data manipulation and adds zero computational cost, it is widely considered the safest first step. If class weights fail to capture the minority pattern, and the dataset is small enough that the model genuinely lacks examples to learn from, SMOTE becomes a viable escalation. But neither technique should be deployed without first testing a simple threshold adjustment on a baseline model.[5]

The choice between SMOTE and class weights represents a choice between two different types of distortion. One distorts the dataset to fit the algorithm, while the other distorts the algorithm's objective to fit the dataset. As machine learning models grow more sophisticated and datasets expand in dimensionality, the mathematical preference is increasingly to leave the data exactly as it was recorded in the real world, shifting the burden of adaptation onto the algorithm itself.[5]