SMOTE for Imbalanced Data: How It Works, Variants & Best Practices

SMOTE for Imbalanced Data: How It Works, Variants, and When Not to Use It

Quick Answer: SMOTE (Synthetic Minority Over-sampling Technique) balances imbalanced datasets by creating new synthetic minority-class samples. It picks a minority sample, finds its nearest minority neighbors, and generates new points along the lines between them. It can improve recall on rare classes. However, it must be applied only to training data, and it does not always beat simpler alternatives like class weights.

Fraud, rare diseases, equipment failures, and customer churn all share one challenge: the event you care about is rare. When 99% of transactions are legitimate, a model can reach 99% accuracy by predicting "not fraud" every time and still be useless. SMOTE is one of the most popular techniques for this problem. This guide explains how it works, its most useful variants, how to apply it without leakage, and when you should skip it entirely.

Key Takeaways

• SMOTE creates synthetic minority examples by interpolating between neighbors.
• Chawla and colleagues introduced it in 2002.
• Apply SMOTE only inside training folds, never before splitting data.
• Variants such as Borderline-SMOTE, ADASYN, and SMOTE-NC handle specific situations.
• With modern gradient boosting, class weights and threshold tuning often perform as well or better.

What Is SMOTE?

SMOTE stands for Synthetic Minority Over-sampling Technique. Nitesh Chawla, Kevin Bowyer, Lawrence Hall, and Philip Kegelmeyer published it in the Journal of Artificial Intelligence Research in 2002. It remains one of the most cited methods in imbalanced learning.

Before SMOTE, the usual approach was random oversampling, which duplicates existing minority rows. Duplication makes models memorize the same points and overfit. SMOTE instead generates new, slightly different examples, giving the classifier a broader minority region to learn from.

How SMOTE Works, Step by Step

1. Choose a minority-class sample.
2. Find its k nearest neighbors within the minority class. The default k is 5.
3. Randomly select one of those neighbors.
4. Pick a random point on the straight line between the sample and that neighbor.
5. Add this new point as a synthetic minority example.
6. Repeat until the desired class balance is reached.

For example, suppose one fraud case has a transaction amount of 2,000 and its neighbor has 3,000. SMOTE might create a new fraud case with an amount of 2,400, with the other features interpolated in the same way.

Popular SMOTE Variants

Borderline-SMOTE

Borderline-SMOTE generates synthetic samples only near the decision boundary, where minority points are surrounded by many majority points. This focuses effort on the hardest cases.

ADASYN

Adaptive Synthetic Sampling creates more synthetic data for minority examples that are harder to learn. It measures difficulty by how many majority neighbors each example has.

SMOTE-NC

Standard SMOTE interpolates numbers, which makes no sense for categorical features like "city" or "device type." SMOTE-NC handles mixed numerical and categorical data by assigning the most frequent category among the neighbors.

SMOTEN

SMOTEN is designed for datasets with only categorical features.

SVM-SMOTE and KMeans-SMOTE

SVM-SMOTE uses support vectors to locate the boundary area. KMeans-SMOTE clusters the data first and oversamples within safe, minority-dense clusters.

SMOTE-Tomek and SMOTEENN

These hybrids combine oversampling with cleaning. After SMOTE generates new samples, Tomek links or Edited Nearest Neighbors remove noisy or overlapping points from both classes.

The Most Common SMOTE Mistake: Data Leakage

Many tutorials apply SMOTE to the full dataset and then split it into training and test sets. This is a serious error. Synthetic points are created from neighbors that may end up in the test set. The model is then evaluated on data that is partly derived from its own training examples, so scores become unrealistically high.

The correct approach:

• Split data into training and test sets first.
• Apply oversampling only to the training set.
• During cross-validation, apply it inside each training fold, never to validation folds.
• Use the imbalanced-learn Pipeline, which handles this automatically when combined with cross-validation tools.

Your test set should always reflect the real-world class distribution.

Does SMOTE Actually Work? What Recent Research Says

SMOTE was designed in an era of weaker classifiers such as decision trees and k-nearest neighbors. Several recent studies and practitioner benchmarks have found that strong learners like XGBoost, LightGBM, and CatBoost often gain little from oversampling once the classification threshold is properly tuned.

There are also side effects to consider:

• Probability calibration suffers. Oversampling changes the class balance the model sees, so predicted probabilities become inflated. If you need accurate risk scores, you must recalibrate.
• Noise can be amplified. If minority points are mislabeled or are outliers, SMOTE creates more of them.
• High-dimensional data is tricky. Nearest-neighbor distances become less meaningful as the number of dimensions grows.

This does not mean SMOTE is useless. It still helps with weaker models, small datasets, and some tabular problems. The key is to test it against alternatives rather than assume it will help.

Alternatives to SMOTE

• Class weights: most libraries let you penalize minority errors more heavily, such as class_weight in scikit-learn or scale_pos_weight in XGBoost.
• Threshold tuning: instead of using 0.5, choose the cutoff that maximizes F1, recall at a set precision, or business value.
• Random undersampling: useful when you have massive amounts of majority data.
• Balanced ensembles: BalancedRandomForest and EasyEnsemble train on balanced subsets.
• Anomaly detection: when positives are extremely rare, methods like Isolation Forest may fit better.
• Collecting more minority data: often the most effective solution when it is possible.

Choosing the Right Evaluation Metrics

Accuracy is misleading for imbalanced data. Use metrics that focus on the minority class:

• Precision: of the predicted positives, how many are correct.
• Recall: of the actual positives, how many were found.
• F1 score: the harmonic mean of precision and recall.
• PR-AUC: the area under the precision-recall curve. It is more informative than ROC-AUC when positives are rare.
• Matthews Correlation Coefficient: a balanced single-number summary.

Compare models with and without SMOTE using these metrics on an untouched test set.

Frequently Asked Questions

Should SMOTE be applied before or after the train-test split?

Always after. Apply it only to training data to avoid leakage.

What is a good k value for SMOTE?

The default of 5 works well in most cases. Lower values help when the minority class is very small.

Can SMOTE be used for text or images?

It is not ideal. Data augmentation methods usually work better, such as back-translation for text or rotations and crops for images.

Does SMOTE work for multiclass problems?

Yes. The imbalanced-learn implementation supports multiclass oversampling with custom sampling strategies.

Final Thoughts

SMOTE remains a valuable tool for imbalanced classification, but it is not a magic fix. To use it well:

• Apply it only to training data.
• Choose the right variant for your feature types.
• Always benchmark it against class weights and threshold tuning.

Used thoughtfully, SMOTE can improve minority-class detection. Used blindly, it can hide real problems behind inflated scores.