Glossary
What is a blending ensemble
In short
Blending is a way to combine the forecasts of several models with one more model, which trains on a separate held-out part of the data. A blending ensemble is a simpler form of stacking.
Machine learning · blending
MMOOZZGG.com editorsChecked: 2 min read
For a child
You bake pancakes from three recipes and want to know how to mix the batter for the tastiest result. You keep some pancakes not for yourself but for your grandmother. She tastes them, tells you which pancakes the mix needs more of, and you remember her advice. She did not see how you cooked, so her opinion is honest. A computer does something similar: it sets aside some examples for an honest check.
In plain words
First the data is split into two parts. Several models train on the large part. The models never see the small part, which is called the holdout. Then each model makes forecasts for the held-out examples, and the final blender model learns from these forecasts. It chooses the weight for each participant’s opinion. The scheme is simple and fast: each model trains once, without many passes over data splits. The price of simplicity is that the held-out part goes only to the blender, so the participants learn from less data. It is like a cook who keeps a couple of plates for guests who never saw the kitchen and fixes the recipe by their feedback.
For business
The approach is handy when several contractors or teams have brought their own models for one task and you need to build a joint solution from them quickly. It is easy to explain and repeat: one held-out part of the data and one simple model on top. The method became known through the Netflix Prize competition, where one way of blending forecasts was a key part of the solution that took second place. The risks: if the held-out part is small or unlike future data, the weights fit to chance and the forecast drops in real use. Before launch the result is checked on a third part of the data that neither the models nor the blender touched. Whether you need this extra layer at all is decided by comparing it with a single model in a pilot.
For a specialist
Blending is a simplified form of stacking. The training set is split into a main part and a holdout. First-level models train on the main part, and their predictions on the holdout become features for the second-level model, most often a linear or logistic regression with regularization. Unlike stacking, there are no out-of-fold predictions, so each first-level model trains once, and the split itself prevents leakage. The drawbacks: less data for both the first-level models and the meta-model, and higher variance in the weight estimates. The paper by Sill et al. (2009) on the Netflix Prize results describes the ensembles as blending the forecasts of several models. The authors propose feature-weighted linear stacking, where the model weights depend linearly on meta-features, that is, extra descriptions of each example. To measure final quality you need a third, test set that took part in no step.
Do not confuse. Do not confuse it with blending in the production of tea, coffee and wine, where the word means mixing varieties.
Example
Three teams built churn forecast models. An analyst sets aside the data for the last month, sees how each model predicted customer churn, and trains a simple blender model on this. The final forecast is checked on the next month, which nobody has seen.
Related terms
Questions and answers
How is blending different from stacking?
How much data should you set aside for the blender?
Sources
Not sure whether your company needs this?
AI solutions map