Glossary
Random forest: what it is and how it works
In short
A random forest is a machine learning algorithm that builds many decision trees on random parts of the data and combines their answers by voting or averaging.
Machine learning · random forest
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine the whole class guessing how many sweets are in a jar. One child can miss by a lot. But if everyone names a number and you take the average, the answer is often close to the truth. A computer does the same: it asks many simple advisers and listens to all of them together. This group of advisers is called a random forest.
In plain words
It is built on a decision tree: a chain of questions such as “did the client buy this year?” or “is the amount over ten thousand?”, with an answer at the end. One tree makes mistakes easily and memorizes random noise. So you build hundreds of trees, and each one learns from its own random set of examples and looks at only some of the features. The trees come out different and make different mistakes. A vote decides the result: the answer most trees gave wins. For numbers, such as a price, the answers are averaged. It is like a panel of doctors where each one saw different test results.
For business
For a company’s table data (requests, deals, payment history, sensor readings), random forest stays a reliable starting point. It runs without graphics cards, trains fast on an ordinary server, needs little fine tuning and shows which features had the most effect on decisions. This makes it handy for scoring, churn forecasts and finding defects. Limits: hundreds of trees take a lot of memory, the model does not go beyond its history (it will not forecast above past highs), and the built-in feature importance describes the model as a whole, not the decision on one client. It is often placed next to gradient boosting and compared on the same data. Our materials on predictive analytics explain which algorithm fits your task.
For a specialist
A random forest is an ensemble of decision trees described by Leo Breiman (Machine Learning, 45(1), 2001). Each tree trains on a bootstrap sample, as in bagging, and the split at each node is searched only among a random subset of features; answers are combined by voting in classification and by averaging in regression. Breiman showed that the forest’s generalization error converges to a limit as the number of trees grows, so adding trees does not lead to overfitting, and quality depends on the strength of single trees and the correlation between them. Examples left out of a tree’s bootstrap sample give an error estimate without a separate set (out-of-bag); he measured feature importance by shuffling the feature’s values in these examples (permutation importance). In scikit-learn, RandomForestClassifier builds 100 trees by default and considers sqrt(n_features) features at a node, and the feature_importances_ attribute is based on the decrease in impurity. Related methods: extremely randomized trees and gradient boosting, where trees are built one after another.
Example
A logistics company trained a random forest on its trip history to flag deliveries at risk of delay in advance. The model used weather, warehouse load and the driver, and showed that loading time had the most effect on delivery time, though managers had blamed traffic.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map