AI glossary
Machine learning terms
Machine learning terms describe how a model learns from data and how its quality is measured. This category covers training and test data, common algorithms, network types, error metrics and the practices that move a model into production.
MMOOZZGG.com editors·Checked: ·8 min read
Terms in this category
- autoencoderAn autoencoder is a neural network that compresses data into a short code and then restores the data from that code. While doing this, it learns to pick out what matters in the data.
- ensemble learningEnsemble learning means several machine learning models whose answers are combined into one forecast, so that it is more accurate and more stable than that of any single model.
- data augmentationData augmentation is a way to expand a training set with changed copies of existing examples: rotated photos, reworded texts and voice recordings with added noise.
- batchA batch is a group of training examples from which a model calculates its error and updates its weights in one step. Batch size in machine learning is the number of examples in that group.
- baselineA baseline model is a simple starting solution to a task. You compare a new model with it to see whether the new model gives a real gain.
- blendingBlending is a way to combine the forecasts of several models with one more model, which trains on a separate held-out part of the data. A blending ensemble is a simpler form of stacking.
- validation setA validation set is a held-out part of the data on which developers compare model versions and settings during development, without touching the final test.
- variational autoencoderA variational autoencoder (VAE) is a neural network that compresses data into a random code with a set distribution and can create new, similar examples from that code space.
- generative adversarial networkA generative adversarial network (GAN) is a pair of neural networks where one creates fake examples and the other learns to tell them from real ones, and in this contest the first starts to produce believable pictures.
- hyperparametersHyperparameters are training settings that a person sets before the start and that the model does not change itself: the learning rate, the batch size, the number of epochs and the size of the network.
- gradient boostingGradient boosting is a machine learning method that builds many small decision trees one after another, and each new tree fixes the errors of the trees already built.
- gradient descentThe gradient descent algorithm is a way of training a model in which its internal numbers are shifted step by step toward where the error falls fastest.
- class imbalanceClass imbalance in machine learning is a situation where the training data has far more examples of one class than of another, and the model almost stops noticing the rare class.
- data driftData drift is a change in real-world data after a model goes live, so that the data no longer looks like the training data and the quality of predictions falls quietly.
- classificationClassification in machine learning is a task in which a model assigns an object to one of the classes known in advance: spam or not spam, complaint or question, defect or normal item.
- clusteringClustering in machine learning is a method that splits objects into groups of similar ones by itself, with no answers set in advance: customers, products, the texts of requests.
- cross validationCross validation is a way to evaluate a model in which the data is split into parts that take turns as training and test sets, and the results are averaged.
- confusion matrixA confusion matrix is a table that compares a model’s answers with the correct answers, so you can see how many times and in which direction the model made a mistake.
- reward modelA reward model is a neural network that scores the answer of a language model by predicting how much a person will like it. These scores are then used to train the main model further.
- underfittingUnderfitting is an error in which a model is too simple or trained too little, so it does not catch the pattern even in the data it learned from.
- data normalizationData normalization brings numeric features to a comparable scale, for example to the range from zero to one, so that none of them outweighs the others only because of its unit of measure.
- backpropagationBackpropagation is the algorithm that computes how much each weight of a neural network contributed to an error, moving through the network from the output to the input.
- training datasetA training dataset is the part of your data that a model uses to set its weights: examples with known answers that it learns from.
- transfer learningTransfer learning is an approach in which a model already trained on one large task is adapted to a new task using a small dataset.
- overfittingOverfitting is a training error in which a model memorizes the training examples together with their random details and then performs badly on new data.
- featuresFeatures in machine learning are the properties of an object, written as numbers or categories, that a model uses to make a prediction: a customer’s age, an order amount, the day of the week.
- exploratory data analysisExploratory data analysis (EDA) is the first look at a dataset: charts, summary numbers and checks that show what the data contains, what is missing and where the errors are.
- data labelingData labeling is the work in which people or programs add labels to examples used to train a model: what is in a photo, what an email is about, where a customer name is in a text.
- regressionRegression is a machine learning task in which a model predicts a number: a price, demand, a delivery time or fuel use.
- regularizationRegularization is a set of techniques that stop a model from fitting random details of its training data, so that it works better on new data.
- recurrent neural networkA recurrent neural network (RNN) reads data one element at a time and keeps an internal memory of what it has read. This makes it a fit for text, speech and series of numbers.
- convolutional neural networkA convolutional neural network (CNN) is a neural network for images and signals that scans a picture with small windows and builds whole objects from small details.
- synthetic dataSynthetic data is examples created by a program or a neural network, not collected from real life: texts, tables and images used to train and test models.
- learning rateThe learning rate is a number that sets how strongly a model changes its weights at each training step.
- random forestA random forest is a machine learning algorithm that builds many decision trees on random parts of the data and combines their answers by voting or averaging.
- stackingStacking in machine learning is a way to combine several models: their predictions become the input for one more model, and that model gives the final answer.
- test setA test set in machine learning is data that a model has not seen in training or in tuning. You use it once to get the final quality score.
- precision and recallPrecision and recall are two quality metrics of a model. Precision shows what share of the found items is correct. Recall shows what share of the needed items the model found.
- loss functionA loss function in machine learning is a formula that turns a model’s error into one number: the smaller it is, the closer the model’s answers are to the right ones.
- target variableA target variable is the value a machine learning model learns to predict: whether a customer will leave, how much product will sell, how much a flat costs.
- epochAn epoch in machine learning is one full pass of a model over all the training examples.
- dropoutDropout is a technique for training neural networks in which a share of neurons is switched off at random at every step, so the network does not rely on single connections.
- F1 scoreThe F1 score is one number from zero to one that joins the precision and the recall of a model, and it is high only when both are high.
- MLOpsMLOps (machine learning operations) is a set of practices and tools that move a machine learning model from an experiment to stable daily work: building, releasing, monitoring and retraining.
- ROC-AUCROC AUC is the area under the ROC curve. It is a number from 0 to 1 that shows how well a model ranks positive examples above negative ones at any decision threshold.