Glossary
What is an epoch in machine learning
In short
An epoch in machine learning is one full pass of a model over all the training examples.
Machine learning · epoch
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine the teacher gave you a stack of word cards. You read all the cards from the first to the last once. That is one round. Then you shuffled the stack and went through it again, and the words stuck better. A computer also learns in rounds over its stack of examples, and each such round is called an epoch.
In plain words
A model rarely learns a rule the first time. So it is shown the training examples several times in a row. One full round over all the examples is an epoch. Inside a round the data is split into small batches, and after each batch the model corrects itself a little. With too few rounds, the model does not have time to learn. With too many, it starts to memorize these exact examples and handles new ones worse. A pupil who solved the same test version ten times gets lost when he sees a different one.
For business
The number of epochs directly changes the bill for training: each extra pass over the data again occupies graphics cards for the same time. When a contractor fine-tunes a model on your documents or sales history, ask how many passes it plans and by what rule it will stop. A sound practice is to check quality after each pass on held-out data the model has not seen, and to stop training when the metric stops growing. Extra passes do not only waste money, they also spoil the result: the model fits the old records and makes mistakes on new customers. The quality curve shows how many passes pay off, and the server capacity for this load is calculated in advance.
For a specialist
An epoch is one pass of the optimizer over the whole training set. In mini-batch training it consists of about N/B iterations (steps), where N is the number of examples and B is the batch size; the data is usually shuffled before each pass. In Keras the number of epochs is set by the epochs argument of the fit method, and the documentation defines an epoch as an iteration over all the provided x and y data; for an endlessly repeating data set the boundary is set by hand through steps_per_epoch. The number of epochs is a hyperparameter chosen from learning curves. Training error usually falls, while validation error first falls and then rises, which signals overfitting. Hence early stopping: the EarlyStopping callback in Keras checks the monitored metric at the end of each epoch and stops training after a set number of passes without progress (patience). With the restore_best_weights flag it returns the weights of the pass where the monitored value was optimal.
Do not confuse. Do not confuse it with an epoch as a period of history: here the word means one round of model training.
Example
An online store fine-tunes a model that predicts returns. After the third epoch quality on validation orders stops growing, and after the sixth it falls. The team takes the weights after the third pass and saves half of the graphics card time.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map