Glossary
What is data labeling
In short
Data labeling is the work in which people or programs add labels to examples used to train a model: what is in a photo, what an email is about, where a customer name is in a text.
Machine learning · data labeling
MMOOZZGG.com editorsChecked: 2 min read
For a child
Imagine a kindergarten teacher teaching a little child to tell fruits apart. She shows a picture and says: “This is an apple, and this is a pear.” If every card has a caption, the child learns fast. A computer also needs such captioned cards, and people patiently make thousands of them. The neater the captions, the less the computer gets confused. A wrong caption teaches it the wrong thing.
In plain words
A model learns from examples with correct answers. Photos, calls and emails do not contain the answers by themselves; a person adds them. The person looks at an image and draws a box around each car, listens to a recording and writes what the customer asked for, reads a review and marks it “complaint” or “praise”. Such a mark is called a label. A set of examples with labels becomes the textbook for the model. For example, to teach a mail filter to spot spam, thousands of emails are first marked by hand as “spam” or “not spam”.
For business
The quality of the future model depends on labeling as much as on the algorithm: the model learns the errors in the labels along with the right answers. The cost comes from the number of examples, the difficulty of the task and the skill of the labelers. Any trained employee can label the topic of a request, but a doctor must mark a disease on an image. Companies label data in house, hire a contractor, or speed up the work with rules and ready models. The risks are personal data in the export for a contractor, and vague instructions that make two people give one example different labels. So before the launch, teams write instructions with examples and check agreement between labelers on a small sample. The amount of data work is usually estimated at the AI solutions map stage.
For a specialist
Labeling (data labeling, annotation) creates the target variable for supervised learning: a class, a bounding box, a segmentation mask, named entities in a text, a rating of a model answer. Label quality is measured by inter-annotator agreement and by control examples with a known answer. To cut manual work, teams use active learning: the model picks the examples it doubts most and gives them to people. Another way is weak supervision: experts write labeling functions, which are heuristics, and the system estimates their reliability and merges noisy votes into training labels without a gold standard. Snorkel (Ratner and co-authors, 2017) works this way. For large language models, labeling includes people ranking answers, which is used to build the reward model for RLHF.
Example
An online store wants AI to sort its reviews. Over a week, staff mark a couple of thousand reviews by hand by topic: delivery, quality, price. A model is trained on this sample, and then it sorts new reviews without people.
Related terms
Questions and answers
Can an AI model label data?
Sources
Not sure whether your company needs this?
AI solutions map