MMOOZZGG.com

Glossary

What is a chat model?

In short

A chat model is a language model that has been tuned to hold a dialog. It understands the roles of the speakers, follows instructions, and answers each message, instead of only continuing a text.

Language models · chat model

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine a talking parrot that repeats what it hears, and a friend who answers your questions. A neural network after its first lessons is like the parrot: it continues a phrase any way it can. Then it is taught for a long time to take turns, listen to the question, and answer to the point. This turns the parrot into a real partner in conversation.

In plain words

First, the model learns from a huge amount of text, and it can only finish what was started. Then it is trained further on conversations where each message has a role: the user, the assistant, and sometimes a system instruction with rules. The model gets used to the format: after the user’s question it writes the assistant’s answer and stops. This gives the dialog version you talk to in chatbots. For example, to the message “Hi, help me with a letter” it will not continue your phrase; it will ask whom the letter is for and what it is about.

For business

Almost all models connected to assistants and agents are chat models or their reasoning versions. When a company chooses one, it checks how the model keeps system instructions in a long dialog, whether it refuses work tasks for no reason, how well it writes in the customers’ language (for a Russian company, in Russian), and whether it can return an answer in a set format. An important detail for your own server: each open model has its own dialog template, and with the wrong template quality drops sharply. A chat model does not know your prices and rules, so for answers to customers it is connected to a knowledge base. An AI assistant on company documents shows how this looks in practice.

For a specialist

A chat model (also called an instruct model) is a base LLM that went through post-training on dialog data: usually supervised fine-tuning (SFT) and preference training such as RLHF or DPO. Ouyang and co-authors (2022) described this scheme for InstructGPT. Technically, a dialog is still a sequence of tokens: a list of messages with the roles system, user, and assistant becomes text through a chat template with control tokens. The Hugging Face documentation notes that models tuned from one base model can use different templates, and with the wrong control tokens quality drops sharply. The add_generation_prompt flag adds the start of the assistant’s reply; without it, the model may continue the user’s message instead of answering. Modern templates also mark tool calls and reasoning blocks. Through an API, such models are called with an array of messages instead of one prompt string.

Example

A company deploys an open chat model on its own server but forgets to apply its dialog template. The model starts writing questions for the user. After the right template is turned on, it answers as an assistant.

Related terms

Questions and answers

How is a chat model different from a base model? A base model only continues text. A chat model was also trained on dialogs and human preferences, so it answers questions, follows instructions, and knows where its reply ends.
Are a chat model and a chatbot the same thing? No. A chat model is the neural network inside, while a chatbot is a ready service with an interface, scenarios, and connected data.

One chatbot can use different models in turn.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us