MMOOZZGG.com

Glossary

What are AI guardrails?

In short

AI guardrails are checks and rules around a language model. They keep dangerous requests out of it and stop unacceptable answers and actions from coming out.

AI agents · guardrails

MMOOZZGG.com editorsChecked: 2 min read

For a child

A water slide at a water park has side walls: you slide fast and have fun, but you cannot fly off to the side. At the entrance, a lifeguard stops kids who are too young. Smart programs have side walls like that too. They do not stop the program from helping people, but they keep it from being rude, giving away someone’s secret or doing something dangerous.

In plain words

An instruction in a prompt asks the model to behave with care, but you cannot rely on it alone: a model can be confused, and sometimes it makes mistakes by itself. So separate checks sit around it. Some look at the incoming question: does it try to get around the rules, or carry personal data it should not? Others check the answer: are the numbers made up, has the talk drifted off topic, is there rudeness? A third group watches actions: a support agent will not issue a refund above a set amount. When a check fires, the answer is blocked, rewritten or passed to a person.

For business

Without guardrails, a company answers for every word and action of its AI: a promise of a discount that does not exist, advice outside its competence, a leak of internal data. The set of checks depends on the process. For a client chat, the topic, the tone and a ban on price promises matter most. For an internal assistant, it matters more to keep personal data and trade secrets inside. Each check adds delay, and if a separate model runs it, cost as well. Rules that are too strict start to refuse normal requests. So teams tune the rules on real dialogues and review them by the log of triggers. A review of data security in AI agent projects shows where to start.

For a specialist

Guardrails are software checks while the application runs, independent of how the model was aligned in training. By position, there are input rails, output rails, dialogue rails that keep a conversation inside a set scenario, and action guardrails for agents: tool allowlists, amount limits and human approval. They are built on rules (regular expressions, lists, schemas), on classifiers or on separate judge models. The NeMo Guardrails toolkit (Rebedea et al., 2023) sets programmable rules on top of any model and stresses that they do not depend on the model and can be interpreted. Llama Guard (Inan et al., 2023) is a model based on Llama 2 7B, fine-tuned to classify both user requests and model answers by a risk taxonomy. Such checks reduce, but do not remove, the risk of prompt injections and jailbreaks, so they are combined with minimal permissions and monitoring.

Do not confuse. Not to be confused with guardrails in engineering and electrics, such as a speed limiter: here the term means rules for AI.

Example

A shopper asks a store chatbot to confirm a discount that a manager supposedly promised. The output guardrail sees that the reference list has no such offer and blocks the promise. The bot offers the current terms or a contact with a manager instead.

Related terms

Questions and answers

What are guardrails in the context of LLMs? They are checks before and after a call to a language model. They filter requests, answers and actions by the company’s rules, no matter how the model was trained.
Is a system prompt enough instead of separate checks? No. A model can break an instruction in the prompt by mistake or under a clever request.

A check in code always fires, whatever the model decides.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us