MMOOZZGG.com

How to build an AI agent

AI browser agent: an agent that clicks for you in the browser

5 min read7 sources

An AI browser agent opens sites, clicks buttons and fills in forms by itself, like an employee at a computer. It saves hours on routine work, but in an Anthropic test attackers tricked such an agent without protection in 23.6% of attacks. So give it access with care.

On this page · 4 sections

Build it yourself in an evening

What these agents can do

The agent works with the screen as a person does: it looks at a screenshot, finds a button, moves the cursor, clicks and types. Anthropic released this feature on 22 October 2024 and called it experimental: scrolling, dragging and zooming were hard for the model.

In OSWorld, a test where an agent does tasks in a real operating system, Anthropic’s model solved 14.9% of tasks at that time. Working with an interface is hard even for a strong model, and mistakes are normal.

In August 2025, Anthropic began testing an extension for the Chrome browser. The agent in it sees the open page, clicks buttons and fills in forms right in your browser.

On 11 March 2025, OpenAI released the Agents SDK and the Responses API for developers, with built-in web and file search. Agents that search and read sites are built on these blocks.

An open path exists too. The Python library browser-use, under the MIT license, controls a browser and can work with a local model through Ollama. A programmer needs to install it, and its speed with a local model depends on your computer.

Where it is useful

Inside Anthropic, the browser agent handled routine work: managing a calendar, booking meetings, drafting email replies, filling in standard expense reports and testing new website features.

These tasks have one thing in common: the service has no convenient API, a program entry point for other systems, and the actions repeat day after day. The agent does what an employee does and clicks the same buttons.

In a small business, this means collecting prices from supplier sites into a table, moving orders from a marketplace account into the CRM, or checking the order form on your own site after an update.

Start with tasks where a mistake is easy to spot and fix: an email draft, a price table, a form check. Back in 2024, Anthropic advised developers to do exactly this: low-risk tasks first, the rest later.

If a system has an API, an agent works through the API faster and makes fewer mistakes than an agent that clicks buttons on a screen. The browser is for places with no other way in.

If the task is to read many sites and put together a report, give it to a research agent. It only reads and clicks nothing, so the risks are lower.

Risks: data, payments, mistakes

The main risk is called prompt injection: someone else’s text on a page or in an email poses as a command, and the agent carries it out. The agent reads everything it sees and cannot always tell your request from the text of a site.

In an Anthropic test of 123 cases across 29 attack scenarios, the browser agent without protection gave in 23.6% of the time, and with protection in autonomous mode 11.2%. An example from the test: an email posing as a security team demand asked the agent to delete emails without confirmation, and before the protection it deleted them.

On 20 August 2025, Brave showed an attack on the Comet browser. Hidden text in a Reddit comment made the assistant take the user’s email from the account settings, get a one-time code from the open Gmail inbox and send both to the attacker.

The usual web security rules do not help here: the agent acts with your rights and your account logins. The more access it has, the more a mistake costs: email, bank, supplier account.

The second threat is a leak. In May 2023, Samsung banned generative AI for its staff after engineers uploaded internal source code to ChatGPT. A browser agent sees everything open on the page, including internal documents.

The third threat is plain mistakes: the agent clicked the wrong button, chose the wrong date or sent a form with a typo. On payments, such a mistake costs money.

Rules for safe use

Agent developers build in limits themselves. In the Anthropic extension, access is granted site by site. The agent asks for confirmation before publishing, buying or sharing personal data, and financial services are closed to it.

OWASP, a community of security experts, puts prompt injection first in its 2025 list of language model risks. Measures from this list: least rights, human approval of important actions, and marking outside text as untrusted.

A separate browser profile. The agent works without logins to the bank, the manager’s email and payment accounts, and sees only the sites the task needs.

Human approval. An employee approves every payment, sent email and publication. Anthropic advises starting with trusted sites and keeping the agent away from financial, legal or medical information.

Watch the first runs. See what the agent does and stop it at the first strange command.

No customer data. Do not pass personal data to a cloud agent in its tasks; if you cannot do without it, it is a project with a developer. The full list of checks before launch is in the agent security checklist.

Sources and fact-checking

Checked by the editors on 16 October 2026.

  1. OpenAI, 11 March 2025 (openai.com)
  2. Bloomberg, 2 May 2023 (bloomberg.com)
  3. Anthropic, 22 October 2024 (anthropic.com)
  4. Claude Blog, 25 August 2025 (claude.com)
  5. Brave, 20 August 2025 (brave.com)
  6. OWASP GenAI Security Project, 2025 (genai.owasp.org)
  7. GitHub, browser-use, 2026 (github.com)

Found a mistake? Tell us

Publications

Contact us