hiw3c 日报精选
IEEE Spectrum AI 网页快照

认识Shepherd,这款Meta代理,用于约束恶意AI。

原文标题 · Meet Shepherd, the Meta-Agent Keeping Rogue AI In Check

AI代理在互联网上肆虐,配备执行特定任务的指令,但往往会采取不良或意外的行为。这些失控事件涵盖从删除整个邮件收件箱到攻击网站乃至发起网络攻击等多种情况。正如人类员工需要管理者一样,东北大学助理教授施伟安认为,AI代理需要监管。“它们运行的时间可能非常长。然而,运行时间越长,它们可能会出现一些错误,并可能采取一些危险、不安全的行动。我们认为应该有一个中间

来源:IEEE Spectrum AI · spectrum.ieee.org

正文为英文,可一键机器翻译(仅首次需要等待)

Topics

Sections

More

For IEEE Members

For IEEE Members

IEEE Spectrum

Follow IEEE Spectrum

Support IEEE Spectrum

Access Thousands of Articles — Completely Free

Create an account and get exclusive content and features: Save articles, download collections, and post comments — all free! For full access and benefits, subscribe to Spectrum .

Meet Shepherd, the Meta-Agent Keeping Rogue AI In Check

The tool uses version control and sandboxing to recover from AI missteps and “manage” agents

Rina Diane Caballar is a contributing editor covering tech and its intersections with science, society, and the environment.

AI agents are running amok on the internet, armed with commands to accomplish certain tasks but often acting in undesirable or unexpected ways. These rogue incidents run the gamut from deleting an entire email inbox to hacking websites and even launching cyberattacks .

Much like human workers need managers, Weiyan Shi , an assistant professor at Northeastern University , believes AI agents require oversight. “They are running potentially for a long time. However, the longer they run, they may have some errors, and they may make some dangerous, unsafe actions. We feel like there should be a middle layer,” she says.

This middle layer comes in the form of meta-agents that oversee behavior, provide quality control, and kick issues up the chain of command to a human when appropriate. “Worker agents are like the sheep wandering around, and the meta agent is like this shepherd trying to get them back on track,” Shi says.

As such, Shi and her colleagues at Northeastern University and Stanford University named their framework Shepherd , an intelligent guide for agentic sheep. To arm meta-agents with power, the team borrowed a concept from software development : version control. The accompanying paper will be presented at the 2026 NeurIPS conference to be held in December.

Version control for AI agents

Version control systems , such as the popular GitHub , track and manage changes to source code over time. These tools allow software engineers to collaborate on the same codebase without losing work. They can “fork” or create a copy of the codebase that they develop independently, “commit” to save a new version with changes, review revisions, and roll back to a previous version when an error or failure occurs.

Similarly, Shepherd serves as a version control system for AI agents. It’s built using the Python programming language and rooted in functional programming principles. Each agent action, along with the AI agent’s state and its relevant environment (such as an email inbox, a file system, or other) is stored as a commit, with a commit graph consisting of a series of actions. Meta-agents traverse the graph to navigate to any commit, enabling them to observe worker agents, intercept a buggy action, revert the worker to its previous step, and fork it to try another direction.

“There’s a subtlety because it looks to [worker] agents like they’re in the regular world where they can do things like send emails. But in actuality, the action of sending the email has been caught by a Shepherd sandbox, so a meta-agent or a human can come in and decide to actually send it,” says Derek Chong , a researcher at Stanford University’s Natural Language Processing Group who’s part of the Shepherd team. This sandboxing, he adds, functions as a guardrail or governance component.

“If there is a middle layer to help govern these worker agents, then that will help humans have better control,” Shi says. “Humans will still, eventually, have a say, and they can intervene with the meta-agent. They can also help steer these meta-agents.”

To demonstrate Shepherd’s feasibility, the team created three use cases. One used a supervisor meta-agent that intervened when conflict happened between parallel programming agents, helping lift their pair coding pass rate on a particular benchmark. Another developed an optimization meta-agent that proposed targeted edits to a worker agent’s finished workflow and tested them by replaying only parts of the run affected by the edits, helping improve performance. Finally, a training meta-agent during reinforcement learning picked points or steps at which to fork so the worker agent takes a better path, rewarding it along the way.

Shepherd is available as an open-source project , but Chong notes that the team is still in the process of building it from alpha to full production readiness. The Shepherd framework is part of the broader Agent Sciences Research Initiative (ASRI) cofounded by Chong and Shi. This collaboration of researchers from Northeastern, Stanford, and UC Berkeley aims to explore how to make AI agents more reliable and robust, how to evaluate them better, and how to ensure they’re safer.

“It’s becoming more and more clear that agent management is really important,” Chong says. “Agents should be able to inspect other agents and operate on them, just the way that management in real life can happen. We think Shepherd is the way we can build toward that and enable agent management to be democratized. And if you can democratize that, then humans can be empowered to build AI systems that support human flourishing.”

Attempts to Keep Humans in the AI Loop May Actually Push Them Out ›

Are You Ready to Let an AI Agent Use Your Computer? ›

To Thrive in the AI Era, Companies Need Agent Managers ›

What is AI Agent Management? | IBM ›

Rina Diane Caballar is a writer covering tech and its intersections with science, society, and the environment. An IEEE Spectrum contributing editor, she's a former software engineer based in New Zealand.

I Don’t Trust Visual Recordings Anymore. But One Day I Will Again

28 Zero-Day Findings in Under Two Days. Is Robotics Security Keeping Pace?

Why a Computer Scientist Thinks Superintelligence Won’t Happen

Related Stories

Can Agentic Robotics Solve Robotics’ Data-Scaling Problem?

Attempts to Keep Humans in the AI Loop May Actually Push Them Out

Why Read a Research Paper When You Can Turn It Into an AI Agent?