Brex built AI agent policy not by first writing the rules, but by observing what agents actually do



OpenClaw has become one of the most widely used agent frameworks, but it has yet to prove itself on an enterprise scale. To work effectively, agents need real credentials—API keys, OAuth tokens, service accounts—and Brex found that traditional safeguards can’t contain what these agents do with them.

Brex set out to overcome these limitations by building an in-house platform it calls CrabTrap. The open source HTTP/HTTPS proxy intercepts all network traffic, examines policy rules, and uses LLM as an arbiter to decide whether to approve or reject agent requests.

“What we focused on was that the network layer was an untapped application point,” Brex co-founder and CEO Pedro Franceschi told VentureBeat. “Every query an agent makes is an opportunity to intervene, reason, and make a policy decision.”

Franchise wants IT leaders to engage: agent management must move from SDK-level permissions and model guards to a centralized network management plane that enforces and learns from real-world agent behavior.

How Brex targeted the transport sector

The “obvious solution” to the agent security gap (at least initially) was guardrails, and much of the early work focused on comprehensive tools, per-activity permissions, and circulating human approvals. Franceschi noted that as agents evolve, each new capability means another API to surface to adjust or inspect.

“Any agent system with multiple tools and access to the open internet creates an immediate tension for developers: the more skilled you make an agent, the more dangerous it becomes, and the less secure you make it, the less useful it is.”

Existing solutions for this exchange were “weak”: Fine-grained API tokens help at the edges, but can still be abused and limit functionality. Especially for Internet-connected agents, semantic safeguards (such as context, skills, or operational management) can be easily bypassed by rapid injection.

Agents can be “defended” when given read-only access or limited toolsets, but then they can’t do meaningful work, Franceschi said. On the other hand, giving a large writing opportunity and a large instrument surface can result in hallucinations and realistic production results.

Model context protocol (MCP) gateways apply policy at the protocol level, but only for traffic that uses MCP. Meanwhile, LLM providers’ safeguards are tied to one model and can be “transparent” to customize with enterprise-specific policies. And powerful tools like Nvidia OpenShell offer more “per-sandbox output control.”

“When we started, we didn’t have a solution for securely deploying attachments like OpenClaw,” Franceschi said. “Instead of waiting for the industry to catch up, we decided to own the problem and invent the necessary tools.”

In particular, he said, they need a platform that sits between each agent and each network request and can make “nuanced decisions about what to allow.”

That made the transportation layer a key architectural component and a natural starting point, he said.

By working at this layer, CrabTrap is framework-agnostic, language-agnostic, and API-agnostic. It does not require SDK wrappers or per-tool integration. Users have been assigned HTTP_PROXY and HTTPS_PROXY in the agent environment and every outgoing request goes through a proxy before reaching its destination.

However, Franceschi emphasized that Brex did not start with the transport layer because he thought it was the only answer; rather, they believe in “security in layers.”

“The transportation layer was just an underinvested layer, and we saw an opportunity there to add meaningful execution on top of everything else,” he said.

LLM-judge training period

CrabTrap combines deterministic static rules LLM-as a judge For requests that deviate from known patterns, Franceschi explained. The judge only “fires on the long tail of unknown endpoints or unusual request forms,” ​​which for a mature agent is typically less than 3% of requests.

The most pressing problem was how to know if the policy was right? It is “relatively simple” to think about accuracy with static rules. But with LLM refereeing, the system is indeterminate, and users must ensure that the policy approves the correct requests and blocks the rest.

“Our key concept was to load the policy from observed behavior rather than writing it from scratch,” Franceschi said. Starting with real behavior and editing based on real-world learnings was “dramatically more effective than starting with a blank page.”

The Brex team built a policy maker (itself an agent loop) that runs key agents in shadow mode, analyzes historical network traffic, samples representative calls, and produces natural language policies that match what the agent actually does.

From there, they set up a scoring system that tests policy changes before they go live. CrabTrap compares historical audit logs to the draft policy and reports the exact changes to be made. Users can slice results by method, URL, original decision, and contract status.

All of this happens simultaneously with referee calls, so rereading thousands of requests “takes minutes, not hours,” Franceschi said. Brex also designed a live feedback loop: Full audit trails are stored in PostgreSQL and can be queried via the admin API and dashboard. In cases where a resource is continuously denied, the system can notify a human or agent to suggest a policy update for review.

“This closes the loop between observed denials and policy refinement,” Franceschi said.

Main problems and obstacles

Of course, the construction was not without difficulties. It was a huge delay: “Sounds like putting LLM between the agent and every outgoing API request would cause a hang,” he said.

But it was not as big a problem as expected. There were two reasons for this: the LLM judge is only activated in a small fraction of requests (the 3% mentioned above). Agents quickly adapt to predictable traffic patterns; when observed, high-volume patterns become static rules. Second, using small, fast models like Claude Haiku meant that the extra delay was “total” even when the referee fired. This can be further reduced with native models and fast caching, Franceschi said.

A more difficult and less obvious problem, he said, was the emergency injection. The referee accepts the full HTTP request and all content is controlled by the user, so potentially a crafted URL, title or request body can manipulate the referee’s decision.

Brex solved this by structuring the request as a JSON object before sending it to the model, so all the user-driven content “ran rather than being interpolated as raw text,” Franceschi said.

The results and where CrabTrap can improve

Brex tracks several factors to measure the internal impact of CrabTrap: Engagement with agents, network traffic patterns and net promoter scores (NPS). Franceschi said the most meaningful outcome of CrabTrap was “organizational trust.”

Previously, the team faced “real hesitation” when it came to deploying autonomous agents widely across business operations, as existing guardrail options did not provide enough security.

“CrabTrap changed that score,” Franceschi said. Now they have an execution layer they trust, increasing the confidence to extend agent deployment to more parts of the business and delegate more agent configuration and management to users.

Franceschi described traffic-derived politics as “surprisingly powerful.” The team expected the policymaker to develop a “rough starting point” that would require heavy manual editing. In practice, targeting the platform to a few days of real traffic generated policies that fit human judgment on the “vast majority of requests caught.”

In addition, CrabTrap revealed how much noise agents make. “The audit trail made it visible for the first time,” Franceschi said. They used denial logs and traffic analysis not only to adjust policies, but also to harden agents themselves, remove tools, and cut off entire categories of requests that waste both time and engagement.

“The proxy has become a discovery tool, not just an enforcement tool,” he said.

Areas for growth (and input from the open source community)

Brex expects CrabTrap to continue to evolve, especially since they are releasing it as open source. “We hope the community will help shape it,” Franceschi said.

Areas of improvement include deeper authentication functionality such as single sign-on (SSO), fine-grained role-based access control (RBAC); escalation workflows that allow agents to request additional permissions; and policy recommendations based on denial patterns.

Programmatic configuration, or developing API endpoints for agents to “create, fork and apply” policies, could allow the entire policy lifecycle to be automated rather than manually managed, Franceschi said.

As for escalation, if an agent is continuously denied a resource or endpoint, it should forward requests to humans or other AI agents for review and support it with a rationale for why it needs access.

“This makes CrabTrap more like a managed permit system than a hard enforcement boundary,” Franceschi said.

In addition, the policy is built to load from network traffic, but has the ability to include additional signaling around agent traces and resource invocation, as well as additional signaling in the broader context of what the agents are ultimately trying to accomplish. This can help develop more precise and nuanced policies.

Finally, there’s an “open philosophical question” about the right posture for CrabTrap: Should it be a fully transparent layer that the agent itself doesn’t know about, or should it act more like a “benevolent manager”? (ie the agent knows about the layer and can interact with it).

The open source community can help shape these developments, and CrabTrap will only get better with more users, Franceschi said. Brex agents talk to a specific set of APIs; Teams using CrabTrap with different agents, services and policy requirements will uncover “recent events and patterns that we couldn’t hit on our own.”

“We have ambitious plans for where it can go, and we clearly prefer to build,” Franceschi said.

What other builders can learn from CrabTrap

The response was stronger than expected. CrabTrap has over 700 stars on GitHub. Franceschi said Brex has also heard from OpenAI, Y Combinator CEO Garry Tan and programmer Pete Steinberger, all of whom have expressed interest in deploying similar internal infrastructure.

The broader lesson: “Don’t let infrastructure gaps become an excuse to wait," Franceschi advised. For any enterprise looking to seriously deploy AI agents, there are “real blockers,” including security concerns, lack of tools, or unclear safeguards.

“It’s tempting to sit on your hands until the industry catches up,” he said. “The lesson from CrabTrap is that you can own these problems head on.”



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *