Born out of San Francisco AI hackathons, Agency lets you see what your AI agents are doing

After an extended week of coding, you might think that San Francisco builders would retreat to the mountains, beaches, or the Bay Area’s vibrant club scene. But in point of fact, because the week winds down, AI hackathons begin.

Over the past few years, San Francisco has exploded with AI hackathons. Every Saturday or Sunday, technologists give talks on the most recent advances in AI, networking, and—most significantly—turn ideas into working demos. Sometimes hackathons offer money or cloud credits as prizes, but the true winners walk away with a way of a startup.

“There’s no better place in the world to build the most ambitious project of your life than San Francisco,” says agency co-founder Alex Reibman. “You often see a lot of competitions—like hackathons—but they’re not competitive. They’re as collaborative as they are competitive.”

At a hackathon in San Francisco last summer, Reibman decided to try his hand at constructing AI agents that would crawl the net. Agents are a hot topic in Silicon Valley because the AI ​​boom reaches its peak. The term just isn’t precisely defined, but it surely broadly describes AI bots that may perform tasks robotically using interfaces and services that weren’t originally designed for automation—a sort of alternative for mundane tasks that when required human intervention.

But Reibman immediately bumped into an issue. “They sucked,” Reibman said in an interview. “The agents failed 30 to 40 percent of the time, and often in unexpected ways.”

To fix this, Reibman’s team built internal debugging tools to see where their agents were going mistaken. They eventually managed to get the agents to work a little bit higher, however the debugging tools themselves ultimately stole the show and won the hackathon.

“I started showing the tools at a lot of hackathons and events in San Francisco, and people started asking for access to them,” Reibman said. “That was basically the confirmation I needed: instead of building an agent ourselves, we should build tools that make it easier to build agents.”

So Reibman founded Agency along with his co-founders Adam Silverman and Shawn Qiu, offering tools to look at what AI agents are actually doing and catch where they’re going mistaken. A yr later, those tools eventually became Agency’s core product, the AgentOps platform that 1000’s of teams use every month, Reibman tells TechCrunch. The startup has already raised $2.6 million in pre-seed funding, led by 645 Ventures and Afore Capital.

COO Adam Silverman tells TechCrunch that AgentOps is like “multiple device management for agents,” analyzing all agent actions to make sure they don’t go down a rogue path.

“You want to understand whether your agent is going to act dishonestly and determine what limitations you can put in place,” Silverman said in an interview. “A lot of the work is being able to visually see where your guardrails are and whether agents are abiding by them before you put them into production.”

The startup is partnering with Cohere and Mistral, AI modelers who also offer agent creation services, so customers can use the AgentOps dashboard to see how agents interact with the world and the way much each costs. Agency is model-agnostic, meaning it really works with several different AI agent frameworks, but it surely integrates with popular tools like Microsoft’s AutoGen, CrewAI, and AutoGPT.

In addition to the AgentOps dashboard, Agency also offers consulting services (Reibman previously worked at consulting firm EY) to assist firms start constructing agents. The agency wouldn’t disclose any clients by name, but said hedge funds, consultants, and marketing firms use its tools.

For example, Reibman says Agency helped create an AI agent that writes blog posts concerning the firms a client does business with. Now, that very same client uses the AgentOps dashboard to trace agent performance and costs.

Big players like OpenAI and Google are prone to ramp up their agent products in the approaching months, and AI startups like Agency need to search out a option to work with these advances, not against them.

“There are so many layers in the stack that it’s unlikely that an LLM vendor would try to cover all of them,” Reibman said. “OpenAI and Anthropic are building tools to create agents, but there are a lot of layers around them that make sure you have a production-ready code base.”