Instacart is posing the provocative question: What if most of the work your engineers do today should, in fact, be done by machines?

At VB Transform 2026, CTO Anirban Kundu argued that dev teams continue to waste their time on draining, repetitive, high-volume work; this should be absorbed by AI agents so that humans can focus on problems that require judgment, intent, and exception handling. 

In fact, in 97% of cases, Instacart’s builders don’t even read code anymore. 

“In the past, the tactical level was the creation of the code,” Kundu said. “In the most tactical level going forward, it's going to be, ‘How do you navigate around the AI system to give you what you want?’”

AI generating code, performing "pretty serious evals" 

That doesn’t mean humans never look at code; agents handle the bulk of code generation and boilerplate, particularly with newer projects where code is generated or regenerated on a weekly basis. 

“The benefit of that is we don't care about tech debt anymore,” Kundu said. “Things that are not active just get dropped out and then it gets rebuilt, kind of like how we used to build assembly code or object code.”

So why not 100%? The remaining 3% is in legacy, compliance, and latency-sensitive systems and workflows, or driven by a “boatload of code” that is dead, not active, or half-active. These cases still need careful human attention. 

Instacart is slowly “smoothing those parts out,” however, breaking systems down in an aptly-named project Atoms, then building them back up in a cleaner, more modular form. Kundu’s team started with the “monoliths” and is shifting to remote procedure call (RPC)-driven architectures. 

But evaluation remains one of the overarching challenges. Code reviews aren’t as relevant when AI is generating code — as Kundu noted, “the lines of code are going to be correct, the syntax is going to meet your expectations” — so the goal is to move to an “intent model.” That is, training devs so they can ask different models the right questions from an intent perspective. 

Evals are then performed independently: Roughly 7,000 automatic evaluations run each month, and the system answers 8,000-plus real-time developer queries with about 99.9% accuracy.

Identifying "hiccups" that human intuition might have missed

Dovetailing with this, Instacart has built an agentic site reliability engineering (SRE) system trained on years of the company’s own incidents and root-cause analyses rather than generic failure data. Instead of teaching a model how production outages work in the abstract, the team fed it the specific ways Instacart’s systems have broken over time, along with the ways humans diagnosed and fixed them.

As a result, the company has seen accuracy in detecting and mitigating production issues jump from roughly 60 to more than 90%. 

Kundu pointed to one example with Instacart’s internal tool Blueberry. The AI SRE colleague watches 200-some-odd Slack channels, monitors signals, and looks for patterns across human conversations and alerts. 

In one incident, a database shard backed by an EBS volume that had a “hiccup” for a period of time. The human team did not immediately suspect AWS disk issues and were “obviously scrambling” to figure out why this particular shard misbehaved. 

But about 20 minutes in, Blueberry posted on Slack, pointing to a specific blip and tying it to a feature-flag-like system called "roulette" that had been inadequate. "It's supposed to be rolling out in this cadence, [but] it had been too much,” Kundu said.

Blueberry figured it out, and the team resolved the incident. “Would have a human been as quick? I think the problem is human intuition would hold us back a little bit,” Kundu said. 

Humans tend to default to patterns we’ve seen before, then resort to debugging; Kundu called this the “first brain-second brain kind of thing.” But Instacart’s agentic SRE is actually “more comprehensive in its ability to look at everything and then be able to decide what does or doesn't matter.”

Redefining the engineer’s job

Looking ahead, the most tactical work for engineers will be navigating AI systems: Designing and supervising evaluation processes; coordinating multiple simultaneous experiments and features; managing constraints like limited top-of-funnel traffic for testing; figuring out when to escalate; identifying edge cases and where things might break.

Domain expertise is also being rethought in the age of AI. Instead of bottlenecking changes through a single “owner” team that touches the code, Instacart is embedding domain knowledge into definitions and specs that any team can use. 

“We’ve lived in this world where this group or this engineering team is the one that can touch the code and make the modification,” said Kundu. “We're trying to move into a world where the code becomes completely democratized across groups.” 

OpenClaw has become one of the most widely adopted agentic frameworks, but it has yet to prove itself at enterprise scale. Agents need real credentials — API keys, OAuth tokens, service accounts — to work effectively, and Brex found that traditional guardrails couldn't contain what those agents were doing with them.

Brex set out to overcome these limitations by building an internal platform it calls CrabTrap. The open-source HTTP/HTTPS proxy intercepts all network traffic, examines policy rules, and uses a LLM-as-a-judge to decide whether agent requests should be approved or denied. 

“What we noticed was that the network layer was an untapped enforcement point,” Brex co-founder and CEO Pedro Franceschi told VentureBeat. “Every request an agent makes is an opportunity to intercept, reason about, and make a policy decision.”

The takeaway Franceschi wants IT leaders to draw: agent governance should shift from SDK-level permissions and model guardrails toward a centralized network control plane that enforces and learns from real in-the-wild agent behavior.

How Brex targeted the transport layer

The “obvious fix” (at least initially) to the agent security gap was guardrails, and much of the early work has centered on scoped tools, per-action permissions, and human-in-the-loop approvals. But as agents evolve, each new capability means there’s another API to tune or surface to audit, Franceschi noted. 

“Any agentic system with multiple tools and access to the open internet creates an immediate tension for builders: The more capable you make an agent, the more dangerous it becomes, and the safer you make it, the less useful it is,” he said. 

Existing solutions to this tradeoff were “weak”: Fine-grained API tokens help at the margins but can still be misused and constrain functionality. Semantic guardrails (such as context, skills, or prompt steering) are easily bypassed by prompt injection, especially for agents connected to the internet.

Agents can be “defanged” when given read-only access or limited toolsets, but then they can't do meaningful work, Franceschi said. On the other hand, granting broad write access and a large tool surface can result in hallucinations and real production consequences.

Model context protocol (MCP) gateways enforce policy at the protocol layer — but only for traffic using MCP. Meanwhile, guardrails from LLM providers are tied to a single model and can be “opaque” to customize with enterprise-specific policies. And powerful tools like Nvidia OpenShell offer more of a “per-sandbox egress control.”

“When we started, we hadn’t found a solution to deploying harnesses like OpenClaw safely,” Franceschi said. “Instead of waiting for the industry to catch up, we decided to own the problem and invent the necessary tools.”

Notably, they needed a platform that sat between every agent and every network request, and could make “nuanced decisions about what to allow,” he said. 

This made the transport layer a core architectural component and natural starting point, he said. 

By operating at this layer, CrabTrap is framework-agnostic, language-agnostic, and API-agnostic. It doesn't require SDK wrappers or per-tool integration. Users set HTTP_PROXY and HTTPS_PROXY in the agent's environment, and every outbound request routes through the proxy before it reaches a destination.

However, Franceschi emphasized, Brex didn't start at the transport layer because it thought it was the only answer; rather, they believe in “security by layers.”

“The transport layer was simply an underinvested one, and we saw an opportunity to add meaningful enforcement there alongside everything else,” he said. 

The LLM-as-a-judge training loop

CrabTrap combines deterministic static rules with an LLM-as-a-judge for requests that fall outside known patterns, Franceschi explained. The judge only “fires on the long tail of unfamiliar endpoints or unusual request shapes,” which for a mature agent is typically fewer than 3% of requests.

The more pressing problem was how to know that a policy is the right one? With static rules, it's “relatively straightforward” to reason about accuracy. But with an LLM judge, the system is nondeterministic, and users need confidence that the policy approves the right requests and blocks the rest.

“Our key insight was to bootstrap policy from observed behavior rather than write it from scratch,” Franceschi said. Beginning with real behavior and editing down based on real-world learnings turned out to be “dramatically more effective than starting from a blank page.”

Brex’s team built a policy builder (itself an agentic loop) that runs underlying agents in shadow mode, analyzes historic network traffic, samples representative calls, and drafts a natural-language policy that matches what the agent actually does. 

From there, they built an eval system that tests policy changes before they go live. CrabTrap compares historical audit entries against a draft policy and reports the exact changes to be made. Users can slice results by method, URL, original decision, and agreement status. 

All of this runs with concurrent judge calls, so replaying thousands of requests “takes minutes, not hours,” Franceschi said. Brex also developed a live feedback loop: Full audit trails are stored in PostgreSQL and queryable through the admin API and dashboard. In cases where a resource is continuously denied, the system can notify a human or an agent to propose a policy update for review. 

“That closes the loop between observed denials and policy refinement,” Franceschi said. 

Core challenges and roadblocks 

Of course, the build wasn’t without its challenges. A big one was latency: “Putting an LLM between an agent and every outbound API request sounds like it would grind things to a halt,” he said. 

However, it didn’t turn out to be as big a problem as expected. This was for two reasons: The LLM judge only activates on a small fraction of requests (the aforementioned 3%). Agents quickly settle into predictable traffic patterns; once observed, high-volume patterns become static rules. Second, by using small, fast models like Claude Haiku meant that, even when the judge did fire, added latency was “negligible.” This can be further reduced with local models and prompt caching, Franceschi said.

The harder and less obvious challenge was prompt injection, he said. The judge receives the full HTTP request and all content is user-controlled, so potentially, a crafted URL, header, or request body could manipulate the judge's decision. 

Brex addressed this by structuring the request as a JSON object before sending it to the model, so all user-controlled content is “escaped rather than interpolated as raw text,” Franceschi said. 

Results, and where CrabTrap might evolve

Brex tracks a few factors to measure CrabTrap’s internal impact: Engagement with agents, network traffic patterns, and net promoter scores (NPS). The most meaningful result of CrabTrap has been “organizational confidence,” Franceschi said. 

Previously, the team had “real hesitation” when it came to deploying autonomous agents broadly across business operations, because the existing guardrail options didn't provide enough assurance. 

“CrabTrap changed that calculus,” Franceschi said. They now have an enforcement layer they trust, increasing confidence around expanding agent deployment into more parts of the business and delegating more agent configuration and management to users. 

Franceschi described the policies derived from traffic as “surprisingly strong.” The team expected the policy builder to produce a “rough starting point” requiring heavy manual editing. In practice, though, pointing the platform at a few days of real traffic produced policies that matched human judgment on the “vast majority of held-out requests.”

Additionally, CrabTrap revealed how much noise agents generate. “The audit trail made this visible for the first time,” Franceschi said. They used denial logs and traffic analysis not only to tune policies, but to tighten agents themselves, remove tools, and cut out entire categories of requests that were wasting both time and tokens.

“The proxy became a discovery tool, not just an enforcement one,” he said. 

Areas for growth (and input from the open-source community)

Brex anticipates CrabTrap to continue to evolve, particularly as they have released it as open-source. “We hope the community helps shape it,” Franceschi said. 

Areas of improvement include deeper authentication functionality such as single-sign on (SSO), fine-grained role-based access control (RBAC); escalation workflows that allow agents to request additional permissions; and policy recommendations based on denial patterns.

Programmatic configuration, or developing API endpoints for “creating, forking, and applying” policies to agents, could allow the whole policy lifecycle to be automated rather than managed manually, Franceschi said. 

As for escalation, if an agent is continuously denied a given resource or endpoint, it should be able to route requests to humans or other AI agents for review and back that up with a rationale for why it needs access. 

“That turns CrabTrap from a hard enforcement boundary into something more like a managed permission system,” Franceschi said. 

Additionally, the policy was built to bootstrap from network traffic, but there is opportunity to incorporate additional signals around agent traces and resource-calling, as well as broader context on what agents are ultimately trying to accomplish. This can help produce more accurate and nuanced policies. 

Finally, there's an “open philosophical question” about the right posture for CrabTrap: Should it be a fully transparent layer that the agent itself is unaware of, or should it operate more like a “well-intentioned manager”? (that is, the agent knows about the layer and can interact with it).

The open-source community can help shape these developments, and CrabTrap will only get better with more users, Franceschi said. Brex’s agents speak to a specific set of APIs; teams using CrabTrap with different agents, services, and policy requirements will surface “edge cases and patterns we can't hit alone.”

“We have ambitious plans for where it could go, and we’d rather build in the open,” Franceschi said. 

What other builders can learn from CrabTrap

The response has been stronger than expected. CrabTrap has more than 700 stars on GitHub. Franceschi said Brex has also heard from OpenAI, Y Combinator CEO Garry Tan, and programmer Pete Steinberger, all expressing interest in deploying similar internal infrastructure.

The broader lesson: “Don't let infrastructure gaps become excuses to wait," Franceschi advised. There are “real blockers” for every enterprise looking to seriously deploy AI agents, including security concerns, lack of tooling, or unclear guardrails. 

“It's tempting to sit on your hands until the industry catches up,” he said. “The lesson from CrabTrap is that you can own those problems directly.”

When Anthropic's Fable 5 was pulled from international use for nearly three weeks, some over-reliant businesses were left scrambling.

But Liberty Mutual easily pivoted to other platforms. That’s because 18 months earlier, they built their "AI backbone" exactly for this kind of scenario.

In this rapidly moving AI landscape, the 114-year-old property and casualty insurance company recognized independence as an operating advantage.

“Things are changing so fast, you need a backbone that's flexible,” Brian Craig, Liberty Mutual’s senior director of architecture, said at a recent VB Impact event. “You can't lock in right now on one vendor or even one framework.”

Enterprises need flexibility to hook into different models and vendors, depending not so much on the "flavor of the day," but “what can you feel confident about for the next six months,” he said.

Runtime versus control plane

The company’s "backbone" (or control plane) is its own, while everything underneath remains swappable.

The architecture consists of roughly 50 components across security, identity, orchestration, tool restriction, and the policies that govern how agents behave. Each is designed to be independently and immediately replaceable to support interoperability.

The agent runtime below this backbone is AWS's Amazon Bedrock AgentCore; this is not the strategic center, but explicitly “just for running the agents,” Craig said. He chose this offering because it (at least currently) supports multiple frameworks and Liberty Mutual’s model-agnostic philosophy.

“We still have flexibility based on what we write,” Craig said. “But if something comes along and is better, we will move to it quite quickly.”

The software factory

This architecture delivers, as proven by Liberty’s “software factory,” an agentic pipeline that automates much of the software delivery process.

They started with a business process with clear pain: Onboarding electronic content management documents for insurance products. This repetitive, manual task typically required engineers to code every change.

Instead, the team built a factory of coordinated agents working “in tandem and in sequence”:

• An Epic agent consumes high-level requirements.

• A Story agent breaks work dictated by the Epic agent into narrow slices within specific application areas. This agent is “constraining the context, because the smaller the context, the better the output.”

• A planning agent defines the technical execution plan.

• A coding/testing agent handles coding, testing, and basic review.

• A triage (critic) agent sits across all other agents, reviewing quality and feeding back improvements.

• Finally, a librarian agent helps others find the “context of the knowledge for their job.”

Craig and his team learned quickly that a single “do everything” agent was a mistake. “You were asking it to do too many things, which meant you had to give it too much information,” Craig said. Splitting into six agents let them dramatically shrink context windows and tighten scope.

Once the factory hit production, the impact was immediate. In the initial deployment, they did “about three months of work” in roughly a week. They realized that “the current software engineering process has a massive amount of handoffs, which means there's a massive amount of wait time,” Craig said.

Human-paced automation

The factory is not a fully autonomous pipeline; it runs at the speed of human overseers.

Liberty’s first model was a “day shift/night shift” rhythm: Engineers set goals and rules and reviewed the previous night’s outputs during the day, then let the factory run overnight. But in practice, there was never enough work to keep the agents busy all night, and the cadence felt unnatural.

They shifted to a more iterative loop. Users decide when to trigger the factory, how far it runs before pausing, and at which points they want to review outputs. “Then the factory kicked in, and it may only run for less than an hour, and then you would look at it again,” Craig said. “It was more controlled at the speed that our users felt comfortable with.”

The orchestration layer lets teams choose whether to review after the Epic stage, after planning, or once coding and testing complete. “That is up to the users of the factory, but it has completely removed a lot of the wait time that we currently have within our processes,” Craig said.

By contrast, early on, “every time the thing ran, they wanted to look at the output,” and that became the feedback loop that trained both humans and agents.

Some of this is, as Craig put it, “just automating agile at speed” and giving iterative feedback; some from humans, some from the triage agent. “But we're seeing that start to bake in, and it starts to become the rules.”

When the same feedback is coming from two different directions (agent and human), it makes sense to go into Liberty’s context repository. The agents will then rely on that context the next time they need to make that decision, "and the next time again, and you start to speed up."

“It's like a flywheel once you start building these and you start to get them flowing,” Craig said. “You realize it listens to what you say.”

Contracts that match the pace of change

Liberty paired that architectural posture with a contract posture, deliberately shifting from five-year enterprise deals to one-year agreements. The logic is simple as Craig sees it: The AI market moves too fast to lock into one vendor or framework for half a decade or more. Shorter terms let them evaluate — and if necessary, swap — models and platforms at the speed the market actually changes. Cost is part of the story. When a premium frontier model like Fable arrives, the sticker shock is real. “You see the price and go, ‘Goodness, it better be really good,’ Craig said. (It was; they got to use it enough to "fall in love with it.”) The backbone’s interoperability lets his team compare models at different price points and route workloads based on price–performance rather than vendor inertia. The same attitude governs how they plan to use agents from major SaaS platforms. Liberty is a customer of Salesforce and Splunk, who will both “bring their agents to the table.” His team has no interest in replicating engineering work, but they do insist on observability. “We just want to harness it as part of our system,” rather than control it, Craig said. “But we want the observability to understand what their agent is doing with our data, with our users.”

Closing the “control gap”

Importantly, Liberty built observability into its backbone. As Craig explained, it’s not just logging what an agent does, but what it accesses, which identity it uses, and which tools it’s allowed to invoke. Identity and access run in part on Microsoft Entra ID, and agents are given only the tools and permissions explicitly assigned to them. Whenever an agent realizes, ‘I don't have the information,’ it asks for it, and it only gets what it needs, rather than giving authority to “use every tool in the box.” “Because too much information given to an agent is worse than no information,” Craig said. “You just overload it, and it gets confused.” For detection, Liberty runs evaluations with MLflow against “golden datasets.” Whenever prompts or models change, they regression-test and immediately see whether results improved or degraded. One of his team’s new mantras is “you need to walk in the footsteps of a new start." If a new start can't find a guiding document, how will an AI agent? One of the key things enterprises need to do, no matter the business process, is “write stuff down, which is not earth-shattering,” Craig acknowledged. Agents obey written standards more reliably than people, and a context repository is a central artifact of Liberty Mutual’s system. Agents have made human judgment more, not less, central, he emphasized. Nothing ships without a human sign-off, consistent with Liberty’s risk posture as an insurer. “The confidence isn't there yet for us to just let it run wild, and I don't think it ever will be for the likes of Liberty,” said Craig. “We have to be rock solid before we let [anything] into production.” Moving fast for product fit and survival might make sense for other companies, but ultimately, “some people will get black eyes, but that's the joy of innovation these days,” he said.

Most verticals aren’t clean, well-oiled SaaS databases; the reality is ugly documents, proprietary schemas, implicit workflows, and long‑running tasks that most general-purpose models struggle with.

This prompted construction project management company Trunk Tools to build a specialized, three-layer architecture — perception, semantics, agents — based on highly-detailed data to support high-accuracy, highly-relevant industry automation.

Their purpose-built stack has shrunk review cycles from months to days, prevented costly field errors, and given autonomous agents the ability to reason over millions of pages of documentation, the company says.

“We really set out to take the data from dispersed systems, pre-process it, structure it, go through our ontology into a knowledge graph, and then train AI models,” said Sarah Buchner, Trunk Tools' founder and CEO and a former carpenter.

For builders in other verticals, the company's approach could serve as a blueprint for transforming data chaos into agent‑ready, industry-specific workflows.

Where general-purpose LLMs break down on industry data

Foundation LLMs, while powerful, are optimized for breadth, not always depth.

“General-purpose LLMs are trained to be okay at everything, so they're weak at anything niche,” said Kriti Faujdar, a senior product manager working in AI infrastructure, agentic AI, security, and LLM platforms. For instance: Rare terms, domain-specific reasoning, the unspoken context that any practitioner “just knows.”

Web, app, and software developer Sébastien De Bollivier agreed that the biggest bottleneck is reliability on data that is “jargon-dense, abbreviation-heavy, and format-specific.”

“A GPT-4-class model can understand a French legal contract, but will fumble the specific article references practitioners need to cite,” he said.

Besides, the most valuable enterprise data never made it into pretraining anyway, Faujdar pointed out. It's sitting in internal systems and proprietary formats. “RAG helps a little,” she said. “But it's just giving better facts to a model that still can't reason properly in the domain.”

Pre-training on domain data is critical; enterprises should then fine-tune on good task examples and build their own evals. “A few thousand examples from real practitioners beats millions of scraped, noisy ones," Faujdar said.

Mixture-of-experts (MoE) can provide specialization without inference costs blowing up. Pairing RAG with fine-tuning also works well; RAG handles the factual long trail while fine-tuning fixes vocabulary and reasoning.

De Bollivier pointed to the advantage of hybrid stacks: A general-purpose model for reasoning and orchestration, a smaller fine-tuned model (or dense retrieval over a curated corpus) for domain-specific extraction. He advised: “Don't fine-tune to make the model 'smarter' about a domain, fine-tune to make it more reliable on the specific output format your workflow requires.”

The trades and construction are certainly industries seeing traction with these techniques, as are legal and healthcare, De Bollivier said. These verticals have “high stakes for errors plus standardized document formats, equaling clear domain-training ROI.”

One honest caveat worth mentioning, Faujdar said: Specialized models can often fall apart outside their domain, so they’re often not useful outside their expertise (unless they’re re-trained).

Perception, semantics, agents: inside Trunk Tools' three-layer stack

In highly-specialized domains like construction, “data dumps” into large language models (LLMs) don’t cut it, said Trunk Tools' CTO Amrish Kapoor. This is because most transformers are probabilistic models: When given an image, they report back that it is “probably” a tree, or “probably” a child playing next to a tree.

This makes them insufficient for high‑precision symbolic interpretation. For instance, in construction documents, a 2-millimeter-wide symbol has a vastly different meaning depending on where it’s placed.

Further, constrained by context limits, probabilistic models struggle with long‑term project memory. “I don't mean a context window of a few tokens,” Kapoor said. “I'm talking about long term memory that stretches across months and years, because this is how long some of these projects are.”

Instead, the company's three-layer system breaks workflows into:

• Perception (reading and extracting data from messy docs like PDFs, drawings, or scans)

• A semantic/graph layer (making sense of that data and understanding their relationships).

• LLMs and agents on top.

Construction drawings are typically symbolic, Buchner said. A door isn't always labeled ‘door.’ Sometimes it's simply an arc on a wall that a trained eye learns to read based on years of practice.

“The perception layer is what teaches AI to read that language,” she said. The semantic layer then gives that information meaning; for instance, connecting the door to the drawing that details it, the spec that governs it, and the trade that installs it. This helps answer project engineers’ critical questions: Not "is there a door here?" but "does this door create a problem down the line?"

Particularly in construction, that shift matters because the cost of a problem compounds with time. “A conflict caught in design is relatively low cost to address,” Buchner said, “whereas the same problem caught in the field might cost tens of thousands of dollars.”

At a high level, the system identifies the document type and begins extracting information based on content (drawing, schedules, paragraph text). This data is then “transformed and augmented” in the platform, which triggers agentic workflows like knowledge graph relationships and end-user workflows.

For instance, an agent might review an architecture bulletin and produce a visual overlay comparing an older version and a newer version (flagging additions and removals), then generate written narratives that describe what those changes are in simple terms. This helps users understand what’s changed and coordinate with trade partners on updated pricing and change orders.

The scale of construction’s data problem

Construction workflows are “ripe with implicit assumptions and connections between data in its myriad of sources,” Buchner said. And the amount of unstructured data is “humanly impossible” to process or make sense of.

Buchner estimated the average high-rise building generates about 3.6 million pages of corresponding documentation. “If you print it into a stack of papers it would be as high as the building itself.”

All three layers of Trunk Tools' stack — perception, semantic, LLM — are trained on “very specific datasets” from customers with “explicit permissions” and auto‑labeling/IP, Kapoor explained. Customers who don’t want Trunk training on their data can opt out.

Data is deidentified and aggregated, and Trunk Tools also collects “tons more” labeled data through other pipelines like 3D building information modeling (BIM).

The company says it only ships agents that achieve around 95% accuracy. The team maintains continuous evaluation pipelines based on ground truth data from customers and experts. They also employ an LLMs-as-a-judge model.

“This notion of an LLM as a judge is to score how well you're doing, both subjectively as well as objectively,” Kapoor said. Objectivity can be an easy ‘right’ or ‘not right,’ but subjectivity requires more nuance.

For instance, when creating an email or narrative or explanation, an LLM as a judge framework can create a composite score, or a numerical value that aggregates different metrics and tests a model's performance or risk.

There can be challenges, though, particularly with latency, Buchner noted; any time the reasoning capacity of underlying models increases, the risk of latency goes up, too. Trunk Tools maintains a set of evaluation criteria to objectively measure latency whenever changes are made to underlying infrastructure, agents, and API calls.

Then, “before we release to customers, we ensure marginal changes to the end-user experience are well worth the performance enhancements,” Buchner said.

From 60 days to 10: the measurable payoff

Trunk Tools' platform powers seven AI agents purpose-built for construction, such as analyzing request for information (RFI) responses, overviewing bids, or reviewing drawings and submittals.

The submittal agent, for instance, flags missing, conflicting, or noncompliant information in product specs and RFIs. While it’s an essential step in the construction process, “it's a super annoying workflow,” Buchner said, because human reviewers have to compare documents “with a bunch of other parts of documents.”

But the agent is able to do this in seconds, and Trunk Tools says it has reduced submittal cycles from 50 to 60 days to 10, “which has massive schedule and financial implications.”

The company is now at a place where these agents are communicating directly with each other, which is “quite exciting,” Buchner said. So, for example, one agent will review an architectural drawing for accuracy, then autonomously hand it over to agents handling RFIs and asking follow-up questions.

“If the drawings have problems, the RFI agent is taking over and is actively reaching out for clarification,” Buchner explained.

Trunk Tools says its customers report savings of 20 to 40 minutes per field question. Buchner said that users in the field know better than anyone how much of a “time suck” it is to go back and forth from office trailers, dig through project documents in scattered systems or printed PDFs, reconcile discrepancies, and return to coordinate with trade partners.

The company says its customers report these additional outcomes:

• Average 8 minute time savings for single-document retrieval (status checks, location lookups, quantity queries).

• Average 20 minute time savings for standard referencing (cross-referencing 2 to 3 spec sections to form an answer.

• Average 40 minute time savings for multi-document research (listing and filtering queries, mapping relationships, analyzing RFIs and submittals across 4 to 6 documents).

• Average 75 minute time savings for complex tasks (creating RFIs and other communication materials, deep cross-referencing across documents, change tracking).

In one instance, the company's drawing review agent flagged that a structural beam had been moved up 8.5 inches. However, this was not documented by the architect. If the change hadn’t been caught, the project manager would likely have had to strip out and reinstall the right size beam, Buchner said. This rework would have added $10,000 or more to the budget, and “certainly there would have been implications on the schedule.”

Buchner also pointed to other examples: an agent flagged $60,000 in exaggerated pricing with no justification from landscaping subcontractors; identified a fireplace that needed to be sealed prior to drywall installation, saving around $100,000 in labor, materials, and delays; and called out that an electric door required a panel that wasn’t included in electrical drawings.

Learnings for other industries

Trunk Tools' approach to building agents is applicable to any vertical working with high volumes of unstructured, industry-specific data. Builders working in specific verticals must understand the industry’s specific data challenges their end users face and build technical infrastructure that can transform unstructured data into something an “LLM can traverse and understand,” Buchner said. “Only then can you build the connections between data points that ultimately feed agentic workflows.” A lot of money is being invested in foundational models, so enterprises should build modular systems that can leverage the strengths of various models as they continue to improve, Buchner advised. Then, “build your technical advantage where the generic models are not investing and not performing well,” she said.