Rollout and scaled adoption: agents become part of the team

When we last left Box’s Intelligent Prospecting Agent (IPA), it was in the thick of its Pilot phase — proving its worth to a select group of sales reps.

The agent had already demonstrated impressive gains: cutting message creation time from 30–60 minutes down to 5, while enabling personalization at a previously impossible scale. 

As such, we decided to move the agent from a pilot into “Rollout” and then “Scaled Adoption”, the last two phases of Box’s four-part approach to developing AI agents:

Ideation → Pilot → Rollout → Scaled Adoption

Here the story becomes both more interesting and more uncertain. Because moving from a successful pilot to production-ready rollout means handling a very complicated set of different problems - systems integration, change management, and what it means to have AI agents as permanent members of your team.

The Ideation and Pilot phases are about discovering and proving value. The final two are about turning that value into something real, measurable, and durable across the business. This is where AI initiatives either quietly fade out — or actually change how work gets done.
 

From “it works” to “we can trust it”

In Pilot, the IPA had one goal: to prove that it could do what its creators designed it to do. Did it save time? Did reps actually use it? Did it help them do work they otherwise wouldn’t have done?

By the time an agent reaches Rollout, those questions have already been answered. “In Pilot, it does what you said it was set out to do,” says Robert Ferguson, Box’s Head of Corporate Strategy and Chief of Staff to the CEO. “It hits a reliability and an accuracy threshold.” 

 

Rollout is the stage at which Box decides whether, for any new AI agent, that bar has been met.
 

Robert Ferguson, Box’s Head of Corporate Strategy and Chief of Staff to the CEO

But that reliability bar isn’t one-size-fits-all. An ideation agent that helps you brainstorm headlines can tolerate a little noise. But an agent like the IPA, which drafts customer-facing messages, must be held to a much higher standard. “Anything where an AI is interacting with customers and prospects,” notes Ferguson, “you have a high bar.”
 

Rollout is the stage at which Box decides whether, for any new AI agent, that bar has been met. So far, based on our early Pilot metrics, the IPA is clearing it.  

What “production-ready” actually means

Once an agent clears the pilot metrics, the work is far from over; to roll our agent out to hundreds of people, it has to be operational. That means four things.
 

1. Make sure only the right people get access 

An agent that works beautifully in Pilot can create chaos if it finds its way into the wrong hands. “There are certain agents where you don’t want to give access to everybody,” Ferguson notes. “Some of it is just permissions. Or that you don't want everybody going and seeing 28 agents that aren’t relevant.” 

For the Intelligent Prospecting Agent, that might mean AEs and SDRs but not engineers or finance teams. The rollout plan defines exactly who should see what.

2. Lock down the technical plumbing 

Some agents are simple. Others, like the IPA, pull from Salesforce, product usage data, and content Hubs. In those cases, Box’s technical and design teams will step in. “Any time there’s an integration with other data and systems,” says Ferguson, “someone on design and the whole team need to make sure that they’re happy with the technical implementation before they sign it off.”
 

3. Ensure knowledge is trustworthy

During the Pilot, an agent might point at a handful of documents or a snapshot of messaging. That’s fine when 15 people are testing. It’s not fine when 200 people are sending emails to customers. “If it’s pulling the latest go-to-market messaging,” says Ferguson, “we need to make sure it’s actually pointing to the right place. Do we have a plan in place for ensuring that that is maintained?”

At Box, that means building out structured Hubs — for product marketing, sales messaging, customer stories, policies, and more — so agents always pull from a single source of truth. Without this, agents don’t scale. They drift.
 

4. Make governance and safety real, not theoretical 

Box’s own agents respect content permissions automatically; if you can’t see a document, the agent can’t see it on your behalf. But that’s not always true with third party tools. “It’s not just, ‘Do they have the right permissions for who should use the agent,’” Ferguson explains, “but also, ‘Can they get access to information they shouldn’t have otherwise?’”

Rollout is where those rules get nailed down.

Only after all of this is in place does Box flip the switch and give the full audience access.

That moment, when the Intelligent Prospecting Agent appears in every eligible rep’s Box Box AI menu, is the start of Rollout.

But it’s not the finish line.
 

Turning an agent into “the way we work” 

One uncomfortable truth about AI in the enterprise is that access doesn’t necessarily equal impact. Ferguson is blunt about this. “Rollout is just getting the agent published to all the users. You should be thinking, ‘What’s the plan for making sure they use it?’”

That’s what Scaled Adoption is for. It’s the moment when functional leaders decide how the agent redefines a given workflow. For Sales, that might mean every meeting starts with the IPA pulling together account context. For Marketing it might mean every draft goes through an editing agent before publication. For Engineering it might mean documentation is no longer written by hand, because Box’s DocBot does it automatically.

 

If you don’t redesign the workflow, the agent becomes a side tool. And side tools don’t scale.
 

Robert Ferguson, Box’s Head of Corporate Strategy and Chief of Staff to the CEO

The key question, Ferguson explains, is where workflows change. ““This is the moment when leaders say, ‘Okay, the agent worked really well in Pilot. But are there activities the team is doing today that they don’t need to do anymore because this agent replaces them? Can we just slot the agent into people’s existing workflow? Or do we actually need to rethink how they do work?” 

If you don’t redesign the workflow, the agent becomes a side tool. And side tools don’t scale.
 

The two-person model that makes it work

Every scaled agent has two owners:

• The functional leader, who sets expectations.
• The AI manager, who makes it usable.

Ferguson explains the split. “The functional leader is setting the expectations for how often this agent should be used, who should be using it,” he says. “The AI manager is on the hook for developing the communication, the training, the enablement around that.”

This pairing is what turns software into habit. After all, scaled adoption isn’t just about telling people an agent exists; it’s about teaching them what it can and can’t do.

For the Intelligent Prospecting Agent, that means being explicit about what it can and cannot do today, and how its capabilities will expand over time. “Right now it doesn’t have access to live web data,” Ferguson notes. “It’s connected to our CRM. In the future, it will have access to live web data and additional systems. Just making sure that users know what it can and can’t do today is an important part of that.”

Without this level of understanding, users could either come to mistrust the new agent — or trust it too much.

Measuring success on a moving target

Tracking usage as AI agents evolve is no easy task. Box looks at indirect signals first: are people saying they use it? Are they working faster? “Right now it’s a lot of self-reporting,” says Ferguson. 

But ultimately, it’s hard numbers that matter. Are employees more productive? Saving money? Delivering more revenue for the same headcount? “We need to be rigorous about defining real success,” says Ferguson. “If we predicted at the start that this agent was going to save this many hours on certain tasks, do we actually see that show up in terms of how people are spending their time?”

For the IPA, that might mean more meetings booked, higher response rates, or more pipeline created — powered by better, faster outreach.

That’s what scaled adoption looks like when it’s working.
 

What you can learn from our approach

Want to take your own AI agents beyond experiments? Ferguson has a few suggestions: 

1. Don’t rush rollout.  
   Make sure reliability, accuracy, and user value are real before you scale.
    
2. Treat knowledge like infrastructure.  
   If your content is messy, your agents will be too.
    
3. Design for adoption, not access.  
   Redesign workflows. Set expectations. Make the agent the default.
    
4. Pair leadership with enablement.  
   Someone must demand usage. Someone else must make it easy.
    
5. Measure what matters — even imperfectly.  
   Start with self-reporting. End with business outcomes.

The Intelligent Prospecting Agent is still evolving. Soon it will pull from live web data, from richer intent signals, from more systems across Box. But it’s already passed the hardest test.

And that’s the real finish line of any AI initiative: not a demo, not a dashboard — but a new way of working that sticks.