Did the first rollout earn a second?

Imagine a company trying an assistant to handle missing or conflicting information during customer onboarding. Before it starts, identify the cases arriving, the people handling them and what counts as a completed onboarding. Choose comparable types of case and record the period you will measure. Keep exceptions, repeat contacts and quality failures in view alongside the time taken. This is an example for planning a trial, not a result from one we ran.

Tell hands-on work apart from waiting. A quicker first response may leave staff doing the same work later. Less handling time may free up capacity without reducing payroll. Ask finance which result matters and how it will be recognized. Keep forecasts apart from measured results, and note other process changes that might explain an improvement. Hours that might become available should not appear as cash already saved.

Nexus says its deployed agents have measurable business KPIs and that it changes the scope when a KPI does not improve.^1,2

Ask for a measurement plan for this company. Agree what a finished, approved case looks like, which records establish the starting position and who may challenge the calculation. Naming a KPI starts that discussion; it does not show that the result improved.^1,2

Keep a record for every onboarding case in the trial: identifier, case type, allowed inputs, proposed answer, reviewer’s decision and reasons for corrections. Note whether it finished, needed a person to handle an exception or was abandoned. Include repeated attempts and supplier intervention. The difficult cases belong in the result, even when they would not make a persuasive demonstration.

Glean describes events that can be sent to a security monitoring system, or SIEM. WRITER describes exportable user, agent and connector events. Relevance AI describes sending logs and execution traces to a destination the customer owns.^3,4,5

These records may help you reconstruct what happened. Someone who owns the business process must still decide whether the work was accepted. Match the technical event to the case before counting it. Keep only the information the review needs and restrict it to approved readers. A portfolio report can show combined results without copying every company record into the fund’s files.^3,4,5

Agree what a pass looks like before reviewing results. For this example, answers must point to the permitted records, restricted information must stay inaccessible and unresolved conflicts must reach the named owner. Let the process owner set acceptable quality and effort levels. A generic accuracy percentage can hide the cost of a wrong action; choose a rule that reflects this company’s work.

Have the finance owner, process owner and delivery owner review the same work before and after launch. Include software charges, usage, setup, access administration, review, support and changes. Leave missing evidence visible instead of replacing it with an assumed hourly saving. If the mix of cases changed, compare like cases or explain why you cannot yet attribute the difference to the deployment.

Use a decision page with the measured result, review period, quality of the starting comparison, running cost, unresolved risks and next step. The choices are to continue, change the scope, gather more evidence or stop this use case. Base the choice on work the company accepted, not on how many people opened the tool. Even a stopped trial can clarify which work the next evaluation should exclude.

Relevance AI describes a deployment team followed by customer training. Microsoft describes people creating and testing agents before publication.^6,7

For either approach, check whether the named company owner can maintain the workflow. If the result depends on unrecorded help from the supplier or a builder who will not be available, the proposed ongoing arrangement has not been demonstrated. Include that dependence in the decision, even if the output looks good.^6,7

Divide what you have built into parts that can travel and parts that need local decisions. The list of case types, approval checklist or test design may be reusable. Credentials, company records, exception owners and contractual permissions need a fresh look. Ask the next company to find the mismatches before the implementation is priced. Similar work can differ at the moment a record changes or a person becomes responsible.

Give that company the approved workflow description, evidence and limits from the first rollout, reusable files, local requirements, needed changes, new test cases, proposed cost and named owners. If it uses another CRM, try its actual records and permissions. The differences need an assessment before the work can fairly be called a copy.

Take a conditional expansion decision to the portfolio committee. Say what the first company showed, what is still a forecast and what the next company must check before launch. Keep the original case records available for challenge. The fund gets a consistent way to decide; the company keeps the authority to reject a rollout that does not fit its work.