Scale an AI pilot only when actual results pass the quality, safety and operating-cost criteria agreed before testing. Record failures and human review time on the scorecard below. Use go, revise or stop as explicit decisions. A successful demonstration does not replace acceptance evidence.
The roadmap below is a planning tool. If the workflow needs connections between existing systems, review Toimi's business automation and AI integration services.
What should the audit produce?
Start with the work. Ask an employee to show a recent task from beginning to end, including the corrections nobody mentions in the process diagram.
Record volume, handling time, error cost and exceptions. Identify where data lives and who may access it. Name the process owner.
The audit should end with a candidate workflow and a baseline. A list of possible AI features is insufficient for a funding decision. Sometimes a deterministic automation will solve the problem with fewer moving parts. Keep that option in scope.
How do you choose the first pilot?
Select work with a measurable result and a safe fallback. A draft that an employee reviews may be easier to evaluate than an autonomous action that changes a customer account.
Check that representative historical examples exist. Include difficult cases. A pilot that sees only clean inputs can conceal the cost of human intervention.
Separate the person configuring the pilot from the person accepting it. Give the reviewer test cases the builder has not already used for tuning.
What belongs in the pilot charter?
Use this template before asking for estimates:
| Field | Decision to write down |
|---|---|
| Workflow | Exact starting event and final output |
| Baseline | Current volume, time, errors and cost |
| Allowed data | Systems and records the pilot may use |
| Acceptance | Quality, handling time and safety thresholds |
| Spending ceiling | Approved maximum and included items |
| Human review | Who checks outputs and exceptions |
| Stop rule | Failure that pauses the pilot |
| Fallback | How employees resume the old process |
| Ownership | Who approves changes and accepts results |
Set thresholds from the business risk. There is no universal accuracy percentage that makes every workflow safe.
How do you calculate the pilot's value?
Count net time saved. Subtract review and correction time.
Monthly labor value = eligible tasks × net minutes saved per task / 60 × loaded hourly cost.
Capacity has to be used. If nobody can reassign the saved time, call it capacity rather than cash savings. Include additional revenue or avoided errors only when you can support the estimate.
What costs belong in the budget?
Ask vendors to separate discovery, implementation and operation. Compare the same scope.
Pilot cost includes data preparation, connectors, evaluation, employee training and project coordination. Operating cost includes model usage, infrastructure, monitoring, human review and changes caused by vendor updates.
Add internal time. Someone must approve access, explain exceptions and accept releases. An external estimate will not automatically include those hours.
Request a budget ceiling and a rule for exceeding it. Put optional work on separate lines. This lets the owner approve a smaller experiment without silently accepting a larger commitment.
What should an ongoing retainer buy?
A retainer should name recurring responsibilities: monitoring, incident response, evaluation after changes and maintenance of integrations. Specify included capacity and extra-work approval.
Define who maintains the test set. Changes to documents, models or business rules can require a fresh evaluation. Keep version records and rollback instructions.
Separate response time from resolution time. Ask for the support window, escalation path and severity definitions. A monthly fee without those terms leaves the operating obligation unclear.
When should the business stop or delay?
Pause when the workflow has no owner, the data cannot be used safely or the pilot fails its acceptance tests. Delay expansion if review cost removes the expected gain.
The useful next step may be cleaner data or a simpler automation. Keep the audit and failed tests. They make the next funding decision better informed.
What baseline should a pilot beat?
Record the current workflow on comparable tasks. Include completion quality, employee handling time, corrections and cost. Define what counts as an eligible task. Keep exclusions visible. A faster answer that needs extra correction may remove the gain.
Who owns the held-out test set?
Name an acceptance owner who controls the final test cases. Keep those cases separate from tuning. Record the data version, model configuration and evaluation date. If the builder changes the system after studying the final failures, arrange a fresh independent check.
Which quality and safety failures stop rollout?
Test an unavailable source system and an input containing misleading instructions. Agree blocking failures before testing. Include unauthorized access, unsupported actions and failure of the fallback where relevant to the workflow. Specify the expected response to missing or conflicting information. Do not let a high average score cancel a mandatory stop condition.
NIST's AI Risk Management Framework defines four core functions: Govern, Map, Measure and Manage. See the AI RMF Core for their scope. Use that structure to keep an owner, documented context, evaluations and responses to risk in the roadmap. It does not certify your pilot. Reference checked October 15, 2026.
How do you include human review and operating cost?
Count review, correction and exception handling. Include model usage, infrastructure and support. Distinguish released capacity from cash savings. Record the workload used to estimate the recurring cost. Check what changes at a higher volume.
How do you record actual results against agreed criteria?
Copy this scorecard. Enter business-specific thresholds before the run. Then add actual results, including failures. Keep the evidence accessible to the acceptance owner.
| Criterion | Agreed threshold or stop condition | Actual result | Evidence and owner | Decision |
|---|---|---|---|---|
| Task completion and quality | … | … | Test set and version: … | … |
| Difficult cases and missing data | … | … | Failure examples: … | … |
| Permissions and prohibited actions | … | … | Allowed and denied tests: … | … |
| Net handling time | … | … | Review and correction log: … | … |
| Operating cost at tested volume | … | … | Usage, people and support: … | … |
| Fallback and recovery | … | … | Recovery test: … | … |
| Operating ownership | … | … | Named owner and procedure: … | … |
Read each result with its evidence. Mark an untested row as untested. Do not convert it into a pass. Resolve conflicting evidence before signing.
Who signs go, revise or stop?
The process owner signs the business decision. The acceptance reviewer confirms the test result, and the operating owner accepts the support obligation. Record names, date and scope.
Go: mandatory criteria pass and the approved operating case holds at the tested scope. Specify the expansion boundary and next review.
Revise: a defined correction could close the gap within an approved budget. Name the change, owner and retest. Keep rollout on hold until acceptance.
Stop: a blocking failure remains or the operating case does not justify further investment. Preserve the findings and restore the agreed fallback.
Take the completed scorecard to the automation and AI integration discussion. Include failed and untested rows. They define the next decision.