A pilot can generate many messages, demonstrations and task completions without clarifying whether the offer is useful. For Thora Vietnam, measurement should connect an AI automation to a customer decision and to the effort required to deliver it. The scorecard below is a proposed starting point, not a report of company performance.
Choose the decision before the metric
Write the decision you expect to make after a pilot. For example: should the next version automate request routing, or should the team improve the intake form first? The question determines what evidence is needed.
A count of generated drafts cannot answer whether those drafts reduced work. Measure accepted outcomes and the review effort required to reach them.
Use a small balanced scorecard
For a proposed automation, record completed useful tasks, corrections per task, handling time including review, and unresolved exceptions. Add operating cost or support effort when they affect the offer. Keep the definition and denominator beside each metric.
An illustrative result such as three accepted drafts out of five attempts is easier to interpret than a percentage with no context. It is still too little evidence for a general performance claim.
Compare like with like
Use similar tasks and include their difficulty, input quality and reviewer. If the second batch contains easier examples, faster completion may have little to do with the automation. Keep unusual failures in the report instead of quietly excluding them.
Qualitative feedback belongs beside the numbers. A user may reject a faster workflow because its output is difficult to trust or hand over.
Use the scorecard to change the offer
The intended outcome is a specific adjustment: narrow supported inputs, redesign a review step, change the service scope, or stop a weak experiment. A metric becomes useful when it changes a commitment. Thora Vietnam should preserve the evidence and the decision together so the next pilot can test what changed.
Record the baseline before claiming an improvement
Before changing a workflow, Thora Vietnam should agree what a comparable case looks like and observe the current process. Include manual review, corrections and waiting time where relevant. After the pilot, compare cases with similar complexity; note differences in workload, staff familiarity and incomplete records. With a small sample, report individual observations and uncertainty instead of announcing a general performance result.
The following calculation is an illustration only: if an output takes 12 minutes today and the proposed flow takes 4 minutes plus 3 minutes of review and 2 minutes of correction, the net difference is 3 minutes per comparable case. It is not an achieved Thora Vietnam result.
Check delivery economics as well as the user outcome
A workflow can save a customer time while consuming too much implementation or support effort. Track the revenue agreed for the period alongside model/API usage, hosting, tools, review labor, rework and onboarding effort. Separate one-time setup from recurring delivery, and identify shared overhead rather than silently treating it as free.
For a proposed offer, list price and paid demand are different. Record whether a price was merely discussed, accepted in writing or actually paid. A pilot that depends on a temporary discount may still teach something useful, but it needs another test before becoming the standard offer.
Use a scorecard with decisions attached
Choose thresholds with the customer before the test, based on the actual risk and baseline. Leave them unset until agreed; an arbitrary percentage on a dashboard is not a business goal.
Measure | Baseline | Pilot observation | Evidence | Decision rule
Completed useful cases | | | |
Review and correction minutes | | | |
Exceptions and harmful errors | | | |
Direct cost and delivery labor | | | |
Repeat use and buyer commitment | | | |Use repeat behavior to guide future investment
For the next stage, check whether users return without repeated prompting and whether the buyer wants the service to continue at a viable scope and price. Compare groups that started at similar times instead of adding all usage into a flattering cumulative total. Investigate departures and non-use as carefully as positive feedback.
Thora Vietnam should expand only when useful outcomes, repeat behavior and delivery economics point in the same direction. When they disagree, identify the next uncertainty to test rather than averaging them into a single success score.
Further reading
Background for the method; the Thora Vietnam workflow above is a proposed application.
Eric Ries — Why vanity metrics are dangerous
