A practical example
Would AI help staff find the correct procedures?
An organisation wants staff to find current procedures more quickly. Here’s how I’d assess whether AI would help and what to do next.
Should it improve search first, test an AI assistant or stop the idea?
This is an invented example, including the organisation and figures.
My recommendation
I’d improve the guidance and search first.
Staff in this example search three guidance libraries for instructions on routine work. The organisation is considering an AI assistant to make those instructions easier to find.
One library has no named reviewer. An assistant might find its guidance quickly, but that would not tell staff whether the guidance is still correct. I’d check what is current and who keeps it up to date before adding AI.
In 20 timed searches, the median was nine minutes. I’d improve the labels and existing search, then repeat those tasks to see whether that helps.
What would change my mind?
If staff still struggle to find the right guidance, I’d consider a small AI test on the same tasks. I’d compare the answers and total time spent, including checking and correcting them. Any time saved would need to justify the cost and work of keeping the assistant running.
Before recommending that test, I’d need to know who would handle errors and support. The interviews didn’t establish that. The technology team would also need to check access in the organisation’s own systems. The supplier demonstration doesn’t answer that question.
How I’d record the ratings
- AI ideas and value
- EmergentEvidence confidence: Moderate
The task and expected result are clear. The searches and interviews show a problem, but do not compare AI with improved search.
The observed searches and staff accounts establish a defined problem and an initial measure. They do not establish the benefit of AI.
- Organisational readiness
- Ad hocEvidence confidence: Moderate
One team has informal checking and correction arrangements, but practice varies. That fits Ad hoc. No one yet owns AI answer checks or ongoing support.
The supervisor accounts describe some existing ownership. I judge that these local arrangements are a starting point, rather than a consistent process across the reviewed work.
- Technology readiness
- Ad hocEvidence confidence: Limited
The libraries and local access checks are known, but teams handle them separately. I rate this Ad hoc: the proposed AI connection, monitoring and support have no shared arrangement.
I infer the rating from the technology lead’s account and the library list. Confidence is limited because I have not tested local access or the proposed connection. The rating does not approve an AI trial.
Read the full example
The full Word example has the evidence behind my recommendation and the suggested test measures. It includes a six-month plan for the organisation and its providers.