AI reporting: the executive committee must be able to trace the figure back to the case files

A fall in processing time may be real without being enough to justify expanding an AI use case. It depends on which cases were counted, how much human rework was needed and where the measurement ends. To make a decision, the executive committee must be able to trace the reported result back to its calculation, then to the cases behind it.

Two publications highlight distinct limitations

In a survey of 685 CIOs published on 24 September 2026, only 21% say they have complete, near real-time visibility of AI costs attributed by unit, team or use case. Conducted by Harris Poll for Dataiku, this self-reported survey is not an audit of the systems concerned. It addresses visibility of costs, not verification of gains. Dataiku and Harris Poll, 2026.

A preprint by the AI Risk Observatory dated 1 October 2026 examines the annual reports of UK listed companies. For 2025, 41.2% of the reports studied mention an AI risk, but only 4.3% contain a description judged substantive under its criteria. The analysis concerns published disclosures examined using language models; it does not establish that internal controls are absent. AI Risk Observatory, 2026.

Neither publication directly measures whether regulated European companies can trace an internal metric back to the underlying cases. The requirement proposed here is operational: when a figure informs a decision, its constituent elements must be available for examination.

Define the figure and find the cases counted

Consider a hypothetical example: an insurer uses AI to prepare responses to complaints. Its reporting shows a fall in processing time. Does it measure the drafting of the first response or the full process through to an approved response being sent? Are additional research, corrections and reopened cases included in the calculation? Has the proportion of complex complaints changed?

The metric’s definition must specify the period, the comparison baseline, exclusions and the population of cases included. If the result is based on a sample, the selection method must be accessible. The denominator matters as much as the percentage: a gain calculated only from completed cases may conceal abandoned cases or manual transfers.

| What the executive committee sees | What must be available for examination | | --- | --- | | Processing time | Timestamps, included cases, rework and exclusion rules | | Cost per case | Allocated expenditure, human work and allocation method | | Response quality | Review criteria, sample examined and corrections | | Reported incidents | Recorded events, severity, handling and closure status |

An observed change does not, by itself, demonstrate an effect of AI. If the team, business rules or mix of cases has changed, the reporting must show it. Without a sufficiently sound comparison, the gain must be presented as an observed change, without attributing it entirely to the system.

Reconstruct the processing that actually took place

In the insurer example, examining a case must allow authorised people to find the documents used, the AI’s proposed response, the case handler’s corrections and the response sent, together with the relevant dates and versions. They must also know which configuration, instructions and documentary sources were available when the case was processed. Running the request again today would not document the output produced yesterday.

The ECB guide published in May 2024 sets out common definitions, explicit responsibilities and data traceability through to the relevant metrics for bank risk reporting. Its scope is banking: it does not create a general obligation for all AI reporting. We propose applying its operational principle to metrics used to commit a budget or expand a use case. ECB, 2024, sections 3.3 and 3.4.

The executive committee can receive an aggregated summary. Access to supporting records remains controlled, and data retention must be defined according to the purpose of the review and applicable constraints.

Share responsibility for the reported gain

The IT department can document model calls, technical costs and version changes. The business owner must validate the scope and operational result; finance, the valuation method; and the technical team, the traceability of processing. The risk function or internal control must be able to examine exceptions and challenge the criteria used.

In our hypothetical example, time saved on drafting first represents capacity that may become available. It becomes a cost saving only if expenditure actually changes, for example on overtime or an external provider. Converting minutes into euros without an observed change remains an estimate. The calculation must also include supervision, corrections and running costs, so that a local gain is not presented as a gain for the entire process.

Finally, each result should state its coverage: what proportion of cases has the records needed for review? If coverage is incomplete, that qualification must appear alongside the figure.

When deciding whether to expand a use case, the executive committee can ask to see the population and calculation behind a metric, then examine an ordinary case and an exception. If a supporting record or responsibility is missing, that gap must be resolved before the decision that depends on the metric. Authorisation to expand the use case can then specify the evidence on which it rests and the limits accepted.