How can we make ISO 22400 KPI calculations auditable?

Making ISO 22400 KPI calculations auditable is less about the standard itself and more about how you define, implement, and govern the KPI logic in your systems. In regulated, brownfield plants, auditable KPIs require unambiguous definitions, reliable data capture, controlled calculation logic, and reproducible results backed by evidence.

1. Start with precise, written KPI definitions

ISO 22400 describes concepts and reference calculations, but each plant still makes choices. To be auditable, you should maintain a KPI definition sheet for each KPI that includes at least:

• Name and identifier (e.g., ISO22400_OEE_V1, ISO22400_Availability_V2).
• Scope: line, machine group, shift, product family, plant; plus time horizon (shift, day, week).
• Exact formula, including units and references to the specific ISO 22400 clause or figure where applicable.
• Time base definitions: what counts as planned time, operating time, planned/unplanned downtime, and which status codes map to each bucket.
• Included and excluded events: e.g., warmup, maintenance, testing, changeovers, engineering trials, rework.
• Data fields used: system, table, tag, or signal names (from MES, historian, PLC, ERP, QMS, etc.).
• Aggregation rules: how you roll up across shifts, machines, or orders (e.g., weighted by planned time or output quantity).
• Known limitations and assumptions: for example, how you handle missing machine states, partial cycles, or backfilled production counts.

Auditors will challenge anything that is ambiguous or inconsistently applied across lines or plants. Written definitions are the baseline for repeatability.

2. Make data lineage from KPI to raw signals traceable

To be auditable, anyone should be able to start from a reported KPI value and trace back to:

• The underlying time series of machine states and production counts.
• The work orders, part numbers, and shift calendars involved.
• The exact calculation logic and version that produced the result.

Practical steps:

• Retain time-stamped raw data from MES, SCADA/PLC, historian, and ERP at a resolution that supports reconstruction of events (for many KPIs, 1-second to 1-minute resolution is typical).
• Maintain a data dictionary that maps source tags and fields (e.g., PLC bits, MES state codes, ERP order status) to KPI categories such as operating time, minor stop, changeover, scrap, or rework.
• Record data transformations (e.g., state code reclassification, time bucket merging, filtering of obvious noise or outliers) with versioned logic.
• Keep referential links between production events, work orders, batches, and KPIs (e.g., a KPI instance references specific order IDs and date ranges).

If your KPI platform cannot show how a value was derived from raw data, auditors will treat it as a dashboard number rather than as reliable evidence.

3. Put calculation logic under change control

Many plants implement ISO 22400 KPIs in multiple places: historian scripts, MES reports, BI tools, or custom SQL. This is a common source of non-auditable discrepancies.

To keep calculations auditable:

• Centralize KPI logic as much as possible in a single, validated layer (e.g., MES or an analytics engine) and treat that as the system of record.
• Apply formal change control to KPI definitions and logic: documented change requests, impact assessment, testing, approvals, and effective dates.
• Version all calculation code and configurations (SQL, ETL flows, scripts, BI measures) in a repository where you can reconstruct the exact logic used on any historical date.
• Document deviations from ISO 22400: if you implement a plant-specific variant of OEE or availability, label and document it as such rather than calling it “ISO 22400” without qualification.

In regulated environments, ad hoc dashboard logic without version control is a major audit risk, even if the formulas are mathematically correct.

4. Validate the full calculation pipeline

In aerospace, pharma, and other regulated sectors, KPI numbers are often used to support capacity decisions, improvement programs, and sometimes compliance evidence. That makes the calculation pipeline itself subject to validation expectations.

Consider a basic validation approach:

• Define intended use: for example, “shift-level ISO 22400 availability and OEE for internal performance management, not used directly for product release decisions.”
• Perform installation and operational checks: confirm data flows from each source (MES, historian, ERP) are complete, time-synchronized, and secure.
• Develop test cases: use controlled historical periods where states and counts are known (e.g., a planned training day with a known stop pattern) and verify that your pipeline reproduces the expected KPI values.
• Document limitations: for example, “Cycle counts on Line 3 prior to date X are underreported during minor stops due to PLC configuration; KPI values before that date are not fully comparable.”

If your organization is subject to formal CSV or software validation expectations, your KPI tooling and data integration stack may fall in scope. Work with quality and IT to set appropriate validation depth.

5. Ensure consistent handling of time, shifts, and calendars

ISO 22400 KPIs such as availability, utilization, and OEE depend heavily on how you define planned time and schedule exceptions. These details are common audit failure points.

• Use controlled calendars for shifts, holidays, and site-specific events, ideally managed in a master system (MES, HR, or scheduling tool) and propagated downstream.
• Define standard rules for how you treat early starts, overtime, partial shifts, and overlap between shifts.
• Classify schedule exceptions explicitly: e.g., planned maintenance, trials, and engineering work that should be removed from planned production time.
• Synchronize time zones and clocks across OT and IT systems so that event sequences remain reconstructable.

Auditable KPIs require that two analysts using the same rules and data can independently reproduce the same results for a given period.

6. Design reports for drill-down and reproducibility

Auditability also depends on practical usability. Reports and dashboards should support tracing a number back to its components.

• Make drill-down supported: from monthly OEE to daily, shift-level, machine-level, and event-level views.
• Show KPI components: for example, for OEE, separately display availability, performance, and quality, with their numerators and denominators.
• Display applied filters and versions: time range, scope, excluded events, and KPI definition version.
• Allow export of underlying event data for sampled periods to support manual recalculation and evidence reviews.

If an auditor cannot inspect how a KPI changed when they adjust time filters or drill down into a particular machine or order, they will question its reliability.

7. Manage coexistence with legacy MES, historians, and BI tools

Most plants already calculate some form of OEE or ISO 22400-like KPIs in multiple systems. Full replacement is rarely realistic due to validation burden and downtime risk. Instead:

• Pick one system as the KPI system of record for ISO 22400-aligned metrics, then document that all official numbers come from there.
• Map and reconcile existing metrics in legacy tools to the new definitions; document differences (e.g., “Old_OEE includes planned maintenance as downtime; ISO22400_OEE excludes it.”).
• Phase out non-comparable KPIs or clearly label them as legacy indicators to avoid mixing them with ISO 22400 KPIs in formal reports.
• Implement interface tests to ensure data extracted from MES/ERP/historian into the KPI engine matches the source (record counts, sums, sample spot-checks).

Auditability suffers when multiple conflicting KPI values exist for the same period without a clear explanation. Reconciliation and labeling are essential during transition.

8. Capture governance, ownership, and training

Even with robust technical controls, ISO 22400 KPI calculations will not be auditable unless people understand and follow the rules.

• Assign ownership for each KPI (typically operations or industrial engineering) with clear responsibilities for definition, review, and continuous improvement.
• Set a review cadence where KPI definitions, data quality, and observed anomalies are periodically checked and updated under change control.
• Train key users (engineers, supervisors, analysts) on the definitions, typical pitfalls (e.g., double-counting, misclassified downtime), and how to respond to audit questions.
• Maintain an evidence pack for each key KPI: definition documents, sample calculations, validation records, and recent change logs.

9. What auditors will typically look for

While specific expectations vary by regulator and customer, auditors reviewing ISO 22400-style KPIs used in decision-making commonly test:

• Can you explain the formula and link it to ISO 22400 where applicable?
• Can you trace a reported value back to raw data, with consistent time stamps and event records?
• Is the calculation logic controlled and versioned, with documented changes and approvals?
• Are there documented data quality controls and known limitations?
• Can two people independently recalculate and match a sample KPI period using the same data and rules?

If you can provide clear answers and evidence for these points, your ISO 22400 KPI calculations will generally be considered auditable, even in complex, mixed-vendor environments.