Run data quality checks in four areas before publishing KPIs: metric definition, source data integrity, transformation logic, and operational governance. The right checklist depends on how many systems feed the KPI, how stable your master data is, and whether the metric will be used for management action, customer reporting, or quality evidence.
At a minimum, do not publish a KPI until you can answer three questions clearly: what exactly is being measured, which system or systems are authoritative, and how exceptions are handled. If those answers are not controlled, the KPI may still be useful for internal exploration, but it is not ready to be treated as a reliable operating signal.
Definition and scope check. Confirm the KPI formula, inclusion and exclusion rules, time window, aggregation level, and population being measured. Many disputes are definition problems, not data problems.
Source system authority check. Identify the system of record for each input field. If ERP, MES, PLM, QMS, historian, and manual logs all contribute, document which source wins when values conflict.
Completeness check. Measure missing records, null fields, missing shifts, missing work orders, missing machine states, and delayed transactions. A KPI can look stable while silently excluding part of the operation.
Timeliness and latency check. Verify data arrival times, refresh frequency, and cutoff logic. Publishing a daily KPI from sources that close at different times can create false variance.
Uniqueness and duplicate check. Detect duplicate events, repeated uploads, replayed interface messages, and duplicate production confirmations. This is common in retry-heavy integrations.
Validity and range check. Look for impossible or out-of-bounds values such as negative cycle times, future timestamps, scrap quantities above production quantities, or utilization above 100 percent unless the definition explicitly allows overlapping capacity logic.
Consistency check. Confirm that units of measure, asset names, shift calendars, reason codes, product identifiers, and status values are normalized across sources. Mixed coding schemes are a common brownfield failure mode.
Referential integrity check. Ensure records link correctly to work orders, operations, part numbers, resources, lots, serials, and personnel where applicable. Orphan records can distort both numerator and denominator.
Reconciliation check. Compare KPI inputs and outputs against trusted reports from source systems. For example, production counts should reconcile within an agreed tolerance to MES or ERP postings, and quality counts should reconcile to QMS or NCR records.
Transformation logic check. Test joins, filters, rollups, timezone handling, calendar boundaries, and business rules in the KPI pipeline. Most KPI defects are introduced during mapping and transformation, not at the dashboard layer.
Master data alignment check. Validate work center hierarchies, part master, routing versions, BOM relationships, customer or program mappings, and reason code dictionaries. If master data is unstable, trend analysis may be misleading even when transactions are accurate.
Exception handling check. Define how rework, split lots, partial completions, late entries, backflushing, reversals, and corrected quality events affect the KPI. If exception logic is undocumented, the metric will not hold up under scrutiny.
Historical stability check. Recalculate prior periods and see whether the KPI changes materially after late transactions arrive. If history is unstable, label the KPI as provisional until the close window is complete.
Auditability check. Confirm that calculation version, source extract time, lineage, and approval status are recorded. In regulated operations, traceability of the metric logic matters as much as the displayed number.
If your KPIs depend on multiple legacy systems, focus heavily on mapping quality, transaction timing, and code harmonization. Mixed MES, ERP, PLM, QMS, spreadsheets, and manual logs often produce KPI disagreements because the systems were not designed around a shared canonical model.
Typical failure modes include:
the same event recorded in two systems with different timestamps
different definitions of completed production, scrap, downtime, or release status
manual corrections made in one system but not propagated to others
routing or resource changes that break historical comparisons
interface retries that create duplicate records
work performed offline and entered later in batch form
That is why KPI publication should usually sit behind controlled data validation and change control, not only dashboard development. Full replacement of legacy stacks is often proposed as the fix, but in regulated, long-lifecycle environments that frequently fails due to qualification burden, validation cost, downtime risk, integration complexity, and the need to preserve traceability across existing processes. In practice, most plants need governed coexistence, not wholesale replacement.
A practical approach is to assign release criteria to each KPI. For example:
documented formula and owner
named systems of record
data completeness above a defined threshold
reconciliation variance within an agreed tolerance
known exceptions documented
calculation logic version-controlled and approved
user-facing label for provisional versus final values
The thresholds are site-specific. A near-real-time operational dashboard may tolerate more latency or correction than a KPI used for formal quality review or customer-facing performance commitments.
Do not assume a KPI is reliable because the source system is validated, because the dashboard looks consistent, or because the number matches expectations. A validated source application does not automatically validate the extraction, mapping, transformation, aggregation, and presentation layers around it.
Also do not assume one-time data cleansing is enough. KPI quality degrades when master data changes, new equipment is added, reason codes evolve, integrations are modified, or operators adopt workarounds under schedule pressure. Ongoing monitoring is part of KPI governance.
Before publishing KPIs, run checks for definition control, completeness, timeliness, duplicates, validity, consistency, referential integrity, reconciliation, exception logic, and auditability. If you cannot trace the number back to governed logic and trusted source data, publish it as provisional at most, or do not publish it.
Whether you're managing 1 site or 100, Connect 981 adapts to your environment and scales with your needs—without the complexity of traditional systems.
Whether you're managing 1 site or 100, C-981 adapts to your environment and scales with your needs—without the complexity of traditional systems.
