What data do I need before I can build predictive quality models?

At minimum, you need labeled outcomes, traceable inputs, and enough historical context to connect the two. If you cannot reliably answer what happened, to which unit or lot, under which conditions, and what the quality result was, you are not ready for a production predictive quality model.

The practical data foundation usually includes:

• Quality outcome data: pass/fail, defect codes, nonconformance records, rework, scrap, concession or deviation status, inspection results, test measurements, and severity where applicable.
• Product and genealogy data: part number, revision, serial or lot, work order, route step, assembly relationships, supplier lot, and material genealogy.
• Process execution data: timestamps, operation sequence, machine or line, recipe or program version, parameter setpoints and actuals, cycle times, alarms, holds, and process step completion status.
• Equipment and tooling context: asset ID, tooling ID, calibration status, maintenance events, changeovers, downtime events, and known equipment state transitions.
• Measurement system context: gage ID, method, sampling plan, inspection program version, and evidence that the measurement system is stable enough to support modeling.
• Material and supplier context: supplier, heat or batch, incoming inspection results, certificate linkage where used, storage conditions if relevant, and substitutions or shortage-driven changes.
• People and shift context: operator or crew, certification or training status if tracked, shift, handoff points, and manual override or exception events.
• Engineering and change context: revision changes, approved process changes, ECO or ECN linkage, temporary instructions, and effective dates.

Just as important as the fields themselves are a few non-negotiable data qualities:

• Time alignment: events need consistent timestamps and known sequence. A model built on misordered events will often look accurate in testing and fail in live use.
• Stable identifiers: the same unit, lot, operation, machine, tool, and defect should be represented consistently across systems.
• Enough negative examples: if defects are rare, you may need a long time horizon, aggregation strategies, or narrower use cases. Very low defect rates are common in regulated manufacturing and can make model training difficult.
• Defined labels: if one plant codes a defect as scrap and another codes the same outcome as rework or use-as-is, the model will learn noise.
• Change history: when routes, specs, tolerances, and inspection methods change, the model needs that context or retraining discipline.

What is usually missing

In brownfield environments, the limiting factor is rarely raw volume. It is usually linkage and trust. Common gaps include:

• Inspection data that is disconnected from machine conditions or process parameters
• NCR and CAPA records that are too unstructured or too delayed to serve as reliable labels
• MES, ERP, QMS, historian, SCADA, and test systems using different identifiers for the same unit or lot
• Manual data entry with inconsistent defect coding
• Missing effective dates for revision or routing changes
• Short data retention windows for high-frequency equipment data
• Measurement variation that is larger than the process signals you are trying to detect

If those issues exist, adding more model complexity will not fix them.

How much history is enough

There is no universal minimum. It depends on process stability, defect frequency, product mix, and whether you are predicting a narrow defect mode or a broad quality outcome. A stable high-volume process may support a useful model with months of consistent data. A high-mix, low-volume environment may require much longer history, stronger engineering features, or a more constrained use case such as predicting reinspection risk at a specific operation.

You should expect better results when the use case is narrow, the failure mode is well defined, and the data lineage is clear. Broad promises like predicting all quality issues across the plant are usually not credible without very mature data foundations.

What to validate before deployment

Before using a model operationally, confirm that:

• The target label matches a real business decision and can be acted on without creating uncontrolled process changes
• The input data is available in time for the decision point, not only after the fact
• The model can be traced to source records and versioned under change control
• False positives and false negatives are understood in operational terms
• Users know what action is allowed when the model flags risk
• Retraining, monitoring, and rollback are defined

In regulated settings, this matters as much as model accuracy. A technically good model can still fail if it cannot be validated, explained to stakeholders, or governed through revisions and process changes.

Do you need a data lake first?

No. You need accessible, governed, and linked data more than a specific platform. Some teams start with a focused pipeline from MES, QMS, inspection, and historian data for one process. That is often more realistic than waiting for an enterprise-wide architecture program to finish. But if your core systems cannot exchange stable identifiers or event times, the integration work comes first.

Full system replacement is usually not the right prerequisite. In long lifecycle, regulated environments, replacing MES, ERP, PLM, QMS, and shop-floor data sources just to enable predictive quality often fails because of qualification burden, validation cost, downtime risk, and integration complexity. A staged coexistence approach is usually safer: improve traceability and data mappings around the highest-value use case, then expand.

So the short answer is: you need outcome labels, unit or lot genealogy, process and equipment context, measurement integrity, and disciplined change history. If any of those are weak, address that first. Predictive quality models are usually limited more by data readiness and operational governance than by algorithm choice.