If you’ve read my previous articles, you probably know that I consider uncertainty evaluation the gold standard for instrument capability studies. You will also know that conventional gage studies can give misleading results. The VDA-5 standard is a big step forward for the evaluation of industrial measurements. It combines the practicality of gage studies with a rigorous uncertainty evaluation approach. This is definitely heading in the right direction, although the methods it uses can still miss some important sources of uncertainty, as I will show towards the end of this article.
VDA-5 is published by the German Association of the Automotive Industry, the VDA which stands for the German ‘Verband der Automobilindustrie.’ The VDA represents automotive manufacturers including BMW, Volkswagen and Daimler, as well as many supply chain companies. One of the functions of the VDA is to create industry standards and it is responsible for the German automotive industry’s quality management system (QMS). VDA-5 is primarily intended for geometrical or dimensional measurements although many of the principles can be applied to other types of measurement.
VDA-5 Is a Hybrid of Uncertainty and MSA
VDA-5 is a hybrid methodology, bringing together an uncertainty-based approach—which involves uncertainty budgets and conformance limits—with the widely used Measurement Systems Analysis (MSA) procedures such as gage R&R studies. In essence, this simply means that an uncertainty budget should be used which considers all influences on the measurement including systematic effects with a gage R&R study used to quantify the random uncertainty within this budget. Once the uncertainty of a measurement has been evaluated in this way, then conformance limits can be calculated by reducing the specification limits by the uncertainty. This gives a range of values for the measurement result, within which we can have confidence that the part is conforming to the specification. Overall, the approach may be considered a scientific way of proving, at a given confidence level, conformance with a product specification. VDA-5 also shows how specification limits can be increased by the uncertainty to prove non-conformance.
Other important MSA methods, not considered by conventional uncertainty evaluation literature, deal with attribute gaging. These are tests which give a binary pass/fail result rather than a variable dimension. Examples are go/no-go plug gauges and visual inspection processes. The VDA-5 also brings these methods within an uncertainty framework.
The uncertainty aspects are based on the Guide to the Expression of Uncertainty in Measurement (GUM) and ISO 14253: Inspection by measurement of workpieces and measuring equipment — Part 1: Decision rules for proving conformance or non-conformance with specifications. The MSA procedures are taken from the Automotive Industry Action Group (AIAG) MSA Manual. The AIAG was founded in 1982 by North America’s largest automotive manufacturers (Ford, General Motors and Chrysler) in order to develop a quality improvement framework. It has grown to include over 800 companies including Toyota, Honda and Nissan. The MSA manual was first published in 1990 and its methods have become the standard, particularly for gage studies, with commonly used industrial statistics software such as Minitab referencing it. Within VDA-5, methods to determine whether an instrument is capable are taken from ISO/WD 22514-7.
Errors and Uncertainty
The discussion of errors with respect to uncertainty is a contentious issue. It must always be remembered that an error is a distinctly different quantity to uncertainty. An error is an unknowable quantity. We can never know the true value of anything; all we know is the result of a measurement. An error is the difference between the result of the measurement and the true value. Since the true value is unknowable, the error is also unknowable. The uncertainty of a measurement gives us an idea of the possible range of values within which we can have confidence that the true value lies. The uncertainty of a measurement is therefore mathematically equivalent to the uncertainty of the measurement’s error, but it is not the same as the error itself.
The GUM takes great care to avoid confusion by avoiding the use of errors in its consideration of uncertainty, instead talking about influence quantities and effects. It does, however, note that its definition of uncertainty of measurement "...is not inconsistent with other concepts of uncertainty of measurement such as: -a measure of the possible error in the estimated value of the measure and... an estimate characterizing the range of values within which the true value of a measure and lies... Although these two traditional concepts are valid as ideals, they focus on unknowable, quantities... (error and true value)... Nevertheless, whichever concept of uncertainty is adopted, an uncertainty component is always evaluated using the same data and related information." (GUM 2.2.4)
Uncertainty Budgets in VDA-5
I’ve explained in detail how to calculate an uncertainty budget and deal with more complex sensitivity coefficients in my previous articles. In summary, the sources of uncertainty in the measurement are listed in a table, as shown in Figure 1. For each source a value is obtained which may be the standard deviation for a normal distribution or the half width for a uniform distribution. This value may be determined experimentally (a Type-A evaluation) or by some other means (a Type-B evaluation). A divisor is used which gives the equivalent standard deviation. A sensitivity coefficient may also be required, this gives the ratio of a change in the uncertainty source to the resulting change in the measurement result. Dividing the values by the divisor and multiplying by the sensitivity coefficient gives the corresponding standard uncertainty. Taking the root sum squared value of these standard uncertainties gives the combined standard uncertainty. This is essentially the estimated standard deviation for the measurement result.
The GUM provides very generic advice for preparing uncertainty budgets with a few specific examples of calibrations. VDA-5 goes further in providing specific guidelines detailing which influences to include and how to obtain estimates for them. For example, the traceability of the reference standard must be demonstrated and its uncertainty included in the budget, normally by taking the value from its calibration certificate. Other suggested influences to consider include:
- Measuring system: resolution, setting to one or several test parts, linearity and repeatability
- Environment: temperature, lighting, vibrations, contamination and humidity
- Human /operator: different measuring forces, alignment (parallax, cosine), physical/psychological constitution, qualification, motivation and care
- Test part: geometrical deviations, material properties and lack of stability
- Measurement method
- Mounting
- Mathematical and statistical procedures: elimination of outliers or filtering
Gage R&R Studies in VDA-5
The VDA-5 recommends that random uncertainty is evaluated using a Gage R&R study to determine this uncertainty under reproducibility conditions. ANOVA is used to identify the variation attributable to different factors such as variation between parts, different operators and repeatability of the measurement instrument. The ANOVA method, as applied to Gage Studies, can be understood by creating a Gage R&R in Excel. The VDA-5 also recommends that D-optimum experimental designs are used to minimize the number of measurements required in a study.
Measurement Capability in VDA-5
A clear process is defined in the VDA-5 in order to assess whether a measurement instrument and process are capable of proving the conformance of a part with a specification. This starts by evaluating the instrument and then moves on to consider the process. It can be summarized as:
- Check instrument’s resolution is less than 5 percent of the parts tolerance
- Evaluate uncertainty of measurement system (instrument) and ensure it is capable
- Evaluate uncertainty of measurement process and ensure it is capable
- Ensure measurement process is stable
The capability ratios (Q) used in steps 2 and 3 are expressed as percentages and given by:
where U is the expanded uncertainty of the instrument or process and T is the total range of the tolerance for the part.
It is recommended that for the measurement instrument QMS does not exceed 15 percent and for the measurement process QMP does not exceed 30 percent. Using these recommended limits and rearranging the equation it is also possible to express the capability of a measuring system or process as the minimum tolerance as shown below.
Proving Conformance or Non-conformance
A measurement result, together with its uncertainty, gives a range of values within which we can have confidence that the true value lies. We can therefore apply this concept to determine whether we can have confidence that a product is conforming to a specification. Consider that we have measured the length of a part and got a result of 99.1mm with an expanded uncertainty (k=2) of 0.2mm. This is telling us that we can have a confidence of 95% that the true length of the part is between 98.9mm and 99.3mm. Now, suppose that the specification for the part is 99mm +/- 0.2mm. Can we be confident that the part conforms to this specification? Apparently not, since our measurement tells us that it may be as big as 99.3mm but the maximum size allowed by the specification is 99.2mm.
This logic is applied in practice by setting conformance limits. To do this, we reduce the size of the tolerance zone (specification limits) by the expanded uncertainty of the measurement. The resulting tighter limits are known as specification limits. This method is described by ISO 14253 and referenced in VDA-5. The ISO 14553 / VDA-5 method is, however, overly cautious. For example, when a 95% confidence level is specified, assuming a symmetric distribution, in practice conformance is proven with 97.5% confidence. This is because the expanded uncertainty, expanded by a Coverage Factor (k), gives an interval about the measurement result within which there can be confidence that the value of the measure and lies. In basic statistical terms, it is a two-tailed test. However, we can generally assume the measurement uncertainty is significantly smaller than the tolerance to which conformance is being proven. If this is not true, then the measurement cannot tell us anything meaningfully about conformance with the specification anyway. Therefore, again assuming a symmetrical distribution, only one tail is interacting with one of the specification limits, as shown in section a) of the diagram below. To prove conformance with a specification at an actual confidence of 95%, a z-score should be used instead of a coverage factor, as recommended by JCGM 106, and shown in part b) of the diagram below. For a normal distribution, the distance between the specification limits and the conformance limits should therefore be approximately 1.645 times the standard uncertainty. The resulting product is not an Expanded Uncertainty according to the VIM definition and does not have a specific term used to describe it.
Sources of Uncertainty Not Considered by VDA-5
In terms of the state-of-the art in uncertainty evaluation, the VDA-5 is already somewhat out of date in its exclusive use of uncertainty budgets. The first supplement to the GUM has pointed out that this method is an approximation which should be validated using numerical simulations. However, for most practical purposes, uncertainty budgets provide a good approximation and are perfectly acceptable.
There is, however, another often overlooked source of uncertainty. This comes down to the very definition of uncertainty of measurement. Many production and quality engineers have confidence that Gage R&R studies prove the capability of measurement processes. Metrologists, on the other hand, often assert that uncertainty evaluation, as defined by the GUM, is more reliable. When we consider the two methods a little more closely, it becomes clear that they are not actually providing quite the same information. Importantly, both of them can miss significant sources of uncertainty. In brief, Gage R&R studies do not include Type B uncertainties, systematic effects which may not be observed during a study but which we have some prior knowledge of. The VDA-5 neatly addresses this issue by combining Type B uncertainties, such as calibration uncertainty, with the Gage R&R using an uncertainty budget. On the other hand, uncertainty evaluations have not traditionally modelled the interaction of measurement uncertainty and process variation, which under certain circumstances can be very significant. For example, an uncertainty budget may include a Type B uncertainty of the coefficient of thermal expansion for the part. The contribution that this uncertainty makes to the combined uncertainty of measurement (its sensitivity coefficient) depends on the size of the correction made for thermal expansion. This will be different for each measurement. These interactions are not fully modelled by any current standard methods and are the subject of ongoing research. This is something I will cover in more detail in a future article.
In conclusion, the VDA-5 standard is a big step forward for the evaluation of industrial measurements. It combines the practicality of gage studies with a rigorous uncertainty evaluation approach. Although this approach can still miss some important sources of uncertainty, it is the best methodology currently available.