This involves quality assurance, quality control and metrology. We use quality assurance to gain confidence that quality requirements will be fulfilled. Quality control is used to check that requirements have been fulfilled. This is a subtle difference and in practice the terms are sometimes used interchangeably. Metrology is the science of measurement. It is how we ensure that we can confidently compare the results of measurements made all over the world.
These principles can apply to products or a services, but I’m going to focus on manufacturing and how these three fundamental concepts relate to each other in that context. I have therefore avoided the details of specific methods and I don’t get into any of the maths. I’ll save that for a later article.
Origins of Measurement
The Egyptians used standards of measurement, with regular calibrations, to ensure stones would fit together in their great construction projects. But modern quality systems really began during the industrial revolution. Before then, mechanical goods were built by craftsmen who would fettle each part individually to fit into an assembly. This meant each machine, and every part in it, was unique. If a part needed to be replaced then a craftsman would need to fit a new part.
In the late 18th Century, French arms manufacturers started to make muskets with standard parts. This meant the army could carry spare parts and quickly exchange them for broken ones. These interchangeable parts were still fettled to fit into the assembly but instead of fitting each part to the individual gun it was fit to a master part.
A few years later, American gun makers started using this method but adapted it to suit their untrained workers. They filed gauges to fit to the master part, workers would set jigs and production machines using the gauges, and also use the gauges to check the parts. This enabled a row of machines, each carrying out a single operation with an unskilled operator, to produce accurate parts. The parts could then be quickly assembled into complex machines.
The foundation for modern manufacturing had thus been laid, over 100 years before Ford would apply these ideas to a moving production line.
Calibration, True Value and Measurement Error
A system of master parts, gauges and single-use machines worked when an entire product was produced in a single factory. Modern global supply chains need a different system.
Instead of having a physical master part, we have a drawing or a digital CAD model. Specified tolerances ensure the parts will fit together and perform as intended. Rather than every manufacturer coming to a single master part to set their gauges, they have their measurement instruments calibrated. The instruments are then used to set the production machines and to check the parts produced.
All quality depends on this calibration process.
The most important concept to understand is that all measurements have uncertainty. If I asked you to estimate the height of this text you might say, ‘it’s about 4 mm’. Using the word ‘about’ implies that there is some uncertainty in your estimate.
In fact, we can never know the exact true value of anything, all measurements are actually estimates, and have some uncertainty. The difference between a measurement result and the true value is the measurement error. Since we can’t know the true value, we also can’t know the error: these are unknowable quantities.
All we can quantify about the world around us is the results of measurements and these always have some uncertainty, even if this uncertainty is very small.You might have some level of confidence, say 95 percent, that it is true. If you were to increase the limits, to say give or take 2 mm, then your confidence would increase, perhaps to 99 percent. So the uncertainty gives some bounds within which we have a level of confidence that the true value lies.
OK, philosophy class over!
In a future post, I’ll expand of these ideas and how the uncertainty for a particular confidence level can be calculated.
Uncertainty and Quality
Once we have determined the uncertainty (or ‘accuracy’) of a measurement we can then apply this to decide whether a part conforms to a specified tolerance. For example, let’s say that a part is specified to be 100 mm +/- 1 mm. We measure it and get a result of 100.87 mm.
Is the part in specification?
The simple answer is: “We don’t know, maybe it is, but maybe there was an error in our measurement and actually the part is more than 101 mm. Maybe there was an even bigger error and the parts is actually less than 99 mm!”
Unless we know what the uncertainty of the measurement is, we have no idea how confident we can be that the part is within specification. Let’s suppose that the uncertainty of the measurement was given so that the measurement result is 100.87 mm +/- 0.1 mm at 95 percent confidence. Now we can say with better than 95 percent confidence that the part is within specification.
So understanding and quantifying the uncertainty of measurements is critical to maintaining quality.
Now, let’s consider calibration and the associated concept of traceability. This is a fundamental aspect of uncertainty. A calibration is a comparison with a reference, and the uncertainty of this comparison must always be included, for reasons explained below.
A traceable measurement is one which has an unbroken chain of calibrations going all the way back to the primary standard. In the case of length measurements, the primary standard is the definition of the meter; the distance travelled by light in a vacuum in 1/299,792,458 seconds, as realised by the International Bureau of Weights and Measures (BIPM) in Paris.
Since the 1930s, the inch has been defined as 25.4 mm and is therefore also traceable to the same meter standard. All measurements must be traceable back to the same standard to ensure that parts manufactured in different countries will fit together.
Uncertainty and Error
The uncertainty of measurements arises from different sources. Some of these will lead to a consistent error, or bias, in the result.
For example, the unknown error present when an instrument was calibrated will lead to a consistent error whenever it is used. This type of effect is known as a systematic uncertainty leading to a systematic error. Other sources will lead to errors which change randomly each time a measurement is made.
For example, turbulence in the air may cause small, randomly changing perturbations of laser measurements, mechanical play and alignment may cause randomly changing error in mechanical measurements. This type of effect is known as a random uncertainty leading to a random error.
It is conventional to divide random uncertainty into repeatability, the random uncertainty of results under the same conditions, and reproducibility, the random uncertainty under changed conditions.
Of course, the conditions can never be exactly the same or completely different so the distinction is somewhat vague. The types of conditions which might be changed are making the measurement at a different time, with a different operator, a different instrument, using a different calibration and in a different environment.
The GUM method involves first considering all of the influences which might affect the measurement result. A mathematical model must then be determined giving the measurement result as a function of these influence quantities. By considering the uncertainty in each input quantity and applying the ‘Law of Propagation of Uncertainty’ an estimate for the combined uncertainty of the measurement can be calculated.
The GUM approach is sometimes described as bottom-up, since it starts with a consideration of each individual influence. Each influence is normally listed in a table called an uncertainty budget which is used to calculate the combined uncertainty.
MSA involves performing Gage Studies in which repeated measurements are compared with a reference under different conditions to determine the bias, repeatability and sometimes reproducibility.
A Type-1 Gage Study is the quick check normally carried out to initially understand the variation in a gage. It involves a single operator measuring a single calibrated reference part 25 or more times and then considers the variation and bias in the results. This type of test is often called a repeatability study outside of MSA.
A Gage Repeatability and Reproducibility (Gage R&R) study is used to get a more detailed understanding of a measurement process. Typically 10 parts are each measured twice by at least three different operators. A statistical technique called ANOVA is then used to determine how much variation is caused by the instrument (‘gage’) and how much is caused by the operator. This considers the change of operator and subsequent changes in time and environment to be a full representation of reproducibility conditions.
MSA is sometimes referred to as top-down since it largely treats the measurement process as a black-box and experimentally determines systematic and random uncertainties. Two important concepts in MSA are accuracy, used as the equivalent of uncertainty; and precision, used as the equivalent of random uncertainty.
The advantage of uncertainty evaluation is that it is capable of considering all sources of uncertainty and, if done properly, gives the most accurate estimate of uncertainty. Problems with this approach include the fact that it requires a metrologist capable of producing the mathematical model and the risk of human error leading to significant influences being omitted or incorrectly estimated.
The GUM method is also only valid for an individual measurement which has been carried out with values known for any corrections applied. It is therefore difficult to correctly apply uncertainty evaluation to predict the uncertainty of an industrial measurement process.
MSA can be much more easily applied and is intended to provide a prediction for the accuracy of an industrial measurement process. The problem with this approach is that certain systematic effects are ignored and reproducibility conditions may not be fully represented, leading to an underestimate of uncertainty.
An example of the omission of systematic effects is that when determining the bias, a comparison is made with a reference which is treated as the true value—in reality the reference also has uncertainty, which should be included. This method relies on all reproducibility conditions being varied so that their effects can be seen in the variation of results when making repeated measurements. It is likely that the way in which these conditions are varied will not fully reflect the variation seen during the life of the actual measurement process.
Measurement and Quality Assurance
So far, I have focused on quality control, i.e. how measurements can prove that parts are conforming to specifications after they have been produced. Now, let’s briefly consider quality assurance, the way we ensure that the process produces good parts in the first place.
This aspect of quality is largely addressed by Statistical Process Control (SPC). A process may be evaluated by making several parts and measuring them to determine the variation and bias in the manufacturing process. Rather than giving these results directly, it is normal to divide the part tolerance by the process precision to give the Machine Tool Capability (CP) or by the process accuracy to give the Process Capability (CPK).
SPC is in many ways equivalent to MSA. It takes a top-down approach to understand random and systematic effects. However, instead of evaluating measurements, it is used to evaluate process outputs. It generally has the same advantages and disadvantages as MSA, and a bottom-up uncertainty evaluation approach can be used if these are a concern.
It may initially appear that there are fundamental differences between MSA and SPC due to the very different terminology within SPC. However, common cause variation (or chance cause variation in older literature) is the equivalent of precision; short-term variability is the equivalent of repeatability; long-term variability is the equivalent of reproducibility; and special cause variation (or assignable cause variation in older literature) is the equivalent of bias.
SPC also places a much greater emphasis on ensuring that a process is in ‘statistical control’. In broad terms, this means that effects are random and normally distributed with any significant systematic effects corrected for. This is a strong point of SPC and is sometimes overlooked in both uncertainty evaluation and MSA.
The main tool used in SPC to check for an ‘in-control’ process is the control chart. This gives a simple graphical view of a process where it can be easily spotted when a process is drifting or producing errors which cannot be explained by normal random variations. For example, if several points are all increasing or decreasing then this would indicate the process is drifting.
Stay tuned for future articles on these topics.
Dr. Jody Muelaner’s 20-year engineering career began in machine design, working on everything from medical devices to saw mills. Since 2007 he has been developing novel metrology at the University of Bath, working closely with leading aerospace companies. This research is currently focused on uncertainty modelling of production systems, bringing together elements of SPC, MSA and metrology with novel numerical methods. He also has an interest in bicycle design. Visit his website for more information.