"We found that FDA medical device plant inspection outcomes are highly predictive of future medical device recalls originating from the plant, but only when it is the first time that the FDA investigator has inspected that plant," said George Ball, assistant professor of operations and decision technologies at Indiana University’s Kelley School of Business and the study's lead author. "The newness factor really matters."
The study, "Do Plant Inspections Predict Future Quality? The Role of Investigator Experience," appears in the journal Manufacturing & Service Operations Management.
Ball and his colleagues found that plant inspections are most effective when conducted by investigators who were not already familiar with the manufacturing facility. Even inspecting a plant for the second time increased the risk of a future recall from that plant by 21 percent. That risk rises to 57 percent on the third visit. It should be noted that these findings were independent of the score the FDA investigator ultimately assigned to the plant.
"These increased recall risks may be symptomatic of increased familiarity between plant management and a repeat investigator, enabling the plant to relax its standards," said co-author Enno Siemsen, executive director of the Erdman Center for Operations and Technology Management at the University of Wisconsin.
The researchers tested the predictive relationship between the outcomes of 4,767 biannual FDA plant inspection outcomes and 2,863 medical device recalls original from 2,224 plants between 2000 and 2006. Compared to a low-quality inspection outcome, they found that a high-quality outcome from inspector who had never inspected a plant before predicted a 31 percent reduction in the risk of a recall from that plant.
Previous research has suggested that investigators may become more complacent as the result of repeated visit to the same facilities, but the focus was always on the number of violations found during inspections. According to Ball, that doesn’t provide an accurate assessment of whether a plant is getting better or worse, since it’s possible that fewer violations are due to improved conditions.
Looking at product recalls instead yields data that’s less vulnerable to manipulation and consequently offers a more objective approach.
"We found that rotating investigators provides the most benefit, and from the data in our study there appears to be enough inspectors to have them never revisit the same plant in a normal career," Ball said.
Adopting the first strategy could lead to about 100 fewer costly medical device recalls per year (or about 20 percent fewer recalls) in exchange for an increase of $800,000 USD in additional travel expenses for the FDA.
"Is 100 fewer recalls worth $800,000 a year?" Ball asked rhetorically.
"While the FDA has worked diligently over the last decade to strengthen their collaborative relationship with medical device manufacturers, this study finds a clear risk associated with this approach," said co-author Rachna Shah, associate professor of supply chain and operations at the University of Minnesota. "Overly familiar relationships and unwanted complacency may creep in for FDA investigators with prior experience inspecting that plant. This comes at a high cost for medical device consumers in the form of a significant increase in medical device failures leading to recalls."
Although the focus here was on FDA inspections of medical device manufacturing plants, it’s not much of a stretch to infer that the same problem exists in other industries as well. Intuitively, one would think that being more familiar with a plant would make an inspector more able to catch potential issues, but this study suggests that it’s just the opposite.
Experience is a valuable resource in manufacturing, but sometimes you need a fresh set of eyes.
For more medical device manufacturing news, find out why New Hampshire could be the next medical manufacturing hub.