PTC has sponsored this post.

As an editor at engineering.com, the vast majority of stories I’m told are about success: a novel way that a product was designed, a simulation that saved millions of dollars, AIs that prevent catastrophe and products collecting data from the field to optimize future designs.

It’s rare I hear about failure, even though it’s through failure that we all learn best. I wonder how many engineering mishaps can be avoided if our community shared about the bad as well as the good?

Enter Anji Seberino, CAD Field Subject Matter Expert and Director at PTC, an engineering expert who understands the potential benefits of learning about the ubiquitous pitfalls that engineers face. To that end, she has agreed to break tradition and tell us three horror stories and the lessons she, and the organizations she worked for, learned.

“I think these nightmares are more common than we hear about,” said Seberino. “We don’t tend to talk about them. The consequences that can happen are sometimes financial and sometimes more serious. But it’s as simple as taking the time up front to figure out how you want to do engineering. We should never be settling for something that is good enough. We should be wanting to do the best engineering we can do.”

Chances are these stories will sound familiar—they were familiar to me—but the solutions are often harder to find. So, instead of your engineering team needing to rediscover the solution to these common problems, heed Seberino’s warnings or read How to Choose the Optimal Math Software for Engineers.

The Tale of Reinventing Fourier’s Wheel

Early in Seberino’s career, she worked for a defense contractor that was producing autonomous target tracking software. Her job was to develop the part of the software that could take a Fast Fourier Transformation (FFT), an inverse FFT (IFFT) and correlate image data. Unfortunately, the tools available to her did not have a library of signal processing functions. As a result, she had to code it herself.

“I wrote the C code for the FFT, IFFT and correlation functions that we needed to use. The problem was transforming a 2D dataset into the frequency domain, effectively taking the FFT. It’s a very complicated numerical recipe,” said Seberino. “There was a problem with my code, and it took days, probably a week, to debug. That happened because I wrote the functions myself, they were complex and I didn’t have a software solution and library available that has those functions compiled, tested and ready to go. It was a nightmare.”

The algorithm needed to be finalized so it could be integrated on a system that was on a tight timeline. The amount of time that Seberino’s team took debugging the code caused major delays because system component and integrators were waiting for the code. Colleagues checked it over, code reviews were performed, and a lot of work hours were spent on this.

“On top of that,” she said, “even after debugging it, I was insecure about whether or not it was working correctly. So, even after finding a fix, I never had total confidence that it was doing what it was supposed to be doing. I would have been more confident if I was using a solution whose job it was just to produce that function, because then I would know it was right. If I had used a precompiled FFT library from a reliable source, this never would have happened.”

If those potential errors made it to the field, then the targets wouldn’t have been placed at the correct spot. This could have been a catastrophic event for whomever was using the weaponry. As a result, Seberino’s team thought it best to validate the code using calculation software.

“We actually ended up purchasing the calculation software for the validation process.” In other words, to solve the problem, Seberino’s team purchased software that she should have had access to in the first place, to validate a code that should have been used to produce that software.

“Thinking back on it,” Seberino added, “I know that wasn’t the right way to do it. We didn’t take the right approach from the beginning. It’s easy to look back and see that now. But when you’re in the midst of it, you don’t see it. That’s why it’s really important to put thought upfront so you don’t end up in a situation like that.”

To find which software has access to the tools you need, read How to Choose the Optimal Math Software for Engineers.

The Mystery of the Past Employee’s Incomprehensible IP

Seberino’s second horror story involved a spreadsheet she inherited from a previous employee. It would collect, reduce and analyze data from the field and simulations. The results would then be used to inform future development. The spreadsheet was straightforward when it came to reducing the data using a series of filters, but the analysis portion was incomprehensible.

A PTC video on the benefits of purpose-built computational software.

“There were so many sheets in the file, and they all had interdependencies. Since I inherited the spreadsheet, the analysis was happening, but I couldn’t explain it to anyone or myself,” said Seberino. “I would click on a cell or a formula that applied to an entire column of data, thousands and thousands of rows of data, and that formula would start with 25 left-handed parentheses followed by numbers, letters and symbols of all different kinds.”

To be able to properly develop conclusions from data, you need to know how it’s being assessed. But Seberino and her team had no idea what the original owner did when producing the spreadsheet. The team could read the formula and string the math operations together, but comprehending why these operations were happening, finding the intent and keeping track of them was the real challenge.

Seberino explained, “I could tell you what it was doing, but I didn’t know why, or what it was supposed to be doing. It wasn’t transparent, it wasn’t documented. These are the things we struggle with when trying to do things in Excel.”

This problem likely sounds familiar to many engineers. A spreadsheet spreads through a department, team, or company and typically only one person understands how it truly works. When that person leaves the company, all that intellectual property hidden in the spreadsheet goes with them.

“What we ended up doing was pretty ridiculous,” chuckled Seberino. “We created a task force to reverse engineer the spreadsheet. That’s how complex and crazy it was. A small team was needed to dissect the Excel chaos. And then, we documented the formulas once we figured out what they were. So, going forward, anyone that had to use that file would have a reference doc on hand. That exercise alone took about a week of work hours for three or four people.”

This is serious when you think about it. That week of work from those highly trained engineers could and should have been used towards a more productive task. This can get expensive if it happens regularly and you miss deadlines or fall behind on critical work.

“It’s surprising that we still see a lot of this happening today,” noted Seberino. “We still see a lot of reverse engineering happening to figure out what somebody did in a spreadsheet. And sometimes engineers uncover errors in these spreadsheets.”

In this scenario, the company relied on the work of the taskforce to decipher the spreadsheets. The irony is that this only extends and delays the risk. Once all of those on that taskforce leave the company, or the spreadsheet evolves (even with a reference document) the organization could finding itself back at square one.

To help maintain company IP with respect to engineering computations, read How to Choose the Optimal Math Software for Engineers.

The Curse of the Perfect Theory’s Missing Unit

The third nightmare story Seberino witnessed involved a colleague who was trying to design an optical sensor technology. To do so, he developed an inventive theory which originally gained much praise in the organization. The theory was sound, and it was reviewed and approved by multiple physicists and optics experts at the company. Seberino’s colleague was recognized for his innovation. But what happened next nearly got him fired.

“The next step was to implement all the theory that he got so many accolades for, and get the numbers so they could build a prototype,” said Seberino. “That’s where things went wrong.”

The experts checked the theory, but they did not check the spreadsheet. They spent so much time scrutinizing the science, it did not occur to anyone that the spreadsheet could have an error.

“You can probably predict where this is going,” said Seberino. “There were a few errors, one of them was an incorrect unit conversion. Nobody caught it. And long and short of it, they ended up manufacturing the prototype for this super expensive, high-end sensor. If he had done a better job with the calculations, it could have been a huge success story for him.”

“Sometimes it comes down to a plus or minus sign,” added Seberino. “Those are really easy to miss in a spreadsheet. The majority of spreadsheets have errors.”

Though the engineer nearly got fired, how he grasped success from the jaws of defeat is what saved his career. Right after this incident happened, he sent a request for the purchase of purpose-built software to do these computations. Clearly the company also learned a lesson as he quickly got approval.

“The engineer implemented the design in that software, and they built another prototype. The second one came out much better than the first,” said Seberino. “They were able to see the difference. Not only were the calculations correct, but the IP was captured, too—something that isn’t possible in spreadsheets. It could be validated and reused with confidence.”

You see, the norm at the company was for engineers to just use whatever tools they had available at no extra cost, which tended to be spreadsheets and code. Now, the whole department was being trained on the computational software.

“They made a shift to how they did things, but this is what it took to get there,” noted Seberino. “It's a problem we see everywhere. People say all the time ‘Excel is good enough for what I need to do,’ and you really need to take a step back and think about that. Is it good enough? Or are you using it because you can?”

The computational software became a standard in the organization. The implementation of theories and other calculations were now all done in purpose built computational software. To that end, I argue that this fiasco of a failure was transformed into a success, since the organization changed its ways.

“Yeah, I think so, too,” said Seberino. “In the end, it was a big success story because it forced them to take a really close look at how they were doing things and to make a long-term change. That was ultimately very good for the business.”

To avoid unit and computational errors in your computations, read How to Choose the Optimal Math Software for Engineers.

Purpose Built Software is a Sure Lock to Success

It was astonishing how much the problems Seberino described were like my own experiences:

• During my Masters, I had to recreate a known, tested and compiled software within spreadsheets because I didn’t have the funding to get a license.
• During my first co-op, I modified data processing code my boss wrote in Perl. He had forgotten how it worked because Perl doesn’t require you to declare variables before using them.
• During my last co-op, I had to optimize a factory’s workflow by modeling it in Excel. I did not have access to factory simulation software because the R&D team was recently gutted.

A PTC video that shows how Mathcad stands apart.

“You’re proving my point, that these problems are not one offs,” said Seberino. “They are everywhere, they are ubiquitous! It really goes back to the premise that we’re not doing a good enough job upfront by asking what software we should be using. We need to take that part of the process seriously.”

She added, “I think engineers need to take these cases to their management and advocate for the tools they need to do their jobs better and effectively. It goes back to accuracy, precision and reliability. You need to go to your managers and say, this is what I need to be able to do my calculations, do them well, and be a better engineer.”

As for management that refuses, Seberino suggests that engineers prepare for those meetings by bringing lots of case studies of the failures that can happen. But as engineers don’t discuss these issues often, these case studies are typically limited to famous, outdated stories like the NASA Mars Climate Orbiter that crashed into Mars 1999, and the NYC Citicorp building discovered to have a structural design flaw in 1978.

“We see this happening all the time in engineering,” Seberino said. “Spreadsheets get passed around, reused and distributed. That’s not good for the engineer, your career or the organization.” The real solution is to use the software purposefully built to do this work in the first place.

To prevent these issues from affecting your organizations, read How to Choose the Optimal Math Software for Engineers.

To learn more about math software for engineers, visit PTC Mathcad.