Digital Twin Makes Bridge Inspection Safer, Cheaper and Accessible

Most departments of transportation (DoTs) are already there, according to Vogen.

It is nearly impossible to read or hear anything about infrastructure that does not refer to it as crumbling. A bridge failure, such as what occurred in Minneapolis, Minn. (more on that later) in 2007 or the Morandi Bridge[1] in Genoa, Italy in 2018, is as momentous as a plane crash, generating headlines across the globe.    

The American Society of Civil Engineers (ASCE) declared 12 percent of U.S. bridges to be structurally deficient in 2007. The ASCE counts 617,000 bridges in the U.S. and estimates that 42 percent of them are over 50 years old, and in its 2021 Report Card for America’s Infrastructure, the organization estimates that 7.5 percent are structurally deficient.

You might expect Minnesota to be leading in structurally deficient bridges, but the state is not even in the top 10. The biggest proportion of structurally deficient bridges is in the nation’s smallest state. Rhode Island has 22 percent of its structures graded deficient, followed by West Virginia, with 21 percent.

Realizing the need to digitalize bridge inspection, Bentley bought Pittsburgh-based InspectTech in 2012.

“Bridge inspection methods have remained very, very stagnant. Inspectors have to get in a position from which they can properly view and assess what’s going on across every single bit of that structure. This has involved bridge inspection vehicles, the snooper trucks, actually rappelling over the side of the structure to hang over the side of the bridge. Sometimes, a watercraft is deployed underneath. Scaffolding may be built. Anything to be able to get into a position so you can tell what’s happening. Everything that we’ve done for years and years and years has been with those methods. We had to so we could physically get into every nook and cranny of a structure. However long the bridge, however high off of a river or whatever it is crossing.”

A snooper truck, which must park in traffic (bridges are not routinely closed during bridge inspections) and have access to 10 or 20 feet of bridge length and perhaps under only one lane, is rather limited compared to the freedom of movement that a drone provides.

Around 2018, Bentley started seeing the advantages of drone-based inspection, notes Vogen. “Like access to hard-to-reach places. Instead of having to get a snooper truck and move the arm around, we can fly a drone up under the deck, along the beams … and get the picture.”

The relaxing of regulations for drone use has helped foster their use. A drone pilot no longer must have a line of sight to the drone at all times.

Given the need for bridge inspection to prevent disasters as well as to a better job and reduce costs, we must resort to technology. We ask Vogen what Bentley brings to the table.

“For many, many years, Bentley has brought software that allows for the overall oversight, coordination, documentation and compliance with the inspection requirements in the U.S. The U.S. Federal Highway Administration has mandates for bridge inspections for 50-plus years. Those standards have not changed for quite a few years. They require an inspection of every structure every two years. That inspection [must be] done on site, visually, hands-on, generating inspection reports that detail the condition of every structure in the inventory of the Federal Highway Administration on an annual basis. We have provided the tools to allow the inspector to be in the field, to do their visual assessment, to make notes, to take pictures or videos, to build the report that’s required for each structure and to create what has been termed the ‘tape’ that must be delivered to the federal government each year.

“The ‘tape’ is a textual record with a specified format. There’s one for every bridge, every two years. There’s a couple hundred fields for every single structure, line after line, in a text file. The tape gets submitted for every structure. The federal government combines all the reports into one with the entire inventory for the whole of the U.S. so it can evaluate the overall condition, costs of rehab that are required or estimated.”

A drone, or unmanned aerial vehicle (UAV), can be flown on a predetermined path, like a helix around the bridge, or back and forth along its length, all the time taking still pictures or video.

Bentley software can piece together the photos to create a visually accurate and detailed model of the entire bridge in a process using photogrammetry technology. Bentley acquired photogrammetry technology from France-based Acute3D in 2015. Since then, it has integrated photogrammetry into its applications.

Bentley’s photogrammetry stiches the pictures together on the fly, requiring no registration marks or a painstaking manual effort of sorting and connecting the thousands of images that return after a drone’s 20-minute flight—a common requirement of earlier photogrammetry programs.

“Our ContextCapture software takes those multitudes of photos and turns them into an at-scale reality mesh. There will be benchmarks or way points or key points that are used to assist in the accuracy and the measurability of that model,” said Vogen.

The resulting model is based entirely on still photographs. It can be augmented with LiDAR, according to Vogen, but doesn’t require it.

The bulk of a bridge inspector’s time is spent not in the actual bridge inspection but in driving between the office and the structures that need inspecting.

“It is not good use of an engineer’s time,” noted Vogen. Bridge inspection is hard enough. Why burden the engineer with all the extra baggage?

“An engineer has to bring all their equipment. They have to take photographs. They’ve got to take it all back to the office. When they are done with a bridge, they have to drive to the next. They probably can’t do two in one day if they have to drive back and forth. They will go to the next bridge tomorrow. Ask the typical bridge inspector how many hours of their life they’ve spent driving to structures. Is that a good use of a skilled engineers time?

“But what if they could have a drone crew?” pondered Vogen. “Then engineers can be constantly used for their skills. So can the drone crew. Drone pilots don’t have to be trained engineers, just competent, certified pilots.”

Nor is it good use of resources to teach an engineer how to rappel or fly drones. A licensed pilot is required to fly a drone. Organizations that own, maintain and fly drones can be hired to “fly a bridge.”

Using a professional, dedicated drone team has another advantage, according to Vogen. “They save the flight path that was used there. And if we want them to come back every six months, every year, they fly the bridge again and create another model. We do an apples-to-apples comparison showing changes.”

“We’ve been recording details with photos for 40 years, but with each inspection, we start all over again,” said Vogen.

What’s wrong with that?, we have to ask.

Vogen resorts to a parallel to answer our question.

“Go take a picture of a flower in your garden, go back the next day and take a picture of that same flower. Were you the same distance from it, the same angle, the same perspective? Will you know if the flower grew a half an inch or an inch or what? When an inspector takes a picture of a crack and compares it to a picture of the crack two years later, how well [are they] really assessing what changed?”

Or was the crack missed and not part of the photographic record? A camera-equipped drone on a repeatable flight path is more likely to be complete than a changing inspector, who might focus on some things to the exclusion of others. Also, with a visual digital twin, inspectors don’t have to go back to the bridge. The bridge model, in effect a 3D photograph of the entire structure, can be viewed repeatedly with no additional field trips.

The model is generated from thousands of pictures. It is geometrically accurate. Unlike normal photogrammetry models that have no scale, Bentley’s model is dimensionally accurate and to scale. Models are calibrated with known distances on the ground to achieve a geometrically accurate model.

That’s better than you can do with a photograph—the time-honored method of recording details like cracks, corrosion, the size or thickness of a gusset plate … or other details worth noting, says Vogen.

One such organization is Denver-based Collins Engineers, which has been working with the Minnesota Department of Transportation (MnDOT) in a 5-year program to use the latest technology, including drones and photogrammetry, to inspect bridges.

The MnDOT had been using traditional methods to inspect bridges, including snooper trucks (see image above) that park on the edge of a bridge, often in traffic, and extend an elaborate, reticulated arm to position one or more brave inspectors under a bridge. A snooper truck costs three quarters of a million dollars, said Jennifer Zink, state bridge inspection engineer for MnDoT, “compared to around $40K for a drone.

It was in Minneapolis that the I-35W bridge collapsed into the Mississippi River in 2007, killing 13 and injuring 145, instantly making Minnesota the poster child for crumbling infrastructure. It was a bad rap. The National Transportation and Safety Board (NTSB) was to determine that failure resulted from a design flaw. Gusset plates were a half-inch thinner than they should have been. Having nearly 300 tons of construction equipment stockpiled on the bridge deck didn’t help. It was not just a matter of bridge deterioration, deferred maintenance or lack of inspection.

The state of Minnesota jumped into action, replacing the I-35W bridge in 14 months, replacing every undersized gusset plate and repairing and replacing the state’s bridges as necessary, as part of $2.5 billion bridge improvement program.

If the prerequisites to bridge inspection already include rappelling, lack of acrophobia and a love for the great outdoors, why not add drone flight to the list?

Not a great idea, says Vogen. A civil engineer doing bridge inspection used to have to deal with the Federal Highway Administration. With drones, they must deal with the Federal Aviation Administration (FAA). The FAA has only recently relaxed restrictions for drone flight (earlier this year), making drone-based bridge inspection possible.

“Federal regulations in place for quite some time demanded the drone pilot had to have line of sight to the drone. The FAA now allows use of the drones without line of sight,” Explained Vogen.

Advantage: Drone

With drone-based reality capture, the bridge inspector can stay in the office, be safe and be more productive.

“The Minnesota DoT found drones saved 40 percent of the cost of a bridge inspection,” said Meg Davis, industry marketing director of Road and Bridge, Bentley Systems. “And Collins Engineers found that 90 percent of bridge inspection can be done in the office.”

A bridge inspector doesn’t have to learn to master drone flying and be certified. A set of photographs can be a deliverable from the drone team. These match those flying drones and performing bridge inspection each to their respective specialties, avoiding learning curves and amateurish bungling.

With drone-based bridge inspection accepted as the modern and safe method of bridge inspection, the next revolution may be the method by which the resulting photographic 3D model is to be viewed.

We’re in the early stages of viewing the images in the most optimum way, says Vogen.

Bentley is a pioneer in adopting the Microsoft HoloLens so that bridge inspectors can immerse themselves in a full-scale model of the bridge they are inspecting.

“We are doing pilot projects with a few departments of transportation,” noted Vogen. “We started development a year ago. Honestly, I was concerned whether this would be a little gimmicky, flashy … with not enough engineering accuracy. But with the software and the support that Microsoft [owner of HoloLens] has been able to provide, the engineering consulting firm [Collins] has been able to do 90 percent of the inspection and assessment in the office. That was mind blowing. When we started, we were hoping to for 10 percent or 20 percent. I think we have hit a home run.”

Bentley had to overcome data, hardware and software challenges to immerse a bridge inspector in a virtual environment.

“The technology that’s being used is extraordinary. The models that we’ve been working with in the pilot projects are 10, 20 … a 100 million polygons. Rendering models that size inside a HoloLens has been, if not a technical marvel, at least a serious technical win. We’re leveraging new functionality from Microsoft that allows huge polygon counts to be rendered in the cloud and streamed into the HoloLens—with all the fidelity and detail that’s needed so that the engineer doesn’t laugh at it. We’re not doing SimCity cubes here,” said Vogen.

Virtual bridge inspection works on so many levels. The full utilization of advanced technology makes us instant fans, but decidedly more important is the added safety for bridge inspectors, not to mention road crews on the bridge deck braving traffic. The most a bridge inspector reviewing a digital twin of a bridge has to fear is falling over the furniture they won’t see while wearing the HoloLens.

Public safety will be enhanced by better and more complete bridge inspection. The civil engineering profession will be recharged with the talent of those who, by limits of ability or fear, were unable to dangle off bridge decks. The division of labor virtual bridge inspection allows—with drone pilots flying, software jocks processing images into models and bridge inspectors able to focus on analyzing models—saved from spending hours a day driving to and from structures, is bound to produce a more contented team. And, bottom line, the savings of virtual bridge inspection over physical inspection (as much as 40 percent) simply can’t be ignored.