Digital photogrammetry was first proposed by Ian Dowman in 1984 as a way to map the topography of terrain using satellite imagery. Photogrammetry is as old as photography itself, but the origins of photogrammetric theory stretch all the way past Leonardo da Vinci, who, in 1480, wrote: “Perspective is nothing else than the seeing of an object behind a sheet of glass, smooth and quite transparent, on the surface of which all the things may be marked that are behind this glass. All things transmit their images to the eye by pyramidal lines, and these pyramids are cut by said glass. The nearer to the eye these are intersected, the smaller the image of their cause will appear.”
The magic lantern is basically very similar to a device that acts likes the current-day slide projector.
This differs distinctly and is not to be confused with the photogram, popularized by 20th-century artist Man Ray, which is a way of taking a picture by placing objects on photographic paper and exposing it to light.
With the birth of computing growing into its adolescence in the cloud, we find ourselves in a time during which digital photogrammetric analysis is becoming more affordable in accessible photogrammetry applications, including those used by drones for aerial survey.
But what is the overall state of photogrammetry today, and how does it affect science, manufacturing, design and engineering?
In this post, we’ll cover a cross-section of technologies that best represent the current standards, practices and innovations in the field of digital photogrammetry. There are far too many photogrammetry software programs to cover each in depth.
The field has exploded with new content and new challenges, but some of the crucial problems that prevent photogrammetry from being as effective as light detection and ranging (LIDAR) in mapping and modeling different objects and terrain remain.
More importantly, how are people in different fields utilizing digital photogrammetry to innovate?
According to a paper entitled “3D survey technologies applied to the archaeology for the new "Municipio" underground station in Naples”, co-authored by photogrammetry expert surveyor Sebastiano Ackermann at Polytechnic of Milan, ”At the beginning, digital photogrammetry has been mainly used to generate orthophotos while laser scanners [were used] to obtain 3D point clouds. However, the improvements of the actual photogrammetric software, mostly thanks to the fusion between “classical” photogrammetry and computer vision world, allow now to obtain accurate dense point clouds in an almost fully-automatic way. The obtained results that will follow encourages the use of this image-based technique to extract 3D point clouds and lower the acquisition time.”
Basicially, photogrammetry is catching up to LIDAR, but what are the differences, really?
Difference Between Photogrammetry and LIDAR
LIDAR and photogrammetry are often juxtaposed and presented as being at odds with each other. Photogrammetry uses photos to make measurements between objects and create a geometric representation of the objects themselves, while LIDAR uses lasers in a similar way to the way radar uses radio waves to detect the position and geometric shape of an object by generating point clouds based on laser shots.
However, there is definitely no reason why the two cannot be used together.
Photogrammetry for Historical Preservation
Factum Arte, which performs a great service for humanity by attempting to capture and record cultural heritage sites and important historical artifacts in as much detail as possible, uses a combination of LIDAR and photogrammetry to record as much information as possible. This means capturing everything about an object and its surroundings.
When you stop and think about how much physical information is contained in a particular object, it is truly awe inspiring. In the case of cultural heritage sites and historical artifacts, the physical context of the objects contained within a site are just as important as the artifacts themselves. Factum Arte has to record surfaces, colors, forms, inscriptions, hieroglyphs and other details specific to each historical site and object it is collecting.
So what does Factum Arte use photogrammetry for in its quest to capture and store the most accurate representations of our history as human beings?
Digital photogrammetry is used to capture sites that are hard or dangerous to access due to the perpetual global conflict that engulfs a large part of our world. Different elements of extremism manifest themselves repeatedly over our history and want to destroy physical representations of cultural history to replace them with their own.
Digital photogrammetry is used by Factum Arte to thwart the total destruction of these at-risk cultural manifestations that are endangered by conflict and extremism. It is also used when they need to capture high-speed elements like people, animals, liquids and anything that isn’t motionless.
For surfaces, digital photogrammetry is used to record translucent surfaces, and Factum Arte built a custom rig that has nine cameras that capture a 3D area of 50 x 50 x 50 cm in four seconds flat. Factum Arte is also in the business of creating extremely accurate facsimiles for various reasons, including the potential damage caused by tourists, pollution and other corrosive elements. Photogrammetry is not currently used to create digital representations that require extreme detail, which the point cloud data created by LIDAR scans is much better suited to capture.
The camera you use to capture photos for photogrammetric processing into a 3D model counts, but most commercially available gear will suffice. A good depth of field helps and a good rule of thumb is that the closer you can take the photo, the better your stitched-together model will appear in its final state. Basically, the higher the resolution, the better the eventual model. One thing purveyors of photogrammetric software often neglect to mention is how long it takes to process high-resolution photos and how much computing power is needed.
With advances in photogrammetry software, cameras ranging from the one in your mobile device to a GoPro or your average DSLR are useful tools for photogrammetric capture and modeling.
Make sure you don’t miss any details in your photos in situ and be prepared for long post-processing times.
Photogrammetry in Aerial Survey
Surveyors working with architects and engineers to create a building or structure need access to detailed topographic maps and blueprints that need to be organized around the most realistic representation of whatever geographic area they using for either a new project, a restoration or both.
The global drone industry is literally taking off and providing architects, engineers, surveyors and many others in different disciplines with an amazing new tool to innovate and refine old methodologies in their respective fields.
The first simple thing to remember is to take overlapping photos of a structure, object or location. Good photogrammetry software will be designed to make 3D models out of photos and will be able to determine where to stitch together captured information and where to cut out excess visual 3D data.
There’s no need to eliminate photos from the ground all together and combing ground photography and aerial photogrammetry from a UAV (with a decent camera attached) can prove ideal for large-scale construction projects that cover a significant amount of land, which may or may not include some tricky geography.
Capturing the topography gives surveyors, architects and engineers a leg up in anticipating particular obstacles for the building and construction of any given project. A consistent hurdle for architects is anticipating the cost of having traditional geometric CAD models that don’t quite fit in with the actual data of a plot of land, a factory, or basically any potential project area.
Manual on-site measurements are not as exact as a photogrammetric 3D model, and they aren’t as easy to share either. Photographing the area from a UAV allows construction workers, surveyors and architects to keep better track of building materials they order and diagnose and anticipate ways to increase cost efficiency and safety.
A Virginia-based construction company called HITT regularly uses UAV photography to capture, plan and assess ongoing transportation, energy and water projects. For example, it recently captured and created 3D models of dirt piles and other materials on a 200-acre construction site that would eventually house 12 buildings on a newly created campus. Using UAV surveys and photogrammetric software, the company coordinated the delivery of building materials and was able to assess the right quantities of each material. This impacted the company by saving time and money.
If you’re interested in trying out a UAV photogrammetric capture of an object, structure or landscape, you’re in luck, because most commercial UAVs come with a decent camera. Barring your level of satisfaction with your “onboard camera,” you can always attach another camera such as a GoPro to up the quality of your captures. Set your camera to time lapse mode, take one picture every 1 to 4 seconds and try to get your photos to overlap with about half of each previous photo.
But that’s a bit too simplistic a set of instructions if you’re just getting started with UAV aerial survey capture for creating photogrammetric 3D models. Some companies, like 3D Robotics, have specialized software that actually controls the drone’s flight pattern as it moves over whatever terrain or site you are looking to capture. 3D Robotic’s Site Scan software will pilot the drone for you in a calculated and optimized flight pattern.
You’ve Got Your Photos Captured. Now How Do You Create Phenomenal 3D Models from Photos?
The answer is software. But what photogrammetry software is out there and how do you know which one is best for your particular needs?
There are dozens and dozens of different photogrammetry software applications in the marketplace, so we will just have to make do with a small cross-section.
1. ReMake (formerly Memento) for Manufacturing
Autodesk ReMake allows you to upload captured photos to the cloud or input them locally from a hard drive to begin the process of meshing them into 3D models. After your photos are uploaded, a mesh is created and that’s where all the manipulation begins. From ReMake, you can do all sorts of useful things to your uploaded photogrammetric mesh. You can edit your model and export it to a 3D file format (OBJ, STL, PLY, FBX or RCM) for use elsewhere in your workflow.
ReMake is also interesting because it allows you to perform measurement and differential analysis between two captured 3D models (which is useful for cool underwater photogrammetry science expeditions The Hydrous project) or check a CAD model versus the photogrammetric capture of an object after it’s manufactured.
AICON’s MoveInspect DPA is an industrial photogrammetry software that works in tandem with a handheld digital camera and AICON 3D Studio, which automatically processes imagery to create high-quality 3D models. Siemens used this combination of hardware and software to test large-scale gas turbine components.
These turbines were huge: 400 tons and 13 m in length by 5 m tall. They are also the crucial core business of the Siemens factory in Berlin-Moabit. Siemens is responsible for producing, testing and ensuring the quality of these components, as they are completely critical to the functioning of global power plants that have to adhere to the strict regulations.
Exact dimensional accuracy and even machining are critical to the stability of the gas turbine components, and through this combination of hardware and software, the original CAD data is compared to the photogrammetric scans. Incredibly, this photogrammetric system replaced a previous use of laser scans by leveraging a codified target system that tethers the photos to predefined markers in the CAD version of each component. This also allowed Siemens to mark off the best positions on each component for machining.
This photogrammetric software from Bentley allows engineers of companies to produce high-quality infrastructure models in a similar way to Autodesk ReMake. What it appears to be particularly good at and designed for is creating GIS 3D models that are actually embeddable online. You can use them with an SRS (Sequence Retrieval System) database for maxed out GIS interoperability.
You can use a wide variety of cameras (from smartphones to expensive DSLR hardware) and create editable texture-mapped details. Then you can publish this data to the Web using a free plug-in viewer.
The United States Geological Survey produced an extremely impressive photogrammetric survey in the field of planetary science with nearly 1,000 analog photos captured from the Viking Orbiter. To illustrate the difference in time spent creating photogrammetric models through analog and digital, think about this: A team composed of photogrammetric analysts spent most of the 1980s to put together this photogrammetric model.
Using SOCET SET GXP from BAE Systems, the production of this kind of model took “60 hours of automated matching and 90 hours of interactive quality control and editing,” according to this SOCET SET brochure. Unbelievable.
SOCET SET is part of modern advances in digital photogrammetry that make it much easier and more practical to compute and change huge amounts of image data into a comprehensive and shareable 3D model for visualizing and analysis.
_____________________________________________________________________
Biggest Problems in Photogrammetry
The computing part of creating accurate and measurable 3D models after you’ve captured reality data involves image matching and analysis among whatever number of photos you’ve taken to compile. But image matching isn’t just part of modern digital photogrammetry. Whole academic careers have been spent researching, developing and implementing practical solutions in different software systems.
Otherwise known as the “stereo correspondence problem,” it concerns the way software determines the X, Y and Z coordinates from measurements of just two dimensions in the form of X and Y coordinates. It is necessary for an object point to be present on at least two images, but here is the key fundamental concept to understand: The more images you use, the more advantages your multiple stereo view will be able to utilize.
Occlusions are surfaces and topological details that are obscured by other features, and the higher the number of photos, the more redundancies occur with object point overlaps. This increases the opportunity for software to improve precision and ensure higher levels of accuracy.
This ratio of the number of photos to the level of accuracy comes along with a relationship that directly affects the length of time it takes to match more images, because more and more computing power is needed to do this effectively and efficiently.
For photogrammetry software programmers, this means creating the best image-matching algorithms based on the specific features you need. For example, you could need area-based or relational-based matching depending on the intended specific end use.
Photogrammetry has benefitted from the use of tie point extraction features utilizing key points (X, Y coordinates). We know that the 3D geometry is created by multiple photos of the same subject, object, terrain or structure from many different spatial positions. What this does is make a visible pixel appear (or be designated) as a single point in more than one image.
So, a straight line is “drawn” from the centermost point of a camera to this pixel point. There are multiple images, so there are multiple lines to multiple points from the center of the camera lens.
Naturally, given the spatial positions of each photo around the object, these lines are going to intersect. These intersection points contain the 3D location of object points. This presents an issue: The position and orientation of each image has to be known, and this is where tie points come in.
The tie points link each image together and are identified in every image where the linkage occurs. However, this requires that the position and orientation of each image be known. Sufficient tie points allow for the reconstruction of the relative position of all images, giving the 2D images their 3D geometry.
Interestingly, computer vision algorithms have been integral in improving the process of automating image-based 3D modeling along with image-matching algorithms in open-source and proprietary software. They generate dense point cloud information from terrestrial and aerial data that is acquired from two types of image data: converging images and parallel axis images.
The next step is running geometric analyses on the point clouds produced with these different algorithms. These are compared against each other and ground truth data (directly observed without inference).
Experiencing Photogrammetry in Virtual Reality (Beats 360 Stereoscopic Video)
As developments in virtual reality continue throughout the coming years, no one knows for sure if it will reach mass acceptance as a consumer product. Viewing captured photogrammetric 3D models in virtual reality offers users the ability to focus on detail in a better way than stereoscopic video.
For example, look at the way viewers are able to move in and around this photogrammetric capture in virtual reality for the HTC Vive. The video comes from German company Realities.io and was posted earlier this year in UploadVR.
As you can see, there is way more ability to move around and experience the details of a photogrammetric capture. According to this article, the photogrammetric scenes were created by a series of snapped images taken in a way that increases the distance a position of an object appears to be from the position of other objects, like the camera and viewfinder.
The capture process takes a couple of days according to Realities.io, but the result is more navigable for viewers and the scene achieves a high level of photorealism.
I have a friend who showed a photogrammetric 3D model of Iceland to a 70-year-old woman who currently lacks the mobility to return to the country in which she grew up. For pure visualization of captured and stitched photogrammetric 3D models, the combination of photogrammetry and virtual reality can’t be beat.