Advances in photogrammetric imaging

Over the past 25 years or so I’ve been involved in many projects using remotely operated vehicles to collect high-resolution images of the seafloor and stitch them together to create photomosaics. That process has evolved tremendously during that time, both in the way we acquire the image data and in the way we process the data.

Historically, photomosaics were generated from overlapping still images collected along carefully planned ROV tracklines. Navigation data and identifiable site features served as tie points for SLAM (simultaneous location and mapping)-based image registration and mosaic construction. The images we collected were stitched together using custom software to create a two-dimensional “site map” of the shipwreck. This was more challenging when creating photomosaics of natural sites like geologic formations, deep sea coral ecosystems, hydrothermal vent sites, etc., because often it was difficult to identify tie points, requiring manual intervention. The quality of the mosaic was usually a function of the quality of data collection and environmental conditions such as visibility and the presence of fish, making it difficult to represent the third dimension accurately.

Commercial software and advances in computing technology, including parallel processing and high-speed graphics cards, have largely eliminated the need for custom photomosaic workflows, significantly reducing data-processing time and complexity.

Unlike the old-school SLAM method described above, structure from Motion (SfM) techniques use the images themselves and objects in them as tie-points to determine the camera orientation. This technique can be used to create highly accurate 3D reconstructions of the sites we image as photogrammetric models, not just 2D photomosaics. Stereoscopic cameras can also improve the data we collect, enabling accurate measurements and producing true-scaled composite imagery. We can also fly the ROV in the third dimension by rising up and down along the height of the wreckage or seafloor formation.

The Voyis Discovery Camera system used during the Heroic Age Expedition employed real-time vSLAM to display image point clouds during vehicle operations. This live feedback improved survey efficiency, reduced reliance on rigid tracklines, and increased coverage by allowing the same areas to be imaged from multiple viewpoints.

Initial, low-resolution reconstruction of the wreck of Quest. Credit: Canadian Geographic and Voyis

Now that the technology exists and we have proven the efficacy of the process on actual expeditions, vehicle engineers at WHOI and potential science users are excited that we can have this capability in-house, presenting a gamechanger for our industry. High-resolution 3D site models provide repeatable snapshots of seafloor environments, enabling researchers to quantify changes through time, including shipwreck degradation, hydrothermal vent growth/decay, and lava build-up. As photogrammetric workflows mature, digital twins of these environments will be able to support remote scientific interpretation and more cost-effective operations. WHOI is incorporating these capabilities into future vehicle programs, including mROV operations.

In addition, as we evolve in the way we conduct seagoing science, especially with remote operations, photogrammetry of sites and the creation of “digital twins” of seafloor environments can have data collected and processed by skilled technicians and allow for the scientific interpretation to be done on shore and even crowd-sourced. This process could eventually lead to a much more streamlined and cost-effective way to collect deep submergence data. We are planning future projects at WHOI, such as supporting future mROV operations, with this in mind and are excited to set the stage for a new future of deep submergence remote operations.