In this feature guest post, Matt Alderton discusses the rise of airborne imagery and how it’s lifting GEOINT to new heights.
Advances in commercial airborne imagery are transforming aviation from a mode of transport into an instrument of understanding. The private sector’s growing appetite for location information is putting fuel in the tanks of both manned and unmanned aircraft.
“Ten to 15 years ago, consumption of remote sensing data was almost completely dominated by government customers. And even if it looked commercial, there was probably government money behind it,” explained Richard Cooke, director of global remote sensing and imagery at Esri. “Now, commercial entities are starting to understand that their enterprises almost always have a location component to them. Because of that, they want more and more location information—and … remote sensing data is absolutely required to drive the majority of the rich content behind location information.”
Given the increasing accessibility and affordability of commercial satellite imagery, whose future is especially compelling with the advent of small sat constellations, many companies have fixed their remote sensing gaze on space-based systems.
And yet, airborne assets continue to offer unique advantages and singular opportunities, the realization of which promises to make geospatial intelligence (GEOINT) as ubiquitous in the private sector as it already is in the defense and intelligence communities.
A TOOL FOR EVERY JOB
From Leonardo da Vinci to the Wright brothers, the forefathers of aviation pursued flight for the euphoric and time-saving experience.
“The desire to fly is an idea handed down to us by our ancestors who, in their grueling travels across trackless lands in prehistoric times, looked enviously on the birds soaring freely through,” Orville Wright once said.
Photographers pursued flight for different reasons than scientists and engineers—they wanted to see what birds saw. In 1858, French photographer Gaspard-Félix Tournachon took the world’s first aerial photograph from a camera tethered to a balloon. Two years later, American photographer James Wallace Black went a step further when he took photographs of Boston from a hot-air balloon. As cameras became lighter, subsequent photographers mounted them to kites and even rockets.
Military commanders recognized the benefit of a bird’s-eye view almost immediately. Though none survived, the Union Army is said to have used aerial photographs from balloons during the American Civil War, and during World War I the French Army used aerial photographs from miniature cameras worn by pigeons. Reconnaissance from airplanes also debuted during World War I and was routine practice by World War II.
When it comes to appreciating, integrating, and exploiting airborne imagery, commercial enterprises are late to the party. Now that they’ve finally arrived, however, commercial users must choose their method—imagery from satellites, manned aircraft, or unmanned aerial vehicles (UAVs)—wisely.
“I don’t think any one is particularly better than the other; they just serve different use cases,” said Chuck Dostal, geospatial sales engineer at Nearmap, an aerial imagery company with operations in the United States and Australia. “There are tradeoffs for each of them.”
One of satellites’ greatest advantages, for instance, is wide-area coverage.
“The extensive coverage we can achieve with satellite imagery is immense,” said Brock Ryder, head of sales at senseFly, a commercial UAV company with operations in Switzerland, the U.S., China, and New Zealand. “You can capture data for a state, a country, or an entire continent.”
Then there’s temporal resolution.
“Satellites are always on,” said Alex Chernushin, director of commercial aerospace and strategic technology at Ball Aerospace. “Depending on what orbit you’re in, you can go over the exact same point on the Earth at a certain time of day and at a given revisit rate.”
Satellite disadvantages, meanwhile, include weather—most space-based systems can’t see through clouds—and speed: Imagery typically is not available in real time because downlinks are slow, and new sensors can take years to deploy. Spatial resolution is also a consideration.
“As you get farther and farther from the ground, resolution gets lower and lower,” said Dave Kroetsch, vice president of unmanned aircraft system solutions at Oregon-based FLIR Systems.
Where satellites fall short, fixed-wing aircraft excel and vice versa. Fixed-wing aircraft capturing images from the sky instead of space have superior spatial resolution but inferior temporal resolution, and more speed but less scale.
“A lot of commercial applications out there right now need really high resolutions, and I think that’s what’s driving the aerial market today and keeping it healthy,” Cooke said.
Among remote sensing platforms, UAVs offer the highest resolution for the lowest cost.
“Where drones really shine is when you’re trying to get incredibly high-resolution data that’s updated on a frequent basis,” explained Jono Millin, co-founder and chief customer officer at DroneDeploy, a San Francisco-based company that makes mapping software for commercial UAVs. “And because drones are so cheap, you can fly them on-demand. We have customers who keep them in their glove compartment; they just drive around in a truck, and whenever they see the need they … make a map and continue on their way.”
The compact size and limited battery life render UAVs impractical for large areas, but ideal for small ones.
“Manned aircraft are typically best for something on a larger scale,” Millin continued. “But if I have an area that’s less than 200 acres, and it’s changing a huge amount on a daily basis, drones are very well suited to that.”
AIR OF OPPORTUNITY
Airborne imagery is appealing to virtually any business that needs to understand assets or activities in a given time and place.
“There’s no disputing that there is huge capital in spatial awareness,” Ryder said. “Nearly all industries now utilize imagery—and aerial imagery, in particular, is phenomenal as a decision-making tool.”
The applications are seemingly infinite. And yet, a few use cases stand out as especially compelling. One of the most frequently discussed is precision agriculture.
“We see quite a bit of demand in agriculture—not for big row crops, which satellite imagery handles pretty well, but for specialty crops,” Cooke said. “If you’re a small farm that has a vineyard or is growing things like berries, you’re probably adopting aerial imagery.”
Small farms are especially interested in UAVs, according to Millin, who said agriculture is the second largest revenue driver at DroneDeploy.
“In the days of old, the process of crop scouting was just walking around in a random pattern hoping to stumble on a potential problem in the field,” he explained. “Now, we have drone technology that can, in real time and with no connectivity, create a map of your field and do some analysis on where we suspect there are potential problem areas.”
But DroneDeploy’s largest vertical is architecture, engineering, and construction (AEC), according to Millin, who cited a 2016 report by McKinsey & Company showing that large-scale construction projects on average are 80 percent over budget and 20 months behind schedule. “With aerial imagery, we can very easily and rapidly map a parcel of property and do the analysis that’s needed to determine whether work is going according to plan, and to figure out who’s doing what, where, and when,” he said. “That kind of information is what ultimately is going to help the construction industry drive down all its inefficiencies.”
Construction foremen can fly UAVs over their job sites daily if needed, said Joshua Ziering, co-founder and chief pilot of Kittyhawk, a San Francisco-based company that helps Fortune 500 companies adopt, scale, and manage UAV operations. “Construction people want to know what’s there, and they want to measure what they have,” explained Ziering, who said aerial imagery from UAVs enables change detection for AEC users, who can observe new structures and measure material stockpiles to assess progress.
For large-scale projects like roads, bridges, and railways, fixed-wing aircraft offer similar visibility.
“The AEC market tends to use a lot of satellite imagery, and it’s moving toward drones for site-based applications. But if you believe policymakers when they say we’re going to spend $2 trillion on infrastructure in the next 10 years, [fixed-wing aircraft] will be the only practical solution for those large infrastructure projects,” Cooke said.
Some of the largest builders of critical infrastructure are energy companies, which are adopting airborne imagery at a rapid clip, according to Millin, who said renewable energy projects such as solar are a growth center.
“The energy sector is using aerial imagery to do initial site surveys that assess the topography of the land. It can then use those topographical maps to do hydrology studies, and to understand what the grading of the earth should be in order to prep the site,” Millin explained. “And in the operations and maintenance phase, having people with handheld thermal cameras walk up and down rows of solar panels to manually identify potential diode failures or string outages is very inefficient. … Using aerial imagery, you can detect those issues faster.”
Maintenance is as critical for oil and gas as it is for renewables.
“Large refineries and chemical plants require constant inspection to make sure they’re up to the standards required of them by the federal government and by the companies themselves,” Millin continued. “Drones allow you to conduct those inspections faster and safer than ever before.”
Fixed-wing aircraft can service pipelines in the same manner, and can also enable remote asset monitoring.
“Oil and gas has a really significant problem to solve, and that is that its asset base is very geographically distributed,” Cooke said. “Even if you’re a mid-sized operator who’s working only in the Permian Basin in West Texas, that’s still a pretty large area in which you have to know on a regular basis where all your assets are.”
Because they face many of the same challenges as agriculture, AEC, and energy, industries such as mining, forestry, and real estate can apply airborne imagery in many of the same ways.
“I can’t think of an industry where we don’t have customers,” Dostal said. “Your imagination is the limit.”
Airborne imagery already has a wide and diverse commercial fan base. And the user community is poised to grow even larger with advances in payloads and processing.
The future starts with next-generation sensors.
“What’s really key is having not just the platforms, but also a selection of sensors that you can plug and play and switch around to get the job done,” said senseFly’s Ryder, whose company makes pocket-sized, hybrid sensors—a multispectral camera, for example, and a dual RGB/thermal mapping camera.
The eBee X from senseFly is a fixed-wing drone that supports interchangeable cameras and can fly three times as long as traditional quad copters on a fully charged battery. (Video courtesy of senseFly)
Industry-specific sensors are a goal post that will drive further commercial adoption, predicted Ball Aerospace’s Chernushin, who cited as an example his company’s Methane Monitor, an airborne LiDAR system that uses pulsed light to detect hazardous methane gas leaks in natural gas pipelines.
FLIR Systems’ Kroetsch said his firm makes thermal sensors designed specifically for users in public safety, construction, and security, among other verticals. One camera, for example, can help construction workers identify holes in a building’s envelope; another can help firefighters see fire through smoke; and another can help security professionals identify human and vehicle intrusions.
“There’s been a proliferation of visible-light cameras that have a bunch of applications in the commercial and enterprise world. But where the real value starts happening is with non-visible imaging,” Kroetsch said. “Those sensors open up a whole new set of applications.”
And yet, when it comes to visible-light imaging, what’s old can be new again. Case in point: oblique aerial camera systems, which in urban environments enable 3D modeling and mapping by way of capturing 45-degree images that showcase the sides of buildings the way traditional imagery showcases the tops of them.
“[Oblique] camera technology is really interesting,” Cooke said. “Companies like Nearmap and Vexcel are flying really intriguing cameras that allow [Esri] to do 3D point cloud generation or 3D textured mesh generation out of imagery, and that’s really differentiating [commercial airborne imagery] from satellite data.”
Echoed Nearmap’s Dostal, “People have been using aerial imagery for a really long time, so they know what value it brings. But they haven’t been using 3D. So, our next objective is to educate our customers on how they can use 3D products.”
The commercial potential for 3D imagery is vast. Real estate agents and developers can use it, for instance, to simulate views that help them value and sell properties. Public safety and security professionals can apply it for improved situational awareness. And everyone from telecommunications providers to automakers will need it as “smart cities” emerge.
“5G has a very different propagation model than 4G does,” Cooke said. “It’s much more subject to structural and vegetation interference, and because it’s a denser signal you have to have more antennas around to propagate the signal. You need high-resolution 3D models to do that propagation analysis to determine the location of cell sites, and airborne oblique imagery gives you that.”
Likewise with autonomous vehicles. “Autonomous vehicles will have devices to sense where they are in relation to what’s going on around them, but the vehicle has to have a dense 3D model as a base map to start with,” Cooke continued.
Importantly, 3D products will also enable the transformation of airborne imagery from a visual into an analytical asset, according to Paul Smith, a business development manager at Hexagon Geosystems. The company’s HxGN Content Program offers a commercial marketplace for orthorectified aerial imagery collected from airborne sensors made by Hexagon’s Leica Geosystems brand. One of those sensors is Leica CityMapper, a hybrid airborne sensor that combines oblique and nadir imaging as well as LiDAR into a single system.
Hexagon’s HxGN Content Program provides access to aerial imagery data for professional use. In this episode, Paul Smith, HxGN Content Program Product Manager, and Rich Grady, President of AppGeo, talk about the program and its streaming service for GIS applications. (Video courtesy of Hexagon)
“Our CityMapper sensors are … being driven for their derivatives, which are three-dimensional data that can be put into machine learning algorithms for feature extraction to discern things like building materials and roof characteristics. So you can understand not only that you’re looking at a house, but that you’re looking at a stucco house with a terracotta roof,” Smith said. “That’s what I see as the future of our hybrid sensors: They’ll be feeding machines.”
Dostal envisions a similar future. “Aerial imagery is rapidly changing into a more intelligent product,” he said, adding that Nearmap recently launched a beta product that applies machine learning algorithms to its imagery database for change detection and feature extraction.
Oil and gas companies, for example, can be automatically alerted to pipeline leaks while insurance companies can automatically receive images of damaged roofs after a storm.
“[Airborne platforms] generate gigabytes and gigabytes of images. How do you process those in a way that summarizes the data for the end user?” Kroetsch asked. “You use artificial intelligence. That’s the next frontier.”
SCALED FOR SUCCESS
Because algorithms must be trained, success in machine learning hinges on both quality and quantity of available training data. Thanks to its high resolution, airborne imagery has quality covered. Quantity, on the other hand, can be challenging due to poor temporal resolution. But that’s changing thanks to another fundamental shift in airborne business models.
“Seven to 10 years ago, the market was too fragmented. There was no consistency in how data was collected, processed, and used, and that made it difficult for anybody to gain a big foothold. As a result, everything was fly-on-contract,” Cooke explained. “A few years ago there was a lot of consolidation in the industry, and with that you started to see some economies of scale that have allowed a lot of aerial fliers to collect on spec.”
Among those fliers are Hexagon, Vexcel, and Nearmap, each of which collects airborne imagery on spec for use by commercial enterprises, who consume it via subscription-based web services.
“They envision a business model akin to what the satellite companies have: You collect it once and sell it many times,” continued Cooke, who said Esri hopes its ArcGIS platform will serve as a clearinghouse for companies’ speculative airborne imagery. “Everybody is going to have their own marketplace, but eventually somebody’s going to become an aggregator of all those marketplaces.”
The aforementioned economies of scale have made it possible for collectors to increase their revisit rates over high-value areas in pursuit of the temporal cadence that both enterprise customers and machine learning algorithms require. They’ve also reduced the cost of airborne imagery and made it accessible to new users.
“We’ve got a model now where everyone from [small businesses] to giant corporations can afford to use our imagery,” Dostal said. “It’s really disruptive, and it’s opening aerial imagery up to the masses.”
Even more disruption is likely. As the regulatory environment evolves, the federal government could eventually green-light commercial use of larger, fixed-wing UAVs, as well as UAVs that can operate beyond line-of-sight, both of which would have significant implications.
And yet, collectors and users of commercial imagery must resist the urge to keep score. Advances in the air won’t displace advances in space; instead, each will enhance the other, leveraging complementary strengths to solve shared problems.
“Airborne and satellite imagery are not necessarily competition,” concluded Kittyhawk’s Ziering. “If anything, I think they’re synergistic.”
This article originally appeared in the 2019 Issue 3 of Trajectory Magazine.