Clearing the Waters: Novel Combination of Imaging Tech and Algorithms

Engineering in the Wild to Save the Chesapeake Bay Oysters

By Kelly McSweeney

In a change of scenery from their usual work, 40 Northrop Grumman employees are developing technical solutions to monitor the Chesapeake Bay oysters. Since the mollusks purify water as they filter their food, they’re essential to the bay's ecosystem, which includes 3,600 species of plants and animals in 18 trillion gallons of water. The problem is there aren't enough oysters anymore.

“At one time, the adult oyster population was so robust that they could filter the entire Chesapeake Bay twice a week,” said Randy Janka, an engineer in mission engineering and systems architectures. “Now, they can do it once a year.”

Saving a Vanishing Population

According to the Chesapeake Bay Foundation (CBF), the oyster population is now less than 1% of levels within the past 100 years. Overdevelopment, pollution, overharvesting, a changing climate and disease have diminished this natural resource. CBF is working to solve the problem by contributing to a regional goal to add10 billion bay oysters by 2025. In order to demonstrate how these efforts are making a difference, CBF is working to monitor the oyster population.

That's why Northrop Grumman has partnered with CBF to develop a comprehensive system for monitoring oysters as part of a conservation reserve program. The project came out of the company’s internally funded Innovation Programs, which incubate innovative employee ideas. It’s an opportunity to support local wildlife conservation efforts while also fostering creativity and problem-solving skills. Five teams are working on different aspects of an end-to-end system for monitoring the Chesapeake Bay oysters, including acoustics, chemical, light field imaging and machine learning.

Loading component...

A Novel Use for Existing Technology Helps the Chesapeake Bay Foundation

It's difficult to monitor sea life in the Chesapeake Bay, which is known for sloshy, murky waters. Previously, CBF paid a professional diver take pictures to be able to count the number of healthy oysters. But this method was inefficient and expensive. Northrop Grumman's innovative solution uses technology to monitor the reefs. While the old process costs $7,000 per dive, the new way costs just $500.

Loading component...

In the new system, engineers used existing technology in a novel approach. First, an operator uses GPS to navigate a boat to the reef's GPS coordinates. Since GPS doesn’t work underwater, sonar is used to get the boat to the exact location to collect data. Then, the operator drops a remotely-operated vehicle (ROV) into the water, and then they use a GoPro to steer the ROV right up to the reefs to take photos. A real-time video feed gets even closer to the reef of interest.

The imaging comes from a light field image (LFI) sensor attached to the front of the ROV. The trick is that bay is so murky that sediment clouds the water and makes it difficult to capture clear photos of the oysters. The LFI system is like a 4D camera that captures images.

"By doing a lot of computer processing with that data, you can look at layers of photos," explained Janka.

Northrop Grumman engineers adapted a dehazing algorithm in the technical literature to see the oysters through the muddy water. "We computationally remove all those particulates to turn a fuzzy image into a clear image," said Janka.

The original algorithm took two to three hours to dehaze an image while running on a powerful desktop computer. This wouldn't be portable or fast enough for monitoring the reefs. The team refactored the algorithm and it's now able to dehaze an image in just two minutes while running between two processors on high-end gaming laptops.

In Dec. 2019, the team headed to the chilly bay for initial data collection to prove that the project was worth continuing. They succeeded and spent the next eight months further developing the system. Next, they will head back to the bay so that the machine learning group can capture more data to train their algorithms.

Learn More about Technology for Conservation at Northrop Grumman

Clearing the Waters: Novel Combination of Imaging Tech and Algorithms

Engineering in the Wild to Save the Chesapeake Bay Oysters

Saving a Vanishing Population

Loading component...

A Novel Use for Existing Technology Helps the Chesapeake Bay Foundation

Loading component...

Loading component...

More Innovation Stories

Turtle Power: Unmanned Aircraft Systems Supply No-Touch Tracking Solution

Polar Bear Conservation: Operation PolarEye

Project Turtle Tech

Loading component...

Loading component...

Saving a Vanishing Population

A Novel Use for Existing Technology Helps the Chesapeake Bay Foundation

More Innovation Stories

Turtle Power: Unmanned Aircraft Systems Supply No-Touch Tracking Solution

Polar Bear Conservation: Operation PolarEye

Project Turtle Tech