Alum Jimmy Freese is fishing for sustainability with AI solutions

The challenges of fishing come from the blind spots.

More than two-thirds of the planet is hidden just below the ocean’s surface, so finding the fish is a challenge. Once a net or line comes up, the problem isn't a lack of sight, but rather an overload of it. Human eyes are the primary ool on deck to index several tons of writhing fish. It’s a feat of visual acuity to identify species, estimate size and age, make decisions on the next haul and comply with protections for endangered species.

To equip commercial fleets with both sight and insight, Jimmy Freese, a 1998 College of Engineering alumni, co-founded AI.Fish. There, he’s improving both sustainability and profitability, creating a rare kind of compounded win for fish, fishers, the industry, the environment and anyone who values the vibrancy of aquatic life.

The AI.Fish tale

In 2019, the growing computational power of AI tools for matrix multiplication got Jimmy’s attention. In earlier days, processing was too slow for advanced image or video recognition to be practical. But what opportunities would emerge if processing power for computer vision wasn’t a barrier?

“We looked into everything across many industries,” Jimmy says. “We got a hit with the people at NOAA that led us to say, ‘let’s do a fish thing.’ We wrote a grant, and next thing you know, a million bucks comes in and we've got a business.”

NOAA, or the National Oceanic and Atmospheric Administration, is best known for tracking climate and weather, but fisheries management and marine commerce are a big part of its mission to conserve and manage aquatic ecosystems and resources. Funding AI.Fish’s development of a sophisticated electronic monitoring tool helps commercial fisheries tackle the challenges associated with sustainable fishing.

AI.Fish camera systems installed on a boat’s decks watch what’s caught, processing visual data to determine the size and species of fish the moment they land on the deck. AI.Fish’s technology is a fast route to identify animals that are endangered and protected, helping fishers put them back in the ocean, and helping regulators assure that happens. It also helps to quickly identify younger fish, which need to grow and reproduce to keep their population stable.

“These are jobs usually done by a person standing on a deck with a clipboard,” Jimmy says. “It’s a critical regulatory function, but it’s expensive and inefficient.”

Jimmy points out that if AI.Fish was just a regulatory tool, it’d be a hard sell. His clients are eager for better data to improve their operations, where profits are hard won. One example: AI.Fish data helps track and avoid fishery freeloaders, such as whales, who help themselves to free snacks when fish are snared on a line. Avoiding what’s literally or figuratively eating into profits traditionally relies heavily on calls by the captain, and very little on the kind of data that guides decision-making in other industries.

“Commercial fishers have been trying to manage this using intuition and spreadsheets,” Jimmy says. “If you’re a Carnegie Mellon nerd like me, you realize we can make this more efficient. AI.Fish dashboards tell a vessel where and how they’re catching the most fish. When you roll in weather data or the experience of a good captain, the model becomes quite powerful.”

‘Carnegie Mellon nerd’ skills on the water

The capacity to dream up a product like AI.Fish came from two unique parts of Jimmy’s background: his CMU training and his lifelong love of oceans.

As an undergraduate chemical engineering major, Jimmy engaged with research to stabilize satellites, and developed computation skills in applied mathematics. A professor fostered his interests, and it’s the foundation of the work he’s doing today.

“There’s a major amount of mathematics in the computer vision work that we do. I couldn't do this without the rigor and technical foundation from CMU,” Jimmy says.

The original plan was a Ph.D. to become a professor like his parents, but entrepreneurship held more appeal than academia. As a native of Hawaii, Jimmy grew up in the ocean and the water called him home. After completing grad school, he returned to Hawaii, merged his engineering skills with his passion for surfing and co-founded his first company in 2001.

That company, AkuShaper, now leads the custom surfboard industry. Jimmy wrote a program to digitally design and shape a surfboard. A creator can test differences in the silhouette or contour of a board to see impact on predicted performance in the water, then manufacture the board with precision using computer numerical control machining.

“Surfers almost literally wear their boards, and like perfect tailoring or a high-end shoe, subtle differences in the cut or fit make a huge difference. Surfboards need to be designed for their wearer and where they’ll be used, especially to perform at professional or competitive levels,” he says.

Today, Jimmy still runs both companies, hiring staffers who share his unique philosophy and personal commitments.

“My colleagues and I aren’t just posing like we love the water — we’ve all lived it. I've lived and breathed the ocean,” Jimmy says. “I was a kid who cared about the ocean and through Carnegie Mellon, I was able to make my life’s work have meaning in the marine space. It’s all circled back to where I started."