AI-driven Fish Identification and Regulation Insights for Smart Fishing

Customer Overview

The founders of a US-based startup couldn't recognize a local catch even after over 80 years of combined fishing experience. This inspired them to build a solution that could identify various fish species and provide corresponding information on local fishing rules.

Project Overview

The client wanted to build a mobile app that could identify fish species from live scans or photo uploads of catches. After recognizing the fish species caught and the user’s fishing location, the app must tell if it is legal to catch that fish.

Challenges

Building a solution that accurately identifies broader and fine-grained fish species and maintains precise rules for saltwater or freshwater fishing, corresponding to fish species and the fishing location.

• Even though thousands of fish species and subspecies are available across the US, the app must accurately identify local fish caught by users in any region.
• Different rules apply to each fishing region, even within the same US state, and they change seasonally. This vast amount of regulatory information is available from various government sources in different formats. The app must maintain up-to-date information in a usable format.
• The app must consider the fish species caught and the user’s location to match the correct and latest regulatory requirements for the user.
• The app must serve users not only in the US but also in other countries. It must identify any local fish and match location-based regulations across the world.

Solution

We built Android and iOS native apps that identify fish species and inform users if their catch is legal. We developed fine-grained image classification vision models to identify fish and used Gen AI LLM models with web scraping to extract fishing rules from government portals.

• To identify fish species accurately, we leveraged our Vision AI expertise and used Fine-Grained Image Classification (FGIR) techniques.
• We built custom AI models and trained them on a dataset of over 72,000 fish subspecies. Using AWS SageMaker, we developed an automated, scalable pipeline for efficient model training, testing, deployment, monitoring, and continuous learning.
• To source rules corresponding to fish species, fishing location, and season, we used web scraping, GPT 3.5 Turbo, and Langchain. We automated rule identification and extraction logic that runs periodically and persists the latest regulations in VectorDB.
• We build workflows for the solution to match the user location and fish species caught with the relevant regulatory data and provide custom results to the users.

Benefits

• AI-driven fish identification provides around 85% accurate output within 2 seconds.
• GenAI-enabled automated data updates lead to fast, precise, and scalable processes.
• The app services users not only in the US but also in Canada and Australia.
• Our client amassed a massive user base with increased revenue and business growth.

Technology and Integration

• AWS SageMaker
• Langchain
• ChromaDB
• GPT 3.5 Turbo
• Python
• React Native

Industry

• Fishing