Dogedex AI App Camera & Machine Learning Module

This document provides a detailed overview of the camera and machine learning module within the Dogedex application. This module is responsible for real-time dog breed recognition, image analysis, other ML-powered features directly from the device's camera feed and Gemini Generative AI.

Overview

The core of this module is to provide an interactive camera experience where users can point their device at a dog and have the application identify its breed in real-time. Beyond dog recognition, the module is architected to support multiple ML functionalities like general image labeling, text recognition (OCR), and barcode scanning.

Core Technologies & Libraries

This module is built upon a modern Android technology stack, leveraging powerful libraries to deliver a seamless and intelligent user experience.

• UI & Foundation:

  • Jetpack Compose: The UI is constructed using Jetpack Compose (androidx.compose.foundation, androidx.compose.material), enabling a declarative and modern approach to building user interfaces.

  • Material Design: Implements Google's Material Design principles for a consistent and intuitive look and feel (com.google.android.material:material).

  • Coil: Used for asynchronous image loading, efficiently handling bitmaps and displaying images fetched from the camera or network (io.coil-kt:coil-compose).

  • Camera & Image Processing:

    • CameraX: The androidx.camera suite (camera-camera2, camera-lifecycle, camera-view) is used to simplify camera app development. It provides a robust, lifecycle-aware API for accessing and processing the camera stream.

• Machine Learning:

  • TensorFlow Lite: The core inference engine for running on-device machine learning models. The app uses the TensorFlow Lite runtime (org.tensorflow:tensorflow-lite) and support libraries (org.tensorflow:tensorflow-lite-support) for model execution.
  • Google ML Kit: A suite of powerful ML APIs for common mobile use cases is heavily utilized:
    • com.google.mlkit:image-labeling: For identifying general objects, scenes, and concepts within an image.
    • com.google.mlkit:text-recognition: For extracting and processing text from images (OCR).
    • com.google.mlkit:barcode-scanning: For detecting and decoding barcodes in various formats.

• Architecture & Dependency Injection:

  • MVVM (Model-View-ViewModel): The code follows the MVVM architectural pattern, with ViewModel classes orchestrating UI logic and data flow.
  • Hilt: com.google.dagger:hilt-android is used for dependency injection, simplifying the management of dependencies and promoting a scalable, testable architecture.
  • Coroutines: Asynchronous operations, such as image analysis and network requests, are handled using Kotlin Coroutines (kotlinx-coroutines-play-services) to prevent blocking the main thread and keep the UI responsive.

Features & Functionality

The MainViewModel.kt class reveals a set of sophisticated, interconnected features.

• Real-Time Dog Breed Recognition

  • The primary feature (Feature.DOG_RECOGNITION).
  • The recognizeImage function processes frames from the camera (ImageProxy).
  • It uses a custom classifier (classifierRepository) built with TensorFlow Lite to analyze the image and return a list of potential dog breed matches (DogRecognition).
  • The most confident prediction is displayed to the user.
  • The system also keeps a list of other probable matches to offer alternatives.

• Multi-Feature ML AnalysisThe module can be switched to perform other ML tasks on a captured photo.

  • The getMLDetailInformation function handles this logic based on the selected Feature.
  • Image Labeling (Feature.IMAGE_LABELING): Utilizes ML Kit's Image Labeling API to generate descriptive labels about the content of a photo (e.g., "sky," "building," "animal").
  • Text Recognition (Feature.TEXT_RECOGNITION): Uses ML Kit's Text Recognition API to perform Optical Character Recognition (OCR) on the image, extracting any visible text.
  • Barcode Scanning (Feature.BARCODE_SCANNING):Employs ML Kit's Barcode Scanning API to detect and parse information from various barcode formats within the image.

• Data Handling and Display

  • After a dog breed is identified by its ML model ID (mlDogId), the getDogByMlId function fetches detailed information about that specific breed from a repository (dogRepository).
  • The application uses LiveData to observe and react to changes in data, such as the identified dog's details (dog), the status of API calls (status), and the results from ML models (dogRecognition, mlKitInfo).

• Image Persistence

  • When a dog is recognized, the corresponding camera frame (Bitmap) is saved to the app's cache directory.
  • A content Uri is generated using a FileProvider, allowing the captured photo to be securely shared or displayed in other parts of the application.

• AI-Powered Breed Validation and Description with Google Gemini

  • The AI key feature of this application is its ability to provide rich, context-aware descriptions of dogs. This is achieved through a sophisticated integration with the Google Gemini Pro Vision API, which acts as an expert verification layer on top of the initial on-device recognition. This functionality is encapsulated in the getDogDescription() method. Here’s how it works:

  • Initial Recognition: The process begins after the on-device TensorFlow Lite model makes an initial guess of the dog's breed (e.g., ${dog.value?.name}).

  • Image and Prompt Composition: The application takes the photo captured by the user and combines it with a carefully crafted multimodal prompt. This prompt instructs the Gemini model to perform a specific, complex task:

    • Analyze the Image: "Analyze the dog shown in the image"
    • Validate the Breed: "Determine whether it belongs to the breed ${dog.value?.name}."
    • Provide Contextual Reasoning: If the model disagrees with the initial classification, it is instructed to explain why and suggest a more likely breed based on visible characteristics.
    • Generate Rich Descriptions: If the model agrees, it is asked to generate a concise, single-paragraph summary including key breed information like average lifespan, weight, height, and temperament.
    • API Call to Gemini: Using the gemini-pro-vision model, the app sends the combined image and text prompt to Google's servers for analysis.
    • Dynamic Content Update: The text response generated by Gemini is then received by the app and used to dynamically populate the dog's description field (dog.value?.copy(description = it)).This intelligent, hybrid approach ensures that the information presented to the user is not just a simple classification, but a validated, descriptive, and highly accurate summary, leveraging the advanced reasoning capabilities of a state-of-the-art multimodal AI.

🚀 CI/CD Pipeline & DevOps Integration

This project implements a professional-grade Continuous Integration and Continuous Deployment (CI/CD) workflow using GitHub Actions. Every code change undergoes a rigorous validation process to ensure stability and security.

🛠️ Pipeline Architecture

The workflow is divided into three sequential stages:

1. Automated Testing:
   • Executes JUnit unit tests on every push.
   • Generates and uploads detailed HTML test reports as artifacts for quality auditing.
2. Secure Build Process:
   • Compiles the Android application using Gradle.
   • Security First: Injects the GEMINI_API_KEY via GitHub Secrets, ensuring no sensitive credentials are ever exposed in the source code.
   • Optimized with R8/ProGuard for code obfuscation and resource shrinking (reducing APK size and protecting intellectual property).
3. Automated Deployment (Releases):
   • Upon successful testing and building, the pipeline automatically creates a GitHub Release.
   • Attaches the verified APK binary for immediate testing and distribution.

🧠 Intelligent Features

• Gemini 3.5 Flash: Leveraged for multimodal analysis (Image-to-Text) to provide expert breed descriptions.
• On-Device ML: Integration with TensorFlow Lite and ML Kit for real-time dog breed recognition.

📦 How to get the latest APK

You can always find the latest, verified version of the app in the [Releases] section of this repository. Each build is tagged and documented automatically.

📦 Videos

• Dog Recognition

  dog_detection.mp4

• Labeling

  labeling.mp4

• Bar Code

  barcode.mp4

• Text Recognition