Problem Statement
The platform combines computer vision, structured logging, and human-review safeguards to assist marine-debris monitoring while acknowledging the limitations of general-purpose object detectors.
Smart Marine AI focuses on end-to-end computer-vision prototype, qualified timing evidence, and human-review safeguards.
Project Objectives
Build an auditable YOLOv8n-based prototype that flags selected COCO object classes as potential marine-debris candidates
Document the limits of treating selected COCO object classes as debris proxies
Implement FPS, p50/p95 latency benchmarks and stress testing
Add structured logging with run_id, model_version, and config snapshots
Provide reproducible local and configured cloud startup paths
Design retry/skip logic and low-confidence warning system for operator trust
System Design
Vision Pipeline
- •OpenCV frame extraction
- •YOLOv8n inference
- •Candidate-class filtering
- •Bounding box output
Reliability Layer
- •Retry/skip policy
- •Low-confidence warnings
- •Error counts in summary
- •Long-run stability
Observability
- •Structured logging
- •run_id + model_version
- •Config snapshots
- •JSON batch artifacts
Deployment
- •Streamlit multi-tab UI
- •Software-only GPS vessel simulation
- •pytest unit + integration
- •psutil memory tracking
Data Flow Steps
Video / Drone Input
Accepts drone footage, webcam streams, or file-based video inputs
Frame Extraction (OpenCV)
Extracts frames consistently and normalizes resolution and color format
YOLOv8n Inference
General-purpose object detection outputs candidate classes, boxes, and confidence scores
Retry & Skip Logic
Retries transient errors, skips persistently failing frames with counts in summary
Low-Confidence Warning
Flags uncertain detections to support operator review and trust
Structured Logging & Output
Records run_id, model_version, timestamps; writes per-frame JSON + batch summary artifacts
Vision Pipeline Simulator
Smart Marine Inference Console
Simulate OpenCV frame extraction, YOLOv8 proxy-class detection, and warning thresholds
YOLO COCO Proxies
• Candidate Classes: 'bottle', 'cup', 'handbag', 'suitcase', 'sports ball'
• Warning Threshold: < 0.40 Confidence (triggers low-confidence operator review flag)
CPU Inference Results
All benchmarks captured as JSON artifacts. 10-minute stress run confirmed no memory leaks or FPS drift.
These committed CPU reports recorded zero detections per frame and do not establish useful detection throughput or model accuracy.
Error Analysis
Detection Analysis: High susceptibility to false positives when encountering generic objects of similar shape/aspect-ratio to target proxy classes.
Proxy Class Limitations: YOLO COCO default classes (e.g. bottle, cup) function as proxies; they do not represent real-time marine plastic classification.
CPU Throughput: Benchmarks average 10.8–18.7 FPS on mid-range CPUs; GPU acceleration is required for true real-time multi-stream workloads.
Confidence Calibration: Safe inference requires low confidence thresholds (e.g., < 0.40) to trigger explicit operator warning states for manual review.
Key Decisions
YOLOv8n model runtime
Ultralytics YOLOv8n offers the best speed-to-accuracy trade-off for CPU inference, which is critical for lightweight deployments
Streamlit for UI prototype
Enables rapid creation of interactive dashboards with file-uploaders and parameter controls without HTML/CSS overhead
Proxy COCO classes over custom dataset
Allows testing pipeline logistics and model-confidence thresholds without the high cost of custom annotation and model training
CPU-only benchmarking focus
Demonstrates baseline hardware resource limits and latency profiles for typical non-GPU deployment scenarios
Operator warning threshold
Triggers low-confidence alerts for operator review, ensuring a reliable human-in-the-loop validation step
Robust retry/skip policy
Prevents silent video-decoder timeouts from crashing the entire batch inference run
Visual Showcase
Engineering Challenges
Maintaining real-time inference on CPU-only hardware
Running object detection models on heavy video streams without GPU acceleration causes severe frame drop and lag.
Employed Ultralytics YOLOv8 nano model and optimized the inference loop by skipping redundant frames and processing at a downsampled resolution.
Achieved consistent 15+ FPS CPU execution, making low-cost local field deployment feasible.
Filtering COCO detections into marine-debris candidates without retraining
COCO datasets contain 80 general classes, while the objective is identifying marine litter specifically.
Built a mapping layer that filters detections to specific proxy object classes (bottles, cups, handbags) and filters out unrelated bounding boxes.
Repurposed pre-trained weights for debris monitoring without the high cost of custom annotation.
Designing reliable retry and skip policies for long-running video processing
Corrupt video frames or minor network timeouts crashed the entire batch processing pipeline, losing progress.
Implemented a robust frame-decoder try-catch sequence with dynamic frame skipping and offset recovery.
Zero pipeline crashes during overnight batch runs of heavy multi-gigabyte video folders.
Balancing inference speed against detection quality
Raising confidence thresholds missed small debris, while lowering it caused false alarms from normal objects.
Calibrated a dual-threshold filter (confidence >0.4 for auto-logging, and 0.25-0.40 for flagged operator alerts).
Optimized detection recall while filtering out background noise like water ripples.
Maintaining structured logs for reproducibility and debugging
Video outputs lacked clear metadata, making it hard to audit when and where a specific object was detected.
Integrated JSON logging with timestamp, frame index, confidence scores, and class categories for every single detection.
Standardized output audit logs easily searchable by external analytical tools.
Boundaries & Learnings
Limitations
- •Uses a pretrained YOLOv8n model trained on COCO rather than a marine-debris-specific dataset.
- •Candidate debris detection relies on selected proxy object classes and requires human verification.
- •No real drone or vessel hardware integration.
- •GPS routing is software-simulated.
- •CPU-only benchmarking; GPU acceleration not evaluated.
Key Learnings
- •End-to-end computer vision inference pipelines.
- •YOLOv8 deployment and OpenCV video processing.
- •Structured logging and experiment traceability.
- •Streamlit dashboard development.
- •Performance benchmarking and latency analysis.
- •Designing human-in-the-loop AI systems.