Why detection at sea is different
Every object-detection tutorial ends with a confident bounding box around a coffee mug. The ocean disagrees with tutorials. At sea you get sun glare that saturates half the frame, whitecaps that look exactly like small buoys, a horizon that rolls ±15°, and salt spray on the lens dome within twenty minutes.
This note documents the perception stack for Remora, our autonomous surface drone, from dataset to a 21 FPS deployment on a Jetson Orin Nano.
The dataset problem
Public maritime datasets exist (Singapore Maritime, SeaShips) but they’re mostly shot from shore or large ships. From a drone 40 cm above the waterline, everything looks different. We built our own set:
- 3,800 frames extracted from GoPro runs at the local harbour and offshore.
- Classes:
buoy_nav,buoy_mooring,vessel,kayak_sup,swimmer,debris. - Annotation in CVAT, roughly 30 hours of clicking.
Two augmentations mattered more than all others combined: horizontal flip and simulated glare (random white ellipse composited at 40–80% opacity). Glare augmentation alone cut daytime false negatives by a third.
Model and training
YOLOv8n, 640 px input, 120 epochs on a desktop RTX 3060:
yolo detect train data=remora.yaml model=yolov8n.pt \
imgsz=640 epochs=120 batch=32 hsv_v=0.6 fliplr=0.5Results on the held-out sea-state-3 test set:
| Class | mAP@50 |
|---|---|
| buoy_nav | 0.91 |
| vessel | 0.88 |
| swimmer | 0.79 |
| debris | 0.54 |
Debris is hard. Debris will stay hard.
Deployment on the Jetson
Export to TensorRT with FP16 is the whole game:
yolo export model=best.pt format=engine half=True- PyTorch on Orin Nano: ~7 FPS
- TensorRT FP16: 21 FPS at 640 px, ~11 W total board power
The detector feeds a simple tracker (ByteTrack) and publishes obstacle bearings to the autopilot over MAVLink OBSTACLE_DISTANCE messages. The autopilot does the actual avoiding — the neural net is only allowed to be an advisor, never a captain.
Failure log
- Rolling horizon confused early models into detecting “vessels” in wave crests. Fix: IMU-driven horizon crop before inference.
- Lens fouling: after 40 min, salt film cut confidence ~20% across all classes. Fix in progress: hydrophobic coating + a wiper we will absolutely over-engineer.
- Low sun: below 15° sun elevation, swimmer recall drops badly. Operational rule for now: no autonomous transits at dawn/dusk near bathers.
Next
Thermal camera fusion for the dawn/dusk gap, and self-supervised pretraining on the 200k unlabeled frames sitting on the NAS.