01/ Problem
Problem
Get a small drone to follow a target and avoid obstacles in real time, on embedded compute, using only stereo perception.
← Back to projects
0042024
StereoSight
Real-time stereo planner on VOXL 2 + PX4.
- C++
- Python
- ROS 2
- OpenCV
- Dijkstra
- DBSCAN
- TFLite YOLO v11n
02/ Approach
Approach
Convert sensor_msgs/PointCloud2 into 2D occupancy grids via downsampling, DBSCAN, 2D histogram binning, median filtering, and morphological dilation. Plan with Dijkstra at 10 Hz, replan when waypoints get blocked. Run YOLO v8n in sim, on-device TFLite YOLO v11n on the VOXL 2 via voxl_msgs/Aidetection.
03/ Hardware
Hardware
ModalAI VOXL 2 + Sentinel drone with PX4 autopilot. Adaptive downsampling kicks in above 50k points to hold real-time.
04/ Result
Result
Both indoor sim and on-device inference flight tests demonstrated reliable target-following with obstacle avoidance. Heartbeats at 20 Hz, trajectory setpoints at 10 Hz.