Created by Réka Gábosi for the dotLumen Interview Challenge

• Tools & Technologies
• Overview
• Motivation
• System Overview
• Technical Breakdown
  • Detection
  • Depth Processing
  • Time-to-Collision Logic
  • Visualization
• Example Outputs
• Challenges and Limitations
• File Structure
• How to Run the Project

• Python 3.10+ — main programming language
• OpenCV — for video processing, visualization, and frame extraction
• NumPy & Pandas — for numerical computation and logging
• YOLOv8 (Ultralytics) — object detection of people and vehicles
• Intel RealSense D455 / Depth Sensors — for depth estimation
• ffmpeg / cv2.VideoWriter — video export

SafeBubble is a real-time depth-aware perception system developed as part of the dotLumen interview challenge. It detects nearby people or vehicles, estimates their distance and motion, and visualizes a dynamic safety bubble that changes color when someone enters too close or approaches too fast.

The system integrates YOLOv8 for object detection and Intel RealSense D455 depth data for spatial awareness, providing instant visual feedback.

I created this project as part of the dotLumen interview challenge, to explore how computer vision and AI perception can enhance spatial awareness for wearable assistive technologies.

This was my first time working on a dual-video setup (color + depth), and I wanted to experiment with real-world video synchronization, object detection, and motion-based safety visualization.

Maintaining a safe distance from surrounding objects or people is critical — and with this project, I explored how AI can model a “comfort zone” that dynamically adapts to proximity and motion.

SafeBubble operates in five stages:

1. Object Detection: where YOLOv8 identifies people, cars and obstacles
2. Depth Sampling: it extracts median or the bottom depth for each detection box
3. Tracking & Velocity Estimation: estimates approach speed from frame history
4. TTC Computation: calculates Time-to-Collision (TTC) = distance / approach velocity
5. Alert Visualization: Projects colored “bubble” overlays representing the safety state.

• I have implemented it with YOLOv8
• Filters are based on confidence (DETECT_CONF = 0.35)
• Targeted classes are person, car, bicycle and motorbikes (DETECT_CLASSES = ('person', 'car', 'bicycle', 'motorbike'))

• Operates on float32 depth maps
• Invalid pixels are autmatically excluded
• It takes the depth values mostly from the bottom part of each detected object - to avoid background noise

• It predicts collision within 3 seconds and alerts the user
• Distance zones:
  • 🔴 DANGER_DIST ≤ 1.5 m
  • 🟠 CAUTION_DIST ≤ 3.0 m
  • 🟢 otherwise safe

• There is a safety bubble right in front of the user -
• The safety bubbles alert level and their colors:
  • 🟢 Safe
  • 🟠 Caution
  • 🔴 Danger
• Each object detected shows their distance and time-to-collision right on the screen, so it's easy to understand

State	Visualization	Description
🟢 Safe Zone		No threat detected
🟠 Caution Zone		Someone slowly entering personal space
🔴 Danger Zone		Risk detected; bub

• I kind-off had a lot of issues cause the depth video is 6 seconds shorter than the colored version. And it glitches out for few seconds here and there. My approach was to find at least a segment of the two where they are matching and try to make something out of that part. So I ended up cutting off the first 3 seconds of the colored video and only ran my code on the times between: 00:12 and 2:34. Cause right after that another glitched happened on the depth video.
• Its not always accurate, it still needs some fixing up, but I didnt had more time to spend on it and tweek around why some things aint correctly alerted.

├── README.md
├── requirements.txt
├── dotLumen_SW-Challenge2.docx
│
├── original_data/                        
│   ├── challenge_color_848x480.mp4
│   ├── challenge_depth_848x480.mp4
│   └── synced_color.mp4
│
├── src/
│   ├── main.py       
│   ├── detector.py               
│   ├── tracker.py                 
│   ├── depth_utils.py          
│   ├── visualize.py 
│   ├── extract_samples.py                 
│   ├── sync_videos.py          
│   ├── config.py 
│   ├── 
alerts.csv           
│   └── yolov8n.pt 
│               
└── outputs/
    ├── demo_out_segment.mp4               
    └── alerts.csv                 

Make sure you have Python 3.8+ installed,

pip install -r requirements.txt

python main.py \
    --color ../original_data/synced_color.mp4 \
    --depth ../original_data/challenge_depth_848x480.mp4 \
    --out ../outputs/demo_out_segment.mp4 \
    --view \
    --start-time 00:12 \
    --end-time 02:34 \
    --smooth-alpha 0.4

This project was created as part of the dotLumen Software Challenge 2 interview assignment.