This program focuses on integrating multiple data streams (video, audio, sensor data) to build intelligent systems that understand complex environments. We explore how fusing visual, auditory, and spatial data enhances decision-making in autonomous systems.
Key Topics Covered
- Video Analytics: Action recognition, anomaly detection, and temporal modeling.
- Audio-Visual Fusion: Combining visual and audio signals for robust perception.
- Sensor Integration: Fusing LiDAR, radar, and camera data for 360° awareness.
- Real-Time Processing: Optimizing multi-modal pipelines for low-latency inference.
Learning Outcomes
- Design multi-modal perception systems for drones and robots.
- Implement cross-modal attention mechanisms for enhanced understanding.
- Build real-time video intelligence systems for surveillance and autonomous navigation.
Who Should Join?
- Researchers working on perception systems for autonomous vehicles and drones.
- Engineers in surveillance, robotics, and smart cities.
- AI practitioners interested in multi-modal learning and sensor fusion.
Format
- Seminars: Deep dives into CLIP, AudioSet, and multi-modal transformers.
- Lab Sessions: Build a multi-modal video surveillance system.
- Projects: Develop a cross-modal retrieval system or sensor-fused drone controller.