Cutting-Edge Multi-Camera System for Multi-Modal Data Capture

Researchers from the Intelligent Systems department have built an advanced multi-camera capture system designed for high-quality data collection from objects and people. By incorporating existing audio devices, this system enables the recording of multi-modal data from human social interactions including both video and audio, offering invaluable resources for understanding human behavior, modeling physical objects, and analyzing human-object interactions. 

High-quality data is essential for multi-modal data processing, interpretation, interaction, visualization, and the development of deep learning methods. The newly built multi-camera capture system is equipped with 12 fully synchronized high-resolution cameras, providing frame-level synchronization, which is crucial for capturing fast-moving objects and precise human movements. The integration of an audio system enhances its ability to capture multi-modal data, making it a central tool for various research endeavors. 

The complete capture system will serve as a communal hub, supporting the department in conducting cutting-edge research on instrumentation, data processing, and behavior understanding. It will also promote collaboration among different research groups and facilitate coursework, allowing students to gather their own data for class projects. The system will be instrumental in addressing key research challenges within the Intelligent Systems department, such as 3D body gesture and posture estimation, dynamic object modeling, and multi-person interaction analysis. 

Led by Assistant Professor Xucong Zhang and largely implemented by PhD student Xiangwei Shi of the Pattern Recognition and Bioinformatics section, this project is supported by the Intelligent Systems Department and the Research and Engineering Team (REIT). While several institutes are also developing similar multi-view, multi-modal capture systems, which are regarded as highly valuable assets for many research tasks, our goal is to establish one of the top three capture systems in Europe. To achieve this goal, we anticipate continued contributions from our stakeholders of INSYGHT Lab toward the capture systems.