X-Capture: An Open-Source Portable Device for Multi-Sensory Learning

Understanding objects through multiple sensory modalities is fundamental to human perception, enabling crosssensory integration and richer comprehension. For AI and robotic systems to replicate this ability, access to diverse, high-quality multi-sensory data is critical. Existing datasets are often limited by their focus on controlled environments, simulated objects, or restricted modality pairings. We introduce X-Capture, an open-source, portable, and costeffective device for real-world multi-sensory data collection, capable of capturing correlated RGBD images, tactile readings, and impact audio. With a build cost under $1,000, X-Capture democratizes the creation of multisensory datasets, requiring only consumer-grade tools for assembly. Using X-Capture, we curate a sample dataset of 3,600 total points on 600 everyday objects from diverse, real-world environments, offering both richness and variety1. Our experiments demonstrate the value of both the quantity and the sensory breadth of our data for both pretraining and fine-tuning multi-modal representations for object-centric tasks such as cross-sensory retrieval and reconstruction. X-Capture lays the groundwork for advancing human-like sensory representations in AI, emphasizing scalability, accessibility, and real-world applicability.

Paper

Similar papers

© 2026 NYSGPT2525 LLC