It is a first-person multimodal open dataset based on Project Aria AR glasses records.
It contains 143 daily activity sequences recorded by multiple wearers in five geographic locations.
These records include multimodal sensor data recorded through Project Aria glasses, as well as machine sensing data.
These include high-frequency globally aligned 3D trajectories, field attraction clouds, 3D eye movement vectors per frame, and time-aligned voice transcriptions.
It provides rich multimodal sensing information and advanced machine sensing output to support AI and AR research.
Significance of this dataset:
1. The advancement of augmented reality (AR) and artificial intelligence (AI) research
The AEA dataset provides a rich collection of multimodal sensor data and machine sensory output, which is of great significance for the development of new augmented reality and artificial intelligence technologies. Using this data, researchers can explore and develop algorithms and applications that can understand and enhance human daily activities.
2. Innovation in personalized and contextualized AI assistants
Using the AEA dataset, you can promote the research and development of personalized and contextualized AI assistants. These assistants can better understand users ‘environment and intentions and provide services and support that are more tailored to personal needs, such as predicting and responding to users’ needs by analyzing users ‘eye movements, voice and environmental interactions.
3. The value of first-person perspective data
This dataset collects data from a first-person perspective, providing a unique perspective and depth for research. This perspective is crucial to understanding how humans interact with their surrounding environment and perform daily tasks, and can help researchers design more natural and intuitive interactive interfaces and experiences.
4. Opportunities for multimodal learning and perception
The AEA dataset provides a valuable resource for multimodal learning and perception by providing multiple types of data such as video, audio, 3D trajectory, and eye tracking. This data can be used to train machine learning models to more comprehensively understand and interpret human behavior and environmental contexts.
Through the research and utilization of AEA datasets, the development of practical applications and products can be promoted, especially in the fields of augmented reality, virtual reality and smart wearable devices. Advances in these technologies will ultimately have a profound impact on the way people work, learn and play.
Data set download:http://projectaria.com/datasets/aea/
Thesis:https://arxiv.org/pdf/2402.13349.pdf
Tools:http://ithub.com/facebookresearch/projectaria_tools
Video: