hiw3c 日报精选
arXiv cs.AI/CL/LG 网页快照

多模态掩码扩散自动编码器用于建模受损驾驶行为

原文标题 · Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving

作者: John Gideon, Anshul Gupta, Megan Applegate-Kenton | 人类认知状态(如分心、压力)、注视与行为以微妙的方式与驾驶环境互动,这些互动可供AI用于辅助驾驶员,但针对人类认知状态级别的真实标签在大规模应用中难以获取。我们提出一种多模态掩码扩散自编码器,即ID-MAE,该模型在损伤预测中加入一个自监督目标:除了预测i

来源:arXiv cs.AI/CL/LG · arxiv.org

正文为英文,可一键机器翻译(仅首次需要等待)

Computer Science > Computer Vision and Pattern Recognition

Title: Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving

Abstract: Human cognitive state (e.g., distraction, stress), gaze, and behavior interact with the driving environment in subtle ways that AI could use to assist drivers, but ground-truth labels for human cognitive state levels are costly to obtain at scale. We propose a multimodal masked-diffusion autoencoder, ID-MAE, that adds a self-supervised objective to impairment prediction: alongside predicting the impairment label as a masked token, it reconstructs masked gaze and trajectory. To probe the effects of different modalities and types of impairments on model inference, we introduce DIVERT, a driving-simulator dataset containing 103 hours of driving data (n = 109) with ten impairment inductions spanning cognitive, visual, and emotional states, recorded with gaze, vehicle state, and scene video. We demonstrate how ID-MAE affords significant improvements in impairment inference results on both the DIVERT dataset and existing datasets, and enables strong results in additional tasks, such as trajectory and gaze prediction. For location-matched pairs, our model achieves a paired AUC of 0.706 for impairment detection on DIVERT, compared to 0.590 without reconstruction loss, improving over existing baselines. We probe the design space in terms of input modalities, masking scheme, positional encoding, attention mechanism, and decoding. We also investigate the interaction between various impairments in training and inference, demonstrating that visually demanding distractions are separable from cognitive distractions. Code and data will be available upon acceptance.

Submission history

Access Paper:

Additional Features

Current browse context:

References & Citations

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .