Causality Modeling For Concrete Action Verbs

Supported by National Science Foundation (2016 – Present)

Motivations and Objectives

This research project develops novel causality models for concrete action verbs to capture the intended change of state of the physical world.

It augments meanings of concrete verbs based on how they might change the environment (i.e., causality) and meanings of concrete nouns based on how they might be changed by actions (i.e., affordance). It incorporates causality models into learning and inference algorithms for grounding language to the physical world.

This work will provide a new dimension to connect verb semantics to perception and action. Verb causality models will allow the robot to predict a potential change of state from human linguistic utterances. This prediction will provide top-down information to guide visual processing and action modeling.

Related Papers

• Q. Gao, S. Yang, J. Y. Chai, and L. Vanderwende. What Action Causes This? Towards Naive Physical Action-Effect Prediction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), 2018. [Data]
• Q. Gao, M. Doering, S. Yang, and J. Y. Chai. Physical Causality of Action Verbs in Grounded Language Understanding. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016. [Data]
• L. She and J. Y. Chai. Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016.
• S. Yang, Q. Gao, C. Liu, C. Xiong, S. Zhu, and J. Y. Chai. Grounded Semantic Role Labeling. Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), San Diego, CA, June 12-17, 2016. [Data]