State of the art unimodal dialogue agents lack some core aspects of peer-to-peer communication—the nonverbal and visual cues that are a fundamental aspect of human interaction. To facilitate true peer-to-peer communication with a computer, we present Diana, a situated multimodal agent who exists in a mixed-reality environment with a human interlocutor, is situation- and context-aware, and responds to the human’s language, gesture, and affect to complete collaborative tasks.
  • Headshot of Jaime Ruiz wearing a HololensJaime Ruiz
  • As well as: Nikhil Krishnaswamy, Pradyumna Narayana, Rahul Bangar, Kyeongmin Rim, Dhruva Patil, David McNeely-White, Bruce Draper, Ross Beveridge, and James Pustejovsky

Nikhil Krishnaswamy, Pradyumna Narayana, Rahul Bangar, Kyeongmin Rim, Dhruva Patil, David McNeely-White, Jaime Ruiz, Bruce Draper, Ross Beveridge, and James Pustejovsky. 2020. Diana’s World: A Situated Multimodal Interactive Agent. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09: 13618–13619. https://doi.org/10.1609/aaai.v34i09.7096

@article{krishnaswamy_dianas_2020,
	title = {Diana's World: A Situated Multimodal Interactive Agent},
	volume = {34},
	issn = {2374-3468, 2159-5399},
	url = {https://aaai.org/ojs/index.php/AAAI/article/view/7096},
	doi = {10.1609/aaai.v34i09.7096},
	shorttitle = {Diana's World},
	abstract = {State of the art unimodal dialogue agents lack some core aspects of peer-to-peer communication—the nonverbal and visual cues that are a fundamental aspect of human interaction. To facilitate true peer-to-peer communication with a computer, we present Diana, a situated multimodal agent who exists in a mixed-reality environment with a human interlocutor, is situation- and context-aware, and responds to the human's language, gesture, and affect to complete collaborative tasks.},
	pages = {13618--13619},
	number = {9},
	journaltitle = {Proceedings of the {AAAI} Conference on Artificial Intelligence},
	shortjournal = {{AAAI}},
	author = {Krishnaswamy, Nikhil and Narayana, Pradyumna and Bangar, Rahul and Rim, Kyeongmin and Patil, Dhruva and {McNeely}-White, David and Ruiz, Jaime and Draper, Bruce and Beveridge, Ross and Pustejovsky, James},
	date = {2020-04-03},
	
}