Testing human teammates in Minecraft
ASU researchers are developing artificial intelligence that can communicate with humans within Minecraft
A fire is sweeping through a small town putting people’s homes and lives at risk. The response team arrives and has to operate as a unit to make quick, life-saving decisions, but what happens when one member of the team is not a person, but a virtual agent? How does that affect team performance and trust? And how can such a team best train and prepare for such scenarios?
A team of Arizona State University researchers is answering those and other questions using a popular video game — Minecraft. The Center for Human, Artificial Intelligence, and Robot Teaming (CHART), part of the Global Security Initiative, built an entirely virtual training environment in Minecraft as part of an approximate $3 million award from the Defense Advanced Research Projects Agency (DARPA). The goal of the project is to improve the social intelligence of artificial intelligence and make it better able to assist teams of humans working in complex environments, including in national security missions.
“AI should not replicate humans — it should complement them, each serving interdependent roles,” said Nancy Cooke, director of CHART and the principal investigator on ASU’s portion of the project. “A functional teammate — whether a human or an agent powered by artificial intelligence — understands his or her or its strengths and will have your back when needed.”
ASU is teaming with Aptima to evaluate hypothesis and AI agents, developed by over 200 performers on the Artificial Social Intelligence for Successful Teams (ASIST) program, including representatives from the Massachusetts Institute of Technology, Cornell, the University of Southern California and Carnegie Mellon University. The program is one component of a broader DARPA initiative called the "AI Next" campaign aimed at developing the next generation of AI machines that are more than just tools that execute human-programmed rules, but that function more as colleagues.
“(The ASIST) program will develop foundational AI theory and systems that demonstrate the basic machine social skills needed to infer the goals and beliefs of human partners, predict what they will need, and offer context-aware interventions in order to act as adaptable and resilient AI teammates,” said Joshua Elliott, DARPA program manager.
Rescued victim in Minecraft test bed, image by Christopher Corral
While the project is sponsored by the Department of Defense, the implications for the research go beyond national security.
“Teams that incorporate AI and humans effectively will have an advantage on the battlefield and business,” said Jared Freeman, chief scientist and principal investigator for Aptima. “Our team will help DARPA create and assess socially competent AI by applying our expertise in AI, human behavior, experimentation, and measurement.”
CHART researchers recently pilot tested the Minecraft test bed at a virtual hackathon in June, bringing together more than 100 performers from over 20 organizations. Due to COVID-19, the team pivoted from a planned in-person event to an entirely virtual exercise. Hackathon attendees received data from the ASU research team and reported on the progress that was made through the week either in building agents or testing hypotheses regarding social intelligence.
“AI has its own capabilities and limitations, just as humans do,” Cooke said. “The DARPA ASIST project is about imbuing AI agents with social intelligence so that they understand the intentions of their human teammates and can step in and provide appropriate assistance when needed.”
ASU students connected with the researchers and collaboratively combined forces to conduct experiments within the Minecraft test bed.
“Using Minecraft for the simulated task environment allows us to design complex and highly dynamic tasks,” PhD student Verica Buchanan said. “Working with the researchers is a collaborative effort and building working relationships is very important.”
“Watching the researchers transform the experiment data into stunning and useful data stories has expanded my perspective,” master's degree student Anagha Mudigonda said. “Connecting virtually across various time zones is often challenging and yet this experience enhances our team spirit and collaboration during COVID-19.”
Top image: ASU mascot Sparky rendered in Minecraft.
More Science and technology
Making magic happen: Engineering and designing theme parks
The themed entertainment industry is widespread and diverse, encompassing everything from theme parks to aquariums, zoos, water parks, museums and more. The Theme Park Engineering and Design…
AI-equipped feeders allow ASU Online students to study bird behavior remotely
ASU Online students are participating in a research opportunity that's for the birds — literally. Online Bird Buddies is a project that allows students to observe birds remotely, using bird feeders…
National Humanities Center renews partnership with Lincoln Center for responsible AI research
The National Humanities Center has announced that Arizona State University's Lincoln Center for Applied Ethics is one of four organizations to receive funding for the second phase of their…