Advancing the field of conversational AI with the Alexa Prize

The Alexa Prize Socialbot Grand Challenge brings teams from across the world together to compete in developing an interactive socialbot, with the first-place winner receiving a research grant of $500,000.

Amazon challenges teams from universities across the globe each year to design socialbots that can hold engaging and coherent interactions with humans about a range of news and popular topics, from politics and technology to entertainment, fashion, and sports.

During the competition, Alexa customers engage with the teams' socialbots by saying, 'Alexa, let's chat.' The Socialbot Grand Challenge, and ultimate goal of the competition, is to earn a score of 4.0 or higher (out of 5) from competition judges. Judges must also determine whether at least two-thirds of the customer interactions with the socialbot are coherent and engaging for at least 20 minutes.

No team has reached the Socialbot Grand Challenge, but doing so carries a $1 million research grant for the team's university.

We recently sat down with the latest first-place winner, Jakub Konrád, a Czech Technical University (CTU) Ph.D. student and the team leader for team Alquist. Team Alquist had an average rating of 3.28 and an average finals interaction duration of 14 minutes and 14 seconds.

Learn about the competition and Konrád's experience as he reflects on his work, competing in the challenge for four straight years, and winning the challenge in August 2021. Previous challenge winners include teams from the University of Washington, the University of California, Davis, and Emory University.

Q. You've participated in each edition of the Socialbot Grand Challenge, and you're now the most recent winner. What's kept you coming back to participate?

We love working on conversational AI problems. The Socialbot Grand Challenge offers a great opportunity to reach real customers and see how they interact with our socialbot. Also, the great support we receive from Amazon makes the development easier, including access to the Alexa Skills Kit, Amazon Web Services, and other tools.

Even though COVID-19 has obviously had an impact, the competition is also a great opportunity to meet and get to know other researchers in the field of conversational AI and to exchange our knowledge and ideas with them.

Last but not least, of course, it was the desire to win the competition!

Q. What does this achievement mean for the team and, more broadly, Czech Technical University?

It's a confirmation that the team and our research are going in the right direction and the AI is ready to use in real-world scenarios. It's an amazing accomplishment for our team, which we hope opens many more opportunities.

CTU has managed to compete with other world-class universities in the four editions of the Socialbot Grand Challenge held to date, which gains an additional reputation for the university itself, hopefully opens up new research partnerships, and drives further interest from potential students.

Q. Was there a particular area of focus that vaulted you to the win in 2021?

The current version of the Alquist socialbot effectively combines a flexible generative approach and high-quality dialogue scenarios. That means we prepare a basic scenario for the bot about a particular topic, such as a movie that a customer saw recently, and as the interaction develops, the generative model prepares the additional conversation content on its own.

This strategy allows the bot to deliver an engaging interaction and react coherently to unexpected inputs from the Alexa customer. For example, one person asked Alquist, 'I went to see this movie with my kids and they didn't like it. Why do you think that is?' With neural response generation, our socialbot generates an example response such as, 'I don't know. It's strange because I thought it was a kids' movie.'

The generative model takes into account external facts about the discussed entities to make the interaction more natural. Such an entity can be the name of a movie like Avengers or a person's favorite artist, like Lady Gaga. Another important aspect is that the bot carries personal context throughout the conversation, and then constructs the following turns in the conversation considering such pieces of information.

Q. Was there anything that surprised you in this year's Socialbot Grand Challenge 4 versus prior Socialbot Grand Challenges?

Thanks to the generative models that have advanced dramatically in quality compared to the previous years of the competition, the interactions the socialbots achieved were much longer and more varied in the latest challenge, which was exciting to see. For example, in a few conversations we noticed that Alquist and the customer chatted about their respective partners, even though we never actually programmed anything like that into the system. That was a big surprise to everyone.

Q. Were there unique challenges you encountered during Socialbot Grand Challenge this year, that perhaps you didn't encounter in prior challenges?

The coronavirus pandemic greatly influenced this year's competition. While in previous years we had opportunities to meet other teams participating in the challenge, as well as the Amazon team, this year the challenge happened remotely for everyone. Moreover, our team at CTU was not able to meet in person for extended periods because of the pandemic, which led us to collaborate in different ways.

Q. What do you see as the biggest hurdle in meeting the Socialbot Grand Challenge?

It's challenging to lead a long, meaningful conversation with a person you have only just met. You have little to no information about your conversation partner, which is crucial for an engaging dialogue. Even with the current state of conversational AI technologies, it's difficult to successfully identify and extract the information which might be useful throughout an interaction. It usually requires a non-invasive set of questions about the user, such as, 'Did you have a fun weekend?' That leads the customers to talk more about themselves. We coupled this approach with the module which extracts the relevant information from the customer's response.

Q. What's the first thing you all did once you heard your team won?

Even though we had a lot of success in the three previous Socialbot Grand Challenge competitions, we were still surprised to actually reach first place in 2021. Historically, our bot was more successful with the general Alexa customers than the interactors participating in the finals process. It was exciting to see that this year the bot succeeded during the finals. The first thing we did was let our friends and families know so that we could celebrate together.

Q. What's next for the team?

The first thing we're planning is to take a short break and rest after finishing the challenge. After coming back, we'd like to read through all the papers submitted by our competitors. With a fresh mind after the break and newly gathered information, we believe that we'll see what worked and what didn't and proceed with designing the next iteration of our bot. Furthermore, we'd like to apply to the next Socialbot Grand Challenge and take another shot at the Socialbot Grand Challenge goal itself.

Q. What advice would you give universities and students considering participating in the Socialbot Grand Challenge and more broadly in the field of conversational AI?

Our first and foremost piece of advice would definitely be to join the Socialbot Grand Challenge. It's a fantastic opportunity to deliver your innovations to many users and see how they respond to and interact with your system. Furthermore, we think that conversational AI is an up-and-coming field where you can solve many exciting problems in areas such as well-being, mental health, and education.

Public link

Please use the above public link if you want to share this noodl on another website.