Guest post by Mila Vasylieva, LegalTech Intern Maigon.io
Just several months ago, we wrote about chatbot Meena from Google, and now Facebook presented one of their own – BlenderBot. Currently, it is the largest open-domain chatbot.
BlenderBot focuses on communication and conversational skills, such as personality, empathy and knowledge, and their combination. The chatbot is build using large-scale neural models with 9.4 billion parameters and by implementing techniques for detailed generation and blending skills. The neural networks were trained on 1.5 billion publicly available conversation examples. The neural networks are so big that they had to be divided into smaller ones using techniques like a column-wise model. In this way, it allows the efficient functioning of a bigger network than ever before with the processing of the enormous data sets.
While learning is essential, it is not everything, and Facebook recently introduced a task named Blended Skill Talk. It is based on the previous research and includes such skills as personality, knowledge, empathy, and the ability to mix them. With appropriate settings, this model significantly improves the evaluation of the conversational ability of a bot.
To ensure that bot does not repeat itself or demonstrate other flaws, the developers implemented various generation strategies. These include next token sampling, beam search, and n-gram blocking and allows to improve the qualities of the answers and human evaluation results.
The BlenderBot was also compared to Google’s Meena chatbot, where human evaluators had to go through series of dialogues between humans and chatbots and decide with whom they would prefer to have a long conversation and which one of the two chatbots sounded more human. The results showed that 75% of respondents preferred BlenderBot for a long discussion, and 67% perceived BlenderBot as more human.
Overall, the performance of Facebook chatbots doubled over the past several years. As they are continually working on it, improvements include combining various strategies and techniques, as was described with the release of a new chatbot and also Specificity Control, Poly-Encoders, etc.
The work will continue as they are looking for new evaluation methods that will help to expose weaknesses. Some problems are already identified, like contradictions, repetition of other mistakes, which may be not as apparent in short conversations, but will be more noticeable in a longer one. The other area of concern is how to deal with harmful language, and a possible solution may be building stronger classifiers.
Reproducibility is an integral part of technical development, and BlenderBot was made publicly available to further build upon this work and to move the conversational AI forward.
The original paper from Facebook can be found here.
As Facebook noted itself, even the most advance chatbots like BlenderBot, still have a long way ahead to achieve human-level intelligence in the conversations. However, even at this stage, chatbots are being actively used by various companies, and service providers, including the legal ones, and advancements are more than welcome.