Premise

Chatbots are becoming ubiquitous, but tuning chatbots for natural-language conversations with human users is fraught with complexities. Conversational deficiencies not addressed in development and training stages can undermine user acceptance of chatbots.

Analysis

Chatbots are beginning to inform the patterns of our lives and even taking center stage in customer engagements. Typically, users encounter chatbots through interactive, customer-service dialogue boxes that seem to pop up magically (or annoyingly, depending on your point of view) when within webpages, mobile apps, or even integrated voice response.

Tuning a chatbot involves ensuring that it delivers to users a satisfying conversational experience. If developers aren’t careful, they may build many types of unintended conversational deficiencies into the apps. For example, lack of well-trained natural language processing (NLP) algorithms may cause chatbots to consistently misconstrue what users are requesting or intending. To ensure chatbots produce desired user outcomes, boost customer experiences, and  contribute to other application goals, developers should:

• Establish an ongoing program of chatbot training. Tap into your chatbot’s organic stream of training data. Chatbots support conversational, gestural, visual, auditory, and other interfaces that invite users to generate training data in normal operations. The user-generated data in these streams can be used to label the data so that chatbot algorithms can be dynamically retrained to improve their fitness to the designated learning task.
• Ground chatbot improvements in ongoing performance analytics. To pre-empt user churn, diagnose root causes of performance issues, and optimize chatbots to achieve desired outcomes, developers should refine these apps through a steady feed of insights surfaced by chatbot analytics tools.

Establish An Ongoing Program Of Chatbot Training

Training is the art of ensuring that data-driven algorithms are predictively fit for their core functions. Consequently, chatbots that incorporate artificial intelligence (AI), machine learning (ML), and NLP algorithms must be continually trained with fresh conversational data in order to ensure that they achieve the objectives and key performance indicators for which they’ve been designed. For example, when deployed as a customer engagement tool in e-commerce, a chatbot’s performance might be judged on whether it helps customers to find relevant products, accelerate transactions, and obtain speedy customer service, while also boosting retention, satisfaction, and referrals.

Once developers have established the relevant objectives for chatbots—in consultation with stakeholders such as brand and engagement managers—they can begin to identify the requisite resources needed to train and refine the bots’ machine-learning algorithms. Table 1 discusses the training options to explore if you’re optimizing chatbots for mobile, social, e-commerce, Internet of Things, and other data-solution domains.

Table 1: Chatbot Algorithmic Training Options

Ground Chatbot Improvements In Ongoing Performance Analytics

Chatbots are very much a work in progress. Even if they’re trained well to do perform their intended tasks, chatbots can easily disappoint, irritate, or frustrate users who expect a more seamless and productive conversational experience. The potential issues with chatbot performance are diverse. Table 2 presents the most common chatbot conversational deficiencies.

Table 2: Common Chatbot Conversational Deficiencies

To pre-empt these issues, diagnose their root causes, and optimize chatbots to achieve their performance objectives, developers should be building, training, and iterating chatbots with an eye on the performance metrics presented in Table 3.

Table 3: Chatbot Performance Metrics

To stay on top of these metrics, chatbot developers should leverage tools that provide the analytics necessary to harvest the insights presented in Table 4. 

Table 4: Chatbot Analytic-Driven Insights

Most chatbot development tools provide performance, usage, conversation, and engagement analytics to varying degrees. In addition, there are many stand-alone chatbot analytics tools on the market. These include BotAnalytics, Botimize, Botlytics, BotMetrics, ChatMetrics, ChatbotProxy, ChatUrl, Dashbot, DialogAnalytics, Dimon,  Opearlo Analytics, UxProbe,  VoiceLabs, Witlingo, and Wordhop.

Action Item

Chatbot developers should establish an ongoing program of algorithmic training in conjunction with A/B testing. The primary objective should be whether iterative algorithmic training and operational testing help developers both to reduce the incidence of chatbot conversational deficiencies. By steadily improving the naturalness of chatbots’ dialogues with users, developers can better ensure that these applications boost user retention, satisfaction, and productivity.