We said it before, and we’ll say it again: error handling is a crucial element of the conversational UX for chatbot. In a previous post, we identified two primordial characteristics of good error handling. Firstly, it should be contextual, by avoiding generic messages (like “I’m sorry, I didn’t understand that”) and ensuring that error prompts are always relevant in the context of the dialogue. Secondly, it should be progressive, which consists of giving different error messages if the bot doesn’t recognize the user query multiple times in a row, each time escalating towards more exhaustive answers. Ideally, your error handling should be both contextual and progressive, but it’s easier said than done, as these kinds of behavior demand design considerations that can have profound repercussions on how a bot is implemented.

Error handling in vanilla Dialogflow: follow-up intents and fallbacks

Dialogflow uses fallback intents when it fails to associate a user input with an intent. This, coupled with the follow-up intent mechanism, allows for contextual error handling. For example, suppose a very simple agent containing these intents :

DoTheThing is a normal intent that will be triggered by specific user inputs (in this case, a training phrase for this intent could be, for example, “I want to do the thing”). Intents can also be triggered by events, but it’s not really relevant to the current discussion.

The DefaultFallbackIntent is created automatically when you generate a new Dialogflow agent, and is triggered when Dialogflow can’t match the user input with anything else. You can think of the DefaultFallbackIntent as a safety net that will catch everything, but only as a last resort. Custom fallback intents can also be created, as we’ll see.

In Dialogflow, it’s possible to declare intents as follow-up to other intents. To continue with our same example, let’s suppose that the agent ask the user if he’s sure that he wants to do the thing when DoTheThing is matched:

DoTheThink-yes and DoTheThing-no are follow-up intents: they can only be matched if DoTheThing was previously matched. The way Dialogflow determines that is by keeping track of contexts. Specifically, when DoTheThing is matched, a new context, DoTheThing-followup, is generated. This context is then considered as one of the conditions to trigger DoTheThing-yes or DoTheThing-no, as we can see in this screenshot:

Finally, there is the follow-up fallback, DoTheThing-fallback. This kind of fallback works the same way as the DefaultFallbackIntent, but will only be operational when a specific context is active (DoTheThing-followup), like with any follow-up intent.

This allows a Dialogflow agent to have contextual error messages for virtually any situation: simply create follow-up fallbacks for every point in your dialogue where the user could say something that is not supported by your agent (pro-tip: most of the time, it’s all the time). You can even implement progressive error handling with this, since follow-up fallback intents can be daisy-chained to ensure proper escalation of error prompts:

Here, if a user triggers DoTheThing and then says something that is not recognized, DoTheThing-fallback will handle the situation. If the user then says something that is contextually correct (something that would match DoTheThing-yes or DoTheThing-no), the dialogue will continue normally. If they say another unsupported input, the agent will use the next follow-up fallback, DoTheThing-fallback-fallback.

The problem with slot filling error handling

So all is good, right? After all, we just demonstrated that you can do anything you want with follow-ups and fallbacks. Well, almost anything. Most notably, this approach will not work with slot filling. But before we tell you why and what can be done about it, we need to take a closer look at how slot filling is handled in Dialogflow.

First of all, what is slot filling? It is a simple conversational pattern where a bot is given a list of information to obtain (or slots to fill). Usually, each of these slots will correspond to an entity that the chatbot will try to extract from the user’s input. When a slot is considered filled, the bot will simply move to the next slot, until they’re all filled.

In Dialogflow, slot filling is tied to a given intent. In other words, a user will first need to trigger a specific intent for the agent to start the slot filling. One important detail to note is that the agent will stay on the same intent for the duration of the slot filling.

Let’s see how it works with this image taken from Dialogflow’s documentation:

The agent is given a list of three slots to fill : number-int, color and size, each of a different entity type. When the user triggers the intent that contains the slot filling, the agent will ask the user for each slot, in order. By default, the question is in the form “What is SLOT_NAME?”, but customized prompts can be defined for each slot.

What constitutes an error in slot filling is when the agent is not able to extract the expected entity from a user input. In this situation, by default, the Dialogflow agent will simply repeat the question until it receives a valid answer. Forever.

Error handling in slot filling is problematic because we can’t use the strategies that work in other circumstances (follow-up intents and fallbacks), since slot filling always happens within a single intent, and these strategies are built on top of the intent detection mechanism. It results in a very basic error handling for slot filling, and while it at least provides the possibility of continuing the dialogue, there is simply no error message guiding the user, which is not really interesting in terms of conversational UX.

Let’s try to fix that.

Detecting slot filling error context

(Please note that for the purpose of this article, we’ll suppose that the fulfillment is deployed in a serverless execution environment such as Google Cloud Function, which must remain stateless. Other approaches are of course possible.)

The first step to implement error handling in slot filling is to properly configure your agent to use your fulfillment webhook, and to activate calls to your fulfillment for the slot filling in your intent:

That was easy enough. The next step is to make your fulfillment code detect occurrences of errors in the slot-filling. The problem is that Dialogflow doesn’t communicate that directly. The only information that is clearly exposed in the fulfillment is the detected intent (which doesn’t change for the duration of the slot filling). But you can also extract from the contexts the current slot that the dialogue is trying to fill (more on that later). Even if there is no mention of any failed entity extraction, we already have everything we need to implement slot filling error detection.

The idea is to use contexts to leave a trace of every time slot filling is attempted for any given slot. Here are the main steps for slot filling error detection:

1. Detect if you are in slot filling
2. If so, construct an error ID from the intent’s name and the current slot’s name
3. Check if a context with that name already exists:
   1. If it doesn’t, create it and continue as normal
   2. If it does, you just detected an error

How does this work? Well, by making sure to create a custom context every time your agent tries to fill a slot, you leave evidence of that particular attempt. If the slot is properly filled, the agent will move to the next slot, and your fulfillment will generate another error ID (different from the one from before, since the slot is different). But if a call to your fulfillment is made with your custom context already created, it means that the same slot is trying to be filled again, which means that the user didn’t respond correctly to the slot query. In other words, you just detected an error in slot filling!

Let’s see how each step can be implemented:

One easy way to validate that you are indeed in slot filling is to use the WebhookClient and look in client.contexts for a context (automatically generated by Dialogflow) whose name starts with `${INTENT}_dialog_params_`, where INTENT is the name of the current intent, in lower cases. If the context doesn’t exist, you are not in slot filling. If it does, you are, and you can extract the name of the current slot from the end of that context’s name (immediately after “dialog_params_”).

And now, to create the error ID: since we can find the name of the intent and the name of the current slot, we can use this information to forge a unique error ID for each slot in the dialogue. Doesn’t have to be fancy, something like `error_${INTENT}_${SLOT}` will do the trick.

To check if our custom context already exists, we can use the following function from the WebhookClient:

client.getContext(errorID);

That’s pretty much self explanatory. As for context creation:

client.setContext({
name: `error_${INTENT}_${SLOT}`,
lifespan: 1
});

Here, we set the lifespan of the custom error context to 1, to make it disappear as soon as possible when it’s not needed anymore.

And that’s it for slot filling context error detection. From there, you already have most of the things you need to implement actual error handling: you just have to override the original bot response with a contextually appropriate message, which can be done with this simple command:

client.add(message);

Of course, you now need a way to retrieve or generate these contextually correct messages, from within the fulfillment. In other words, you need to do that programmatically. A good way to implement that while keeping your code clean and language agnostic is to leverage the error ID you created as a key and use a localization library to consolidate all your error messages. From there, it’s a simple matter of calling the localization library with the error ID to have it resolved to a proper error message.

Going progressive

Now that we have implemented contextual error handling for the slot filling, let’s improve it by making it progressive. An easy way to do that is to add a count parameter to your custom context:

client.setContext({
name: `error_${INTENT}_${SLOT}`,
lifespan: 1
parameters: { count: 1 }
});

From there, if an error occurs, you can append the count to the error ID (`${errorID}_${COUNT}`) and use that new ID template to identify your error messages in your localization library (or whatever similar strategy you use) and thus have any number of distinct error messages for any given context. Simply don’t forget to increment the count value in the context each time you detect an error by calling setContext again. You may also want to implement a cap to that number, to avoid having a user reaching an error message that was not implemented.

Next steps

This concludes our short discussion on slot filling error handling in Dialogflow. Of course, the proposed implementation is really bare bone and doesn’t support really useful functionalities like constraints, disambiguation (“Did you mean A or B?”) or certain particular situations (things could be a little more complex if you decided to activate the Knowledge Base feature, for example). These issues will most probably be covered in one of our future blog posts about Dialogflow. Thanks for reading!