There are some ways to create a dialogue system: rule-based, using machine learning algorithms, or using special platforms. In this stage, we consider the rule-based dialogue systems: the most understandable way to teach the computer to communicate with human beings.
Rules and intent
Rule-based dialogue systems, as you can tell from the name, work on rules. The rules mean the instructions on what the system should do in the conversation. The rules are written by the developers as code in a language: Python, Java, C++, Ruby, PHP, and so on.
In the text rule-based systems, the hardest part of development is intent processing. Intent is the intention of the user, which they express to the chatbot in order to perform some action or find the necessary information. How well the chatbot works and how much relevant information it will provide to the user depends on the correct processing of the intent. Intent processing is a special area in NLP, which is called NLU (Natural Language Understanding).
When developing a dialogue system based on rules, it is very important to pay attention to how the dialogue will go, since user requests can vary greatly, and it is quite difficult to predict all possible options. In order for the dialogue system to understand users well, it is necessary to carefully develop the categories of questions and their textual variations.
So, how cab a chatbot understand the information in intent? The main ways to process the intent are keyword search and pattern processing. Let's discuss it in more detail below!
Pattern-processing-based chatbots
Pattern processing is similar to parsing a natural language sentence. The first step in this approach is the tokenization of the text that was received from the user. Next, your chatbot system should compare the received sequence of tokens with the patterns that a developer created for the chatbot. For example, patterns can be such constructions as I *some_token* you, ticket to *some_city_token* at *some_time_token*, and so on. Patterns are created by the developer independently, depending on the tasks that the chatbot will have to perform.
When the chatbot finds a pattern suitable for the received text, it must choose an answer for this pattern from the answers database or generate text according to some pattern. For example, you're talking to a chatbot, and it determined that the text you entered matches the pattern ticket to *some_city_token* at *some_time_token*. After that, the chatbot gets two important entities: city and date, and can search for a plane ticket to this city at the specified time and show the results to the user.
One of the most famous rules-based chatbots is ELIZA, created in 1966. ELIZA is a chatbot in the field of psychology that simulates the work of a psychologist. ELIZA generated text based on pattern parsing. For example, if the user text fitted the I am *some_token* pattern, ELIZA could generate the following response: Why are you *some_token*?
Another famous rule-based dialogue system is ALICE. (Artificial Linguistic Computer Entity). The ALICE program utilizes the artificial intelligence markup language (AIML), an XML schema that aids in defining conversation rules. AIML contains many patterns, categories, and templates to create a rule-based dialogue system. AIML interpreters can be integrated into various programming languages, including Java, Ruby, Python, C++, C#, Pascal, and more.
Pattern-parsing chatbots are a good solution to rule-based chatbots that can be used for chit-chat. The disadvantage of such chatbots is the difficulty of developing sufficiently diverse patterns.
Keyword-recognition-based chatbots
Imagine that you need to create a rule-based chatbot for a bank. In this case, you will need to group all available information about banking services into the groups "cards", "deposits", "loans", "exchange rates", etc. But how in this case to process the intent?
The best solution, in this case, would be a keyword-recognition-based chatbot. As you might guess from the name, such a chatbot will search for keywords in the user's text and, based on them, provide relevant information to the user.
Suppose the user enters the following text: "I want to buy a house on loan". The first step in intent processing is tokenization. Then the algorithm should find keywords among the tokens (in our case, these are "loan" and "house", which match with the "real estate loans" group). After the intent group is defined, the chatbot should give the user information about real estate loans. To create this type of chatbot, the developer should design all possible groups where the intent can go, as well as collect all possible keywords for these groups.
One of the keyword-recognition-based chatbots is IBM Watson Assistant. IBM Watson Assistant empowers developers to define intents and entities, representing keywords or phrases that identify user intentions and extract pertinent information. Through training the assistant with these defined elements, it gains the ability to discern and appropriately respond to user inputs.
This type of chatbot is quite convenient for providing any information, such as FAQ information and customer support, and performing tasks (for example, setting an alarm, timer, etc.). The complexity of developing such a chatbot is the need to collect all possible keywords and synonyms for them.
Advantages and disadvantages of rule-based chatbots
The main advantage of rule-based chatbots compared to chatbots based on machine learning is the lack of the need to have a large amount of data. This is a rather important advantage in areas where there is no data or its amount is insufficient. For example, when building a chatbot that performs simple actions like "turn on/off the flashlight, set an alarm, show the weather forecast", you may find that there is no text corpus that is suitable for this task. In some cases, creating a rule-based chatbot is faster and easier than a machine learning-based chatbot.
The main disadvantage of rule-based chatbots is the complexity of their design, since the developer needs to carefully consider possible conversation scenarios, patterns, or keywords and manually code these rules. Another disadvantage of rule-based systems is that they are quite sensitive to spelling. If a keyword or a word from the pattern is a misspelled, the rule-based system can't find the correct pattern or fit the intent to the right group by keyword.
Conclusion
In this topic, you have learned about:
- the features, pros, and cons of creating rule-based chatbots;
- the intent and how to parse it using patterns and keyword recognition approaches;
- the pattern-processing-based and keyword-recognition-based chatbots, methods of their creation, as well as their advantages and disadvantages.