Unit 13 - Chatbots - Python Text Analysis

Chatbots are old technology. The most famous classic chatbot is ELIZA, a system developed in the 1960s that answered questions based on templates. There are a few implementations of ELIZA out there, and you can test a version by Kenneth Ronkowitz.

ELIZA was built to mimick a particular type of therapist: someone who asks you further questions about what you are saying. The answers where like this:

Human: Well, my boyfriend made me come here.
Eliza: Your boyfriend made you come here
Human: He says I’m depressed much of the time.
Eliza: I am sorry to hear you are depressed
Human: It’s true. I am unhappy.
Eliza: Do you think coming here will help you not to be unhappy
Human: I need some help, that much seems certain
Eliza: What would it mean to you if you got some help

Text is from Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.

You can see that you can create the answers by using pattern matching, introducing things like “I am sorry you are X”, when X is in a pre-defined list of negative mental states.

Today, chatbots range from speech-to-speech systems like Siri, Alexa, Cortana and the like, to text-based chatbots that you may find on your bank’s website. Full conversational systems like Siri need to incorporate speech processing, language understanding, plus a way of getting answers to you (usually by searching the web). Customer service chatbots just need a language model to understand and generate language, plus a database that contains company information that the customer may need.

You can also build a chatbot that retrieves answers from a corpus, or one based on generative AI, which generates answers by encoding your question.

Pattern-matching chatbot¶

You can build a simple chatbot by storing a series of potential questions and answers. That is, “if the user says X, then answer Y”. We are going to try that here. We will use spaCy’s Matcher module to do that.

This particular implementation is based on Ahilya’s. It produces an extremely simple dialogue; just a greeting and only in response to “hello”. You can extend it with other patterns.

Import statements and load spaCy¶

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xd

Create the chatbot matching¶

The code has 2 parts:

Create a few patterns, such as “Greeting” = “hello”
Do a loop to get the input (parsing it with spaCy) and produce output

At any point, when you are done, type ‘quit’ to leave the dialogue.

# set up the matcher to recognize specific patterns
matcher = Matcher(nlp.vocab)
matcher.add("Greeting", [[{"LOWER": "hello"}]])

# define the chatbot's interaction loop
while True:
    message = input("You: ")
    if message.lower() == "quit": # type 'quit' to exit the prompt below
        break
    else:
        matches = matcher(nlp(message))
        if matches:
            print("Bot: Hello there!")
        else:
            print("Bot: I'm sorry, can you please rephrase?")

You: hello
Bot: Hello there!
You: hi
Bot: I'm sorry, can you please rephrase?
You: good morning!
Bot: I'm sorry, can you please rephrase?
You: quit

Add more rules¶

We are adding also “hi” to the list of Greetings, and giving a slightly different answer if “hi” is said, instead of “hello”. Then we add a new type of interaction, “Help_request”, where the system catches any instance of the word “help” in the prompt and answers. Finally, we add a default answer, where the bot doesn’t match anything, so that the user rephrases.

Play around with this and add as many types of rules and as many matches as you want.

matcher = Matcher(nlp.vocab)
matcher.add("Greeting", [[{"LOWER": "hello"}], [{"LOWER": "hi"}], [{"LOWER": "morning"}]])
matcher.add("Help_request", [[{"LOWER": "help"}]]) 

while True:
    message = input("You: ")
    if message.lower() == "quit": 
        break
    else:
        matches = matcher(nlp(message))
        if matches:
            if "hello" in message.lower():
                print("Bot: Hello there!")
            elif "hi" in message.lower():
                print("Bot: Hey! How's it going?")
            elif "morning" in message.lower():
                print("Bot: Good morning!")
            elif "help" in message.lower():
                print("Bot: What can I help you with?")
        else:
            print("Bot: I'm sorry, can you please rephrase?")

You: hello
Bot: Hello there!
You: hello
Bot: Hello there!
You: hi
Bot: Hey! How's it going?
You: hello
Bot: Hello there!
You: quit

Summary¶

We have learned about simple chatbots and started a pattern-matching chatbot. Check the Canvas page for more links to other types of chatbots.

References¶

Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. 10.1145/365153.365168