Challenges in testing ChatBots


        This is the second article on ChatBot Testing. Here is the first article which introduces the chatbots in general and details out why should we test chatbots.

        Chatbots are different to other applications in several ways as listed below which makes them challenging to test and measure the quality.

Self Learning Systems

        Chatbots are built using AI and NLP services and keep learning and improving on their own. This makes it highly unpredictable whether a test case has failed due to the defect or if its improvement. One cannot have the traditional style of testing and write test cases with expected output. Remember, the test data used itself can be used by the bot to learn and update its behaviour, the negative testing may have an impact on the bot !!

Non-linear input

         Voice based chatbots use the speech recognition technologies to convert the voice to text. Chatbots are heavily dependent one such solutions. Along with the dependency comes testing the accent, type voices, speech clarity and speech with background noises(say a song playing in the background or a TV program running in the background J). How does the bot handle such complicated inputs.

 

Error handling

         Ability to test and determine what is out of context and how should an unexpected user input should be handled. This would involve the domain knowledge and the context testing together. For example: Architecture in construction based business would mean different than an IT business.

 

Endless conversational patterns

        Due to the nature of human language it is impossible for software tests to cover all possible situations. This may include the  native jargons used, spelling mistakes, out of context to what bot is designed for and switching between the contexts in the conversations.

 

 

 




Comments

Post a Comment

Popular posts from this blog

Cross Browser testing for React Apps

Before we discuss the need of cross browser testing for React apps lets get a quick essence of why do people use react to develop their apps and what is cross browser testing. React React and React Native , support all the major browsers across platforms ability. It means that if one would want to build an application for iOS and android platform, all they would do is have a single code base and rest is taken care by react to make it cross platform. In addition to it the support for all the major browsers and has been tested, however with a little limitations for older browser where the polyfills are used to support them. Cross browser Testing Cross browser testing helps you validate if your application is compatible across different browsers, platforms and devices. It becomes important today to support all the major browsers and the devices as it is almost impractical to control what platform or browsers the users would use to access your application i...
Read more

Fixing the appium installation with NPM

Problem Statement: npm install -g appium returns the below     npm install -g appium  npm ERR! code ENOENT  npm ERR! syscall chmod  npm ERR! path /usr/local/lib/node_modules/appium/build/lib/main.js  npm ERR! errno -2 Solution The directory npm is trying to modify is not owned by the user that is executing the command. Grant the permissions to the user executing the command and the issue should be resolved.  Here are the commands:  sudo chown -R `whoami` ~/.npm sudo chown -R `whoami` /usr/local/lib/node_module Uninstall the appium npm uninstall -g appium Re-install appium npm install -g appium
Read more

Shell Scripting

Your first Shell script!! Open the terminal type the below command: this is to create a file vi helloworld.sh hit escape button (to enter the insert mode in vi editor) type the below echo "helloworld..!!" to save the script hit escape and colon (:)type wq! (like this :wq!)and hit enter now to execute the program use the below command . ./helloworld.sh hit enter output is as below helloworld..!! Printing on screen Example:1 This statement prints the statement as is on screen echo Hello, how are you doing? output: Hello, how are you doing? Example:2 Printing the value of any variable a=10 echo $a output: 10 Reading the user input the below shell script reads 2 inputs from the user and prints it on the screen. echo "Enter the name:" read name echo"Enter last name:" read lname echo "hello $name $lname, how are you?? " Operators in Shell Mathematical [x -eq y] =...
Read more