Best Practice: Humanification of Test Sets

Humanification in Botium stands for simulation of human behaviour or habits. Read on to know more.

Why Humanification ?

BotiumScript makes it is easy to verify the chatbot’s ability to follow a conversation flow. It is an important step to recognize the need for automation in this area. But in real world, you cannot expect human users to act like a computer script:

  • typographic errors are introduced

  • different typing speeds

  • sausage finger syndrom

With Botiums Humanification layer it is possible to evaluate how your chatbot deals with typical human typing habits.

Humanification is an additional layer on your existing test cases in Botium Box. Our recommendation is to use your own, clean test cases for automated testing, and from time to time use the humanification layer to verify your chatbots ability to handle those common input situations.

Prerequisites

  • Botium Box Enterprise Plan

  • Clean Test Set

Instructions

Step by step

Add Humanification Layer to Test Set

  1. Open the detail view of your test set to apply the humanification

  2. Go to the Transformation Wizard an open the Humanification Transformer

     

  3. Select from the list of Humanification Algorithms you want to apply (see below for details)

     

  4. You can set a percentage of how many of the utterances will undergo the humanification.

  5. Click on Download Test Set and Apply Humanification to clone the test set and apply the selected humanification algorithm(s).

All the humanification algorithms will only be applied to utterances, not convo files.

Evaluate cloned test set

The test set has been cloned to a shared folder in your default folder location. It can be used as any other test set now. If you want to apply additional humanification, you will have to do the clone process again.

Example

Here is an example for a clean utterances list:

And here how it looks like with the humanification layer:

Available Algorithms

  • Case sensitivity

  • Duplicate space

  • Emojis

  • Enter instead of space (experimental)

  • Missing punctuation mark

  • Double character

  • Homophones

  • Mishit

  • Missing character

  • Mixing character

  • qwertz vs qwerty keyboard

  • Sausage fingers

  • Shift characters

Other Humanification Options

Botium Box provides additional humanification options to make your test cases appear human.

Delay user input with PAUSE

PAUSE is one of the integrated logic hooks, which will just wait for a defined amount of time. See here for details.

1 2 3 4 5 6 Call Me Captain A simple Test Case #me hello PAUSE 5000

When using this, your test cases will run in real time and will be significantly slower.

Simulate human writing speed

The capability SIMULATE_WRITING_SPEED simulates human typing speed: the longer the text to send to the chatbot, the more time it will take - just as if a human is typing.

Average typing speed is about 290 ms/keystroke