Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Click the Close button

  • Click Save to save the new chatbot record

  • You can now navigate to the record by using the filter in the Chatbots list and clicking on the record:

...

Step 2: Using the Live Chat to Create a First Test Set

...

That’s what the live chat in Botium Box is for - click the Live Chat button to open a chat session with your chatbot:

...

  • Enter hi in the input field at the bottom and click the send button (or Enter)

...

  • You can now see the chatbot responding to your greeting

  • Click on the Save as Test Case button to save the conversation you just had as test case

...

  • Let’s create a new test set by entering First Steps Test Case in the input field …

  • … and First Steps Test Case as Test Case Name

  • … and finally click on OK

...

  • Now navigate to the test set record we just created by using the Test Sets menu and clicking on the First Steps Test Set in the list - again, you can use the filter on the top right.

...

  • Click on the Test Cases section to inspect the test case we just created

...

  • You can see now a single test case, the one we saved before. You can use the test case designer for inspecting and changing the conversation steps.

...

  • Go to the Test Results menu and in the Start Test Session section, select our First Steps Chatbot and our First Steps Test Set, and click on Start Test Session.

...

  • You can now see Botium doing the hard work in the background, and after several seconds, the first test result is ready.

  • As expected, the test session succeeded

  • In the Test Cases section, you can see all the executed test cases - for our first test set, we only had one single test case.

...

  • Click on the test case to open the conversation transcript. This looks very similar to the live chat we had earlier - but the conversation happened without any manual intervention.

...

Info

Instead of using the main menu you can also use the Recently Visited list in the user menu.

...

  • Open the Test Cases section and click the Write New Utterances List button

...

  • Enter UTT_USER_HELLO as utterance list name

  • Enter the utterances hi, hello and good morning (use the + button to add an additional input field)

  • Click the Save button to save the utterance list

...

  • Click on the bubble with text hi in the First Steps Test Case to bring up the conversation step editor

...

  • Click on the Select From Utterance List button to bring up the selection box for the utterances lists

  • Click on UTT_USER_HELLO to use this utterance list as input for the test case

  • Close the conversation step editor

...

  • Note that the test case now has been changed: where before there was only the hi as input, now there is the UTT_USER_HELLO utterance list we created before

  • Save the test case and let’s run another test session the same way as above

    • Go to the Test Results menu and in the Start Test Session section, select our First Steps Chatbot and our First Steps Test Set, and click on Start Test Session.

...

  • See what happened now ? There are now 3 test cases, and 2 of them failed!

  • Click on Test Cases and expand one of the two failing test cases

...

  • You can see here the reason the test case failed:

    • We are sending the text hello to the chatbot (one of the possible greetings in our UTT_USER_HELLO utterance list)

    • The chatbot, as it echoes everything we tell him, is responding with You said: hello

    • But in our test case, we are expecting the response You said: hi from our initial live chat conversation with the chatbot

    • So two of those three test cases are failing

...

  • Let’s fix this test case failure by navigating back to the First Step Test Set, selecting the First Step Test Case and clicking on the You said: hi conversation step

...

  • Replace the text You said: hi with You said: *

  • Close the conversation step editor and save the test case

...

  • Run again the test session and you will see that now all three test cases are passing successfully

...

  • Let’s launch it in the Start menu by clicking the Let’s Go button in the Regression Testing section.

...

  • Enter a name for the test project and select the First Steps Chatbot, afterwards proceed to the next screen

...

  • In the second screen, click on Select from registered Test Sets, as we already created a test set record in the previous tutorial steps

  • Select the First Steps Test Set, and proceed to the final screen

...

  • On the final screen, click the Save Test Project and Start Testing button to

    • create a test project record …

    • … and start the first test session

...

  • When the test session is ready, you can navigate to the test project in the Test Projects menu

  • Note that in the test projects list, you can already see the result of the latest test session for this test project

...

  • You can now start a new test session by a single click on the Start Test Session Now button

...

  • You can now even see the history of all the previous test sessions for this test project in the Test Case History section (screenshot is from another test project that has more progress than the First Steps Test Project)

...