Writing test cases

Test cases must be written in plain text directly in the interface or on text editor. By using the characters "<" and ">" we simulate a conversation between bot and user.

🚧

Important rules

  • Each turn of the conversation is written as a line.
  • The first turn is always the user and generally a greeting. Writing "hi" will suffice
  • All HTML and SSML markups are ignored (they do not need to be included in test cases)
  • Asserting multiline bot responses - system replaces new lines with spaces when asserting bot response. When writing a test, the user can just put space in the place of a new line in the bot response.

Basic symbols

SymbolDescriptionExample
> user utterance> Hello! What's your name?
<bot response< Hello! I am a Banking Bot!
*< Allow to use * wildcard in bot response*< Today is
*<
?<Allow to use regular expression in bot response?< My version is \d{3}.\d.*
K*<Do not resolve Knowledge Base Items keys in response. Changing the value of this Knowledge Base Item does not affect the correctness of the test case execution result.

This solution support only explicit reference to Knowledge Base Item
K*< My name is {knowledge.name}

These examples are NOT supported:
K*< My age is {knowledge.age + 1}
knowledge.get("proces_"+proces_id)
//comments (working only as a new line)// Author: Adam Kowalski
// This test case validate fix: BOT-1234
aA off
aA on
By default, all assertions are case-sensitive. If you want to disable this, you can use theaA offcommand. This command is valid from this point onwards unless it is turned on again by aA onaA off
> Hello what's your name?
< Paul
aA on
> What?
< Paul
pnct off
pnct on
By default, punctuation is taken into account and will affect the result of your tests. If you want punctuation to be ignored, use the pnct off command. This will be valid for all assertions onwards, unless it is turned on again by pnct on. This works for: ,.;:-–—‐−?!'"()/\[] and `.> Hello, do you care about punctuation?
pnct off
< No!!! Punctuation, won't - mess up the result of this assertion, (haha)! thanks to the command above this line

❗️

Don't resolve Knowledge Base Items option supports only explicit reference to Knowledge Base Item. These examples are NOT supported:

  • K*< My age is {knowledge.age + 1}
  • knowledge.get("proces_"+proces_id)

Skipping bot response assertion

You can skip some bot responses in the test case if you don't want to assert the whole conversation. Example:

Whole conversationSkipped bot responses
> hi
< Hello, my name is XYZ.
< How can I help you?
> what is my balance?
*< Your balance is *
> hi
> what is my balance?
*< Your balance is *

Additional assertions

AssertionDescriptionExample
cmd NAME @ PARAMETER
---
NAME - command name
PARAMETER - command parameter (JSON format is allowed)
Assert commands used by bot during conversation.cmd campaign @ 4625
cmd redirectToNumber @ 48609141365
# tag1 tag2Assert analytical tags used by bot during conversation.# I_1 Topic_6003
assert JAVASCRIPTAssert responses, commands, tags etc. with JavaScript. The JAVASCRIPT expression must return true value for assertion to be successful. See JavaScript asserts data dictionary for available data.Assert number of responses:
assert responses.length === 1

Assert response message:
responses.some(r => r.text === “Hello”)

Data mocking

SymbolDescriptionExample
<< JSON
--
JSON - mocked extra info in JSON format
Mocking extra info data<< {"working_hours": "External_data"}
<t TIME
---
TIME - mocked time in ISO 8601 or timestamp format
mocking current time for a single request<t 2010-06-30T01:20+02:00
<t 1613720910194
<imocking the values returned by integration. Mock should be added BEFORE additional verifications (e.g. cmd, tags)<i { "success": true, "response": { "version": "prod-123" } }
---
> Connect me with a consultant
<i {"success": true, "response": "4123"}
cmd redirect @ 4123
< You will be redirected to 4123 line

Limitations

Between lines with User messages > there could be

  • only one integration mock
  • only one extra info mock
  • only one time mock
  • many tags assertions
  • many bot response assertions
  • many commands assertions

📘

It is recommended to use mocks first then assertions

Example test case

// Author: Adam Kowalski
// This test case validates fix: BOT-1234
> Hi
<< {"working_hours":"day"}
*< *Hi, my name is AdaBoT.* How can I help you?
> I want to make bank transfer.
# I_452
< Ok, our consutlant can help you with this issue.
> Ok, connect me with a consultant.
<i {"success": true, "response": "4123"
<t 1613720910194}
cmd redirect @ 4123
< You will be redirected to 4123 line