Writing test cases

Test cases must be written in plain text directly in the interface or on text editor. By using the characters "<" and ">" we simulate a conversation between bot and user.

🚧
Important rules

Each turn of the conversation is written as a line.

The first turn is always the user and generally a greeting. Writing "hi" will suffice

All HTML and SSML markups are ignored (they do not need to be included in test cases)

Asserting multiline bot responses - system replaces new lines with spaces when asserting bot response. When writing a test, the user can just put space in the place of a new line in the bot response.

Basic symbols

Symbol	Description	Example
>	user utterance	Hello! What's your name?
<	bot response	< Hello! I am a Banking Bot!
*<	Allow to use * wildcard in bot response	< Today is _ < _
?<	Allow to use regular expression in bot response	?< My version is \d{3}.\d.*
*K<**	Do not resolve Knowledge Base Items keys in response. Changing the value of this Knowledge Base Item does not affect the correctness of the test case execution result. This solution support only explicit reference to Knowledge Base Item	K< My name is {knowledge.name} These examples are NOT supported: `K< My age is {knowledge.age + 1}` `knowledge.get("proces_"+proces_id)`
//	comments (working only as a new line)	// Author: Adam Kowalski // This test case validate fix: BOT-1234
aA off aA on	By default, all assertions are case-sensitive. If you want to disable this, you can use the`aA off`command. This command is valid from this point onwards unless it is turned on again by `aA on`	aA off Hello what's your name? < Paul aA on What? < Paul
pnct off pnct on	By default, punctuation is taken into account and will affect the result of your tests. If you want punctuation to be ignored, use the `pnct off` command. This will be valid for all assertions onwards, unless it is turned on again by `pnct on`. This works for: `,.;:-–—‐−?!'"()/\[]` and `.	Hello, do you care about punctuation? pnct off < No!!! Punctuation, won't - mess up the result of this assertion, (haha)! thanks to the command above this line

❗️
Don't resolve Knowledge Base Items option supports only explicit reference to Knowledge Base Item. These examples are NOT supported:

K*< My age is {knowledge.age + 1}

knowledge.get("proces_"+proces_id)

Skipping bot response assertion

You can skip some bot responses in the test case if you don't want to assert the whole conversation. Example:

Whole conversation	Skipped bot responses
`> hi` `< Hello, my name is XYZ.` `< How can I help you?` `> what is my balance?` `< Your balance is `	`> hi` `> what is my balance?` `< Your balance is `

Additional assertions

Assertion	Description	Example
cmdNAME@* PARAMETER NAME - command name PARAMETER - command parameter (JSON format is allowed)	Assert commands used by bot during conversation.	cmd campaign @ 4625 cmd redirectToNumber @ 48609141365
tag1 tag2	Assert analytical tags used by bot during conversation.	I_1 Topic_6003
assert JAVASCRIPT	Assert responses, commands, tags etc. with JavaScript. The `JAVASCRIPT` expression must return `true` value for assertion to be successful. See JavaScript asserts data dictionary for available data.	Assert number of responses: assert responses.length === 1 Assert response message: responses.some(r => r.text === “Hello”)

Assertion

Description

Example

*cmdNAME@** PARAMETER

NAME - command name PARAMETER - command parameter (JSON format is allowed)

Assert commands used by bot during conversation.

cmd campaign @ 4625
cmd redirectToNumber @ 48609141365

tag1 tag2

Assert analytical tags used by bot during conversation.

I_1 Topic_6003

assert JAVASCRIPT

Assert responses, commands, tags etc. with JavaScript. The JAVASCRIPT expression must return true value for assertion to be successful. See JavaScript asserts data dictionary for available data.

Assert number of responses:
assert responses.length === 1

Assert response message:
responses.some(r => r.text === “Hello”)

Data mocking

Symbol	Description	Example
<<* JSON -- JSON - mocked extra info in JSON format	Mocking extra info data	<< {"working_hours": "External_data"}
<t* TIME TIME - mocked time in ISO 8601 or timestamp format	mocking current time for a single request	<t 2010-06-30T01:20+02:00 <t 1613720910194
<i	mocking the values returned by integration. Mock should be added BEFORE additional verifications (e.g. cmd, tags)	<i { "success": true, "response": { "version": "prod-123" } } Connect me with a consultant <i {"success": true, "response": "4123"} cmd redirect @ 4123 < You will be redirected to 4123 line

Symbol

Description

Example

*<<** JSON -- JSON - mocked extra info in JSON format

Mocking extra info data

<< {"working_hours": "External_data"}

*<t** TIME

TIME - mocked time in ISO 8601 or timestamp format

mocking current time for a single request

<t 2010-06-30T01:20+02:00
<t 1613720910194

mocking the values returned by integration. Mock should be added BEFORE additional verifications (e.g. cmd, tags)

<i { "success": true, "response": { "version": "prod-123" } }

Connect me with a consultant
<i {"success": true, "response": "4123"}
cmd redirect @ 4123
< You will be redirected to 4123 line

Limitations

Between lines with User messages > there could be

only one integration mock
only one extra info mock
only one time mock
many tags assertions
many bot response assertions
many commands assertions

📘
It is recommended to use mocks first then assertions

Example test case

// Author: Adam Kowalski
// This test case validates fix: BOT-1234
> Hi
<< {"working_hours":"day"}
*< *Hi, my name is AdaBoT.* How can I help you?
> I want to make bank transfer.
# I_452
< Ok, our consutlant can help you with this issue.
> Ok, connect me with a consultant.
<i {"success": true, "response": "4123"
<t 1613720910194}
cmd redirect @ 4123
< You will be redirected to 4123 line

Updated 10 days ago