Writing test cases
Test cases must be written in plain text directly in the interface or on text editor. By using the characters "<" and ">" we simulate a conversation between bot and user.
Important rules
- Each turn of the conversation is written as a line.
- The first turn is always the user and generally a greeting. Writing "hi" will suffice
- All HTML and SSML markups are ignored (they do not need to be included in test cases)
- Asserting multiline bot responses - system replaces new lines with spaces when asserting bot response. When writing a test, the user can just put space in the place of a new line in the bot response.
Basic symbols
| Symbol | Description | Example |
|---|---|---|
| > | user utterance | > Hello! What's your name? |
| < | bot response | < Hello! I am a Banking Bot! |
| *< | Allow to use * wildcard in bot response | *< Today is *< |
| ?< | Allow to use regular expression in bot response | ?< My version is \d{3}.\d.* |
| K*< | Do not resolve Knowledge Base Items keys in response. Changing the value of this Knowledge Base Item does not affect the correctness of the test case execution result. This solution support only explicit reference to Knowledge Base Item | K*< My name is {knowledge.name} These examples are NOT supported: K*< My age is {knowledge.age + 1}knowledge.get("proces_"+proces_id) |
| // | comments (working only as a new line) | // Author: Adam Kowalski // This test case validate fix: BOT-1234 |
| aA off aA on | By default, all assertions are case-sensitive. If you want to disable this, you can use theaA offcommand. This command is valid from this point onwards unless it is turned on again by aA on | aA off > Hello what's your name? < Paul aA on > What? < Paul |
| pnct off pnct on | By default, punctuation is taken into account and will affect the result of your tests. If you want punctuation to be ignored, use the pnct off command. This will be valid for all assertions onwards, unless it is turned on again by pnct on. This works for: ,.;:-–—‐−?!'"()/\[] and `. | > Hello, do you care about punctuation? pnct off < No!!! Punctuation, won't - mess up the result of this assertion, (haha)! thanks to the command above this line |
Don't resolve Knowledge Base Items option supports only explicit reference to Knowledge Base Item. These examples are NOT supported:
K*< My age is {knowledge.age + 1}knowledge.get("proces_"+proces_id)
Skipping bot response assertion
You can skip some bot responses in the test case if you don't want to assert the whole conversation. Example:
| Whole conversation | Skipped bot responses |
|---|---|
> hi< Hello, my name is XYZ.< How can I help you?> what is my balance?*< Your balance is * | > hi> what is my balance?*< Your balance is * |
Additional assertions
| Assertion | Description | Example |
|---|---|---|
| cmd NAME @ PARAMETER --- NAME - command name PARAMETER - command parameter (JSON format is allowed) | Assert commands used by bot during conversation. | cmd campaign @ 4625 cmd redirectToNumber @ 48609141365 |
| # tag1 tag2 | Assert analytical tags used by bot during conversation. | # I_1 Topic_6003 |
| assert JAVASCRIPT | Assert responses, commands, tags etc. with JavaScript. The JAVASCRIPT expression must return true value for assertion to be successful. See JavaScript asserts data dictionary for available data. | Assert number of responses: assert responses.length === 1 Assert response message: responses.some(r => r.text === “Hello”) |
Data mocking
| Symbol | Description | Example |
|---|---|---|
| << JSON -- JSON - mocked extra info in JSON format | Mocking extra info data | << {"working_hours": "External_data"} |
| <t TIME --- TIME - mocked time in ISO 8601 or timestamp format | mocking current time for a single request | <t 2010-06-30T01:20+02:00 <t 1613720910194 |
| <i | mocking the values returned by integration. Mock should be added BEFORE additional verifications (e.g. cmd, tags) | <i { "success": true, "response": { "version": "prod-123" } } --- > Connect me with a consultant <i {"success": true, "response": "4123"} cmd redirect @ 4123 < You will be redirected to 4123 line |
Limitations
Between lines with User messages > there could be
- only one integration mock
- only one extra info mock
- only one time mock
- many tags assertions
- many bot response assertions
- many commands assertions
It is recommended to use mocks first then assertions
Example test case
// Author: Adam Kowalski
// This test case validates fix: BOT-1234
> Hi
<< {"working_hours":"day"}
*< *Hi, my name is AdaBoT.* How can I help you?
> I want to make bank transfer.
# I_452
< Ok, our consutlant can help you with this issue.
> Ok, connect me with a consultant.
<i {"success": true, "response": "4123"
<t 1613720910194}
cmd redirect @ 4123
< You will be redirected to 4123 line
Updated almost 2 years ago