Automate Guide
AppService status
Start typing to search…

Writing test cases

Test cases must be written in plain text directly in the interface or on text editor. By using the characters "<" and ">" we simulate a conversation between bot and user.

🚧

Important rules

  • Each turn of the conversation is written as a line.
  • The first turn is always the user and generally a greeting. Writing "hi" will suffice
  • All HTML and SSML markups are ignored (they do not need to be included in test cases)
  • Asserting multiline bot responses - system replaces new lines with spaces when asserting bot response. When writing a test, the user can just put space in the place of a new line in the bot response.

Basic symbols

Symbol

Description

Example

>

user utterance

Hello! What's your name?

<

bot response

< Hello! I am a Banking Bot!

*<

Allow to use * wildcard in bot response

*< Today is *

*< *

?<

Allow to use regular expression in bot response

?< My version is \d\{3}.\d.*

K*<

Do not resolve Knowledge Base Items keys in response. Changing the value of this Knowledge Base Item does not affect the correctness of the test case execution result.

This solution support only explicit reference to Knowledge Base Item

K*< My name is {knowledge.name}

These examples are NOT supported:
K*< My age is {knowledge.age + 1}
knowledge.get("proces_"+proces_id)

//

comments (working only as a new line)

// Author: Adam Kowalski // This test case validate fix: BOT-1234

aA off
aA on

By default, all assertions are case-sensitive. If you want to disable this, you can use theaA offcommand. This command is valid from this point onwards unless it is turned on again by aA on

aA off

Hello what's your name?
< Paul
aA on
What?
< Paul

pnct off
pnct on

By default, punctuation is taken into account and will affect the result of your tests. If you want punctuation to be ignored, use the pnct off command. This will be valid for all assertions onwards, unless it is turned on again by pnct on. This works for: ,.;:-–—‐−?!'"()/\[] and `.

Hello, do you care about punctuation?
pnct off
< No!!! Punctuation, won't - mess up the result of this assertion, (haha)! thanks to the command above this line

❗️

Don't resolve Knowledge Base Items option supports only explicit reference to Knowledge Base Item. These examples are NOT supported:

  • K*< My age is {knowledge.age + 1}
  • knowledge.get("proces_"+proces_id)

Skipping bot response assertion

You can skip some bot responses in the test case if you don't want to assert the whole conversation. Example:

Whole conversation

Skipped bot responses

> hi < Hello, my name is XYZ. < How can I help you? > what is my balance? *< Your balance is *

> hi
> what is my balance?
*< Your balance is *

Additional assertions

Assertion

Description

Example

cmd NAME @ PARAMETER

NAME - command name PARAMETER - command parameter (JSON format is allowed)

Assert commands used by bot during conversation.

cmd campaign @ 4625
cmd redirectToNumber @ 48609141365

# tag1 tag2

Assert analytical tags used by bot during conversation.

# I_1 Topic_6003

assert JAVASCRIPT

Assert responses, commands, tags etc. with JavaScript. The JAVASCRIPT expression must return true value for assertion to be successful. See JavaScript asserts data dictionary for available data.

Assert number of responses:
assert responses.length === 1

Assert response message:
assert responses.some(r => r.text === “Hello”)

Data mocking

Symbol

Description

Example

<< JSON

JSON - mocked extra info in JSON format

Mocking extra info data

<< {"working_hours": "External_data"}

<t TIME

TIME - mocked time in ISO 8601 or timestamp format

mocking current time

<t 2010-06-30T01:20+02:00 <t 1613720910194

<i JSON

mocking the values returned by integration. Mock should be added BEFORE additional verifications (e.g. cmd, tags)

<i { "success": true, "response": { "version": "prod-123" } }

> Connect me with a consultant <i {"success": true, "response": "4123"} cmd redirect @ 4123 < You will be redirected to 4123 line


Mock Scopes

Mocks can be defined in two ways, depending on when they should apply:

1. Globally for the whole conversation

  • Definition: Placed at the very top of the test case, before the first user utterance.
  • Behavior: The mock data persists throughout the entire conversation.
  • Availability: Supported from Automate version 259.

2. Per single message

  • Definition: Placed immediately after a user utterance (line starting with >).
  • Behavior: The mock applies to the processing of that specific user message.

Examples

Global Mock Example Use this structure to set data (like user attributes or time) that should remain constant or be present from the start.

// Global mocks defined at the top
<< {"user_segment": "vip", "region": "EU"}
<t 2023-10-01T09:00:00Z

// First user utterance
> Hi
< Hello! I see you are a VIP customer from the EU.

Per-Message Mock Example Use this structure to simulate a specific integration response for a single turn.

> Check my order status
// Integration mock for this specific turn
<i {"status": "shipped", "delivery_date": "2023-10-05"}
< Your order has been shipped.

Limitations

Between lines with User messages > there could be

  • only one integration mock
  • only one extra info mock
  • only one time mock
  • many tags assertions
  • many bot response assertions
  • many commands assertions
📘

It is recommended to use mocks first then assertions

Example test case

// Author: Adam Kowalski
// This test case validates fix: BOT-1234
> Hi
<< {"working_hours":"day"}
*< *Hi, my name is AdaBoT.* How can I help you?
> I want to make bank transfer.
# I_452
< Ok, our consutlant can help you with this issue.
> Ok, connect me with a consultant.
<i {"success": true, "response": "4123"
<t 1613720910194}
cmd redirect @ 4123
< You will be redirected to 4123 line

Updated 16 days ago