Migrating a project between environments

This guide explains how to copy a SentiOne Automate project from one environment to another — for example, promoting a bot from a staging environment to production, or cloning a project into a customer's environment.

Migration runs entirely through the public API, driven by the migrate-project-between-envs.sh script. You provide the project's UUID and an API key for each environment; the script exports the project from the source and imports it into the target.

#!/usr/bin/env bash
#
# Migrate a project + the knowledge base between SentiOne Automate environments
# using the public API.
#
# Requires: bash >=4, curl, jq

set -euo pipefail

API_PREFIX="/api/public/v1"
DEFAULT_CONNECT_TIMEOUT=30
DEFAULT_MAX_TIME=1800
UUID_REGEX='^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$'

# --- helpers ---------------------------------------------------------------

show_header() {
    echo >&2
    echo "##########################################################" >&2
    echo >&2
    for msg in "$@"; do
        echo "$msg" >&2
    done
    echo >&2
    echo "##########################################################" >&2
    echo >&2
}

mask_key() {
    local key="$1"
    local len=${#key}
    if (( len <= 8 )); then
        echo "***"
    else
        echo "${key:0:6}…${key: -3}"
    fi
}

url_encode() {
    jq -rn --arg s "$1" '$s|@uri'
}

slugify_filename() {
    printf '%s' "$1" | LC_ALL=C tr -c 'A-Za-z0-9._-' '_'
}

is_valid_zip() {
    local file="$1"
    [[ -s "$file" ]] || return 1
    [[ "$(head -c 2 "$file" 2>/dev/null)" == "PK" ]]
}

is_valid_uuid() {
    [[ "$1" =~ $UUID_REGEX ]]
}

format_elapsed() {
    local s=$1
    if (( s >= 60 )); then
        printf '%dm %ds' "$((s / 60))" "$((s % 60))"
    else
        printf '%ds' "$s"
    fi
}

state_label() {
    case "$1" in
        ok)        echo "OK" ;;
        failed)    echo "FAILED" ;;
        skipped)   echo "skipped (global KB migration not requested; pass --migrate-global-kb)" ;;
        not_run)   echo "not run" ;;
        completed) echo "completed" ;;
        dry_run)   echo "dry run (no writes performed)" ;;
        *)         echo "$1" ;;
    esac
}

# --- summary + exit trap (installed before mktemp) -------------------------

START_SECONDS="$SECONDS"
SUMMARY_PROJECT_STATE="not_run"     # not_run | ok | failed | dry_run
SUMMARY_PROJECT_NAME=""
SUMMARY_KB_DOCS_STATE="not_run"     # not_run | skipped | completed | dry_run
SUMMARY_KB_DOCS_OK=()
SUMMARY_KB_DOCS_FAILED=()
SUMMARY_KB_ITEMS_STATE="not_run"    # not_run | skipped | ok | failed | dry_run
SUMMARY_KB_ITEMS_ADDED="?"
SUMMARY_KB_ITEMS_UPDATED="?"
SUMMARY_KB_ITEMS_UNCHANGED="?"

PROJECT_UUID=""
TARGET_PROJECT_UUID=""
WORKDIR=""
CLEANUP_WORKDIR="false"

print_summary() {
    local elapsed=$(( SECONDS - START_SECONDS ))
    {
        echo
        echo "##########################################################"
        echo "Summary"
        echo "##########################################################"
        echo

        local proj_label
        proj_label="$(state_label "$SUMMARY_PROJECT_STATE")"
        if [[ -n "$SUMMARY_PROJECT_NAME" ]]; then
            proj_label="$proj_label (\"$SUMMARY_PROJECT_NAME\")"
        fi
        printf 'Project       %s  ->  %s   %s\n' \
            "${PROJECT_UUID:-?}" "${TARGET_PROJECT_UUID:-?}" "$proj_label"

        case "$SUMMARY_KB_DOCS_STATE" in
            completed)
                local total=$(( ${#SUMMARY_KB_DOCS_OK[@]} + ${#SUMMARY_KB_DOCS_FAILED[@]} ))
                local failed_str=""
                if (( ${#SUMMARY_KB_DOCS_FAILED[@]} > 0 )); then
                    failed_str=", failed: $(printf '%s, ' "${SUMMARY_KB_DOCS_FAILED[@]}" | sed 's/, $//')"
                fi
                printf 'KB documents  %d total, %d OK, %d failed%s\n' \
                    "$total" \
                    "${#SUMMARY_KB_DOCS_OK[@]}" \
                    "${#SUMMARY_KB_DOCS_FAILED[@]}" \
                    "$failed_str"
                ;;
            *)
                printf 'KB documents  %s\n' "$(state_label "$SUMMARY_KB_DOCS_STATE")"
                ;;
        esac

        case "$SUMMARY_KB_ITEMS_STATE" in
            ok)
                printf 'KB items      added %s, updated %s, unchanged %s\n' \
                    "$SUMMARY_KB_ITEMS_ADDED" \
                    "$SUMMARY_KB_ITEMS_UPDATED" \
                    "$SUMMARY_KB_ITEMS_UNCHANGED"
                ;;
            *)
                printf 'KB items      %s\n' "$(state_label "$SUMMARY_KB_ITEMS_STATE")"
                ;;
        esac

        if [[ -n "$WORKDIR" ]]; then
            local workdir_note=""
            if [[ "$CLEANUP_WORKDIR" == "true" ]]; then
                workdir_note=" (auto-cleaned)"
            fi
            printf 'Workdir       %s%s\n' "$WORKDIR" "$workdir_note"
        fi
        printf 'Elapsed       %s\n' "$(format_elapsed "$elapsed")"
        echo
        echo "##########################################################"
        echo
    } >&2
}

on_exit() {
    local rc=$?
    set +e
    print_summary
    if [[ "$CLEANUP_WORKDIR" == "true" && -n "$WORKDIR" ]]; then
        rm -rf "$WORKDIR"
    fi
    exit "$rc"
}

# --- usage -----------------------------------------------------------------

usage() {
    cat <<'EOF' >&2
Usage: migrate-between-envs.sh <project-uuid> --url URL --api-key KEY [options]

Copies one project (by UUID) from a source environment to a target
environment. The org-wide knowledge base (documents + items) is migrated
ONLY when --migrate-global-kb is passed — see the warning on that flag.

Required:
  <project-uuid>            UUID of the project to migrate.
  --url URL                 Source environment base URL (e.g. https://chatbots.example.com).
  --api-key KEY             Source environment API key (X-Api-Key header).

Target (optional — defaults to source for same-env runs):
  --target-url URL          Target environment base URL.
  --target-api-key KEY      Target environment API key.
  --target-project-id UUID  UUID to use for the project on the target
                            (defaults to <project-uuid>). Use a fresh UUID
                            when the source UUID is already taken on the
                            target by another tenant.

Options:
  --include-nlu-models      Include trained binary NLU models in the project export.
  --publish-flows           Publish all flows after import on the target.
  --migrate-global-kb       Also migrate the ORG-WIDE knowledge base (documents
                            + items). Off by default. WARNING: this is not
                            project-scoped — it overwrites the target org's
                            entire KB and is hard to roll back. Do not use
                            against a shared target that holds other tenants' KB.
  --fail-fast               Abort KB-doc loop on first per-doc failure.
  --dry-run                 Probe source/target metadata and print what *would*
                            be migrated. No writes to the target.
  --workdir DIR             Where to stage downloads (default: mktemp). The
                            script writes API-key auth files there at mode 600.
  --debug                   Log every HTTP request to stderr. API keys are
                            masked. WARNING: URLs may still be sensitive.
  -h, --help                Show this help.

Exit codes:
  0  success
  1  any failure (project import, KB doc/items import, network, etc.)
  2  invalid CLI usage
EOF
}

# --- argument parsing ------------------------------------------------------

SOURCE_URL=""
SOURCE_KEY=""
TARGET_URL=""
TARGET_KEY=""
INCLUDE_NLU_MODELS="false"
PUBLISH_FLOWS="false"
MIGRATE_GLOBAL_KB="false"
DEBUG="false"
DRY_RUN="false"
FAIL_FAST="false"

while [[ $# -gt 0 ]]; do
    case "$1" in
        -h|--help)
            usage
            exit 0
            ;;
        --url)              SOURCE_URL="$2";          shift 2 ;;
        --api-key)          SOURCE_KEY="$2";          shift 2 ;;
        --target-url)       TARGET_URL="$2";          shift 2 ;;
        --target-api-key)   TARGET_KEY="$2";          shift 2 ;;
        --target-project-id) TARGET_PROJECT_UUID="$2"; shift 2 ;;
        --include-nlu-models) INCLUDE_NLU_MODELS="true"; shift ;;
        --publish-flows)    PUBLISH_FLOWS="true";     shift ;;
        --migrate-global-kb) MIGRATE_GLOBAL_KB="true"; shift ;;
        --fail-fast)        FAIL_FAST="true";         shift ;;
        --dry-run)          DRY_RUN="true";           shift ;;
        --workdir)          WORKDIR="$2";             shift 2 ;;
        --debug)            DEBUG="true";             shift ;;
        -*)
            echo "Unknown option: $1" >&2
            usage
            exit 2
            ;;
        *)
            if [[ -z "$PROJECT_UUID" ]]; then
                PROJECT_UUID="$1"
            else
                echo "Unexpected positional argument: $1" >&2
                usage
                exit 2
            fi
            shift
            ;;
    esac
done

if [[ -z "$PROJECT_UUID" || -z "$SOURCE_URL" || -z "$SOURCE_KEY" ]]; then
    echo "Missing required argument: <project-uuid>, --url, or --api-key." >&2
    usage
    exit 2
fi

TARGET_URL="${TARGET_URL:-$SOURCE_URL}"
TARGET_KEY="${TARGET_KEY:-$SOURCE_KEY}"
TARGET_PROJECT_UUID="${TARGET_PROJECT_UUID:-$PROJECT_UUID}"

SOURCE_URL="${SOURCE_URL%/}"
TARGET_URL="${TARGET_URL%/}"

if ! is_valid_uuid "$PROJECT_UUID"; then
    echo "Invalid <project-uuid>: $PROJECT_UUID" >&2
    exit 2
fi
if ! is_valid_uuid "$TARGET_PROJECT_UUID"; then
    echo "Invalid --target-project-id: $TARGET_PROJECT_UUID" >&2
    exit 2
fi

if ! command -v jq >/dev/null 2>&1; then
    show_header "jq is required but not installed"
    exit 1
fi

# Install the exit trap BEFORE creating temporary state so a Ctrl-C between
# mktemp and the trap line can't leak a workdir.
trap on_exit EXIT

# --- workdir + auth files --------------------------------------------------

if [[ -z "$WORKDIR" ]]; then
    WORKDIR="$(mktemp -d -t migrate-between-envs-XXXXXX)"
    CLEANUP_WORKDIR="true"
else
    mkdir -p "$WORKDIR"
fi
chmod 700 "$WORKDIR" 2>/dev/null || true

SOURCE_AUTH_FILE="$WORKDIR/source-auth.hdr"
TARGET_AUTH_FILE="$WORKDIR/target-auth.hdr"
(
    umask 077
    printf 'X-Api-Key: %s\n' "$SOURCE_KEY" > "$SOURCE_AUTH_FILE"
    printf 'X-Api-Key: %s\n' "$TARGET_KEY" > "$TARGET_AUTH_FILE"
)

# --- curl plumbing ---------------------------------------------------------

CURL_COMMON_OPTS=(
    --silent --show-error --fail-with-body
    --connect-timeout "$DEFAULT_CONNECT_TIMEOUT"
    --max-time "$DEFAULT_MAX_TIME"
)

auth_label() {
    case "$1" in
        "$SOURCE_AUTH_FILE") echo "source" ;;
        "$TARGET_AUTH_FILE") echo "target" ;;
        *) echo "$(basename "$1" .hdr)" ;;
    esac
}

debug_log() {
    if [[ "$DEBUG" == "true" ]]; then
        echo "[debug] $*" >&2
    fi
}

curl_get_json() {
    local url="$1" auth_file="$2"
    debug_log "GET $url (auth: $(auth_label "$auth_file"))"
    curl "${CURL_COMMON_OPTS[@]}" \
        -H "@${auth_file}" \
        -H "Accept: application/json" \
        "$url"
}

curl_get_to_file() {
    local url="$1" auth_file="$2" out="$3"
    debug_log "GET $url -> $out (auth: $(auth_label "$auth_file"))"
    curl "${CURL_COMMON_OPTS[@]}" \
        -H "@${auth_file}" \
        -o "$out" \
        "$url"
}

curl_get_status_to_file() {
    local url="$1" auth_file="$2" out="$3"
    debug_log "GET $url -> $out (status probe, auth: $(auth_label "$auth_file"))"
    curl --silent --show-error \
        --connect-timeout "$DEFAULT_CONNECT_TIMEOUT" \
        --max-time "$DEFAULT_MAX_TIME" \
        -o "$out" \
        -w '%{http_code}' \
        -H "@${auth_file}" \
        -H "Accept: application/json" \
        "$url" 2>/dev/null || echo "000"
}

curl_post_file() {
    local url="$1" auth_file="$2" file="$3"
    debug_log "POST $url (file=$file, auth: $(auth_label "$auth_file"))"
    curl "${CURL_COMMON_OPTS[@]}" \
        -H "@${auth_file}" \
        -F "file=@${file}" \
        "$url"
}

curl_put_doc() {
    local url="$1" auth_file="$2" file="$3" params_json="$4"
    local file_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
    [[ "$file" == *.csv ]] && file_type="text/csv"
    debug_log "PUT $url (file=$file [$file_type], params=$params_json, auth: $(auth_label "$auth_file"))"
    curl "${CURL_COMMON_OPTS[@]}" \
        -X PUT \
        -H "@${auth_file}" \
        -F "params=${params_json};type=application/json" \
        -F "file=@${file};type=${file_type}" \
        "$url"
}

print_response_body() {
    local file="$1"
    if [[ ! -s "$file" ]]; then
        echo "(empty response body)" >&2
        return
    fi
    if jq '.' "$file" >&2 2>/dev/null; then
        return
    fi
    local mime=""
    if command -v file >/dev/null 2>&1; then
        mime="$(file -b --mime-type "$file" 2>/dev/null || true)"
    fi
    case "$mime" in
        text/*|application/json*|application/xml*|application/*+json*|"")
            cat "$file" >&2
            ;;
        *)
            echo "(non-text response: ${mime}, $(wc -c <"$file") bytes — kept at $file)" >&2
            ;;
    esac
}

# --- stage 1: AI credentials ----------------------------------------------

show_header "Fetching AI credentials from source and target environments"

SOURCE_CREDS_JSON="$WORKDIR/source-credentials.json"
TARGET_CREDS_JSON="$WORKDIR/target-credentials.json"

if ! curl_get_json "$SOURCE_URL$API_PREFIX/organization/ai-credentials" "$SOURCE_AUTH_FILE" >"$SOURCE_CREDS_JSON"; then
    echo "ERROR: failed to fetch source AI credentials. Server response:" >&2
    print_response_body "$SOURCE_CREDS_JSON"
    exit 1
fi
if ! curl_get_json "$TARGET_URL$API_PREFIX/organization/ai-credentials" "$TARGET_AUTH_FILE" >"$TARGET_CREDS_JSON"; then
    echo "ERROR: failed to fetch target AI credentials. Server response:" >&2
    print_response_body "$TARGET_CREDS_JSON"
    exit 1
fi

src_credential_name_for_id() {
    jq -r --arg id "$1" '.[] | select(.id == $id) | .name' "$SOURCE_CREDS_JSON"
}

target_credential_id_for_name() {
    local name="$1"
    local matches
    matches="$(jq -r --arg name "$name" '[.[] | select(.name == $name) | .id]' "$TARGET_CREDS_JSON")"
    local count
    count="$(jq 'length' <<<"$matches")"
    if [[ "$count" == "1" ]]; then
        jq -r '.[0]' <<<"$matches"
    elif [[ "$count" -gt "1" ]]; then
        echo "WARN: target has $count credentials named \"$name\" — picking the first; rename duplicates on the target to disambiguate." >&2
        jq -r '.[0]' <<<"$matches"
    fi
    # count == 0 → echo nothing
}

# --- dry-run short-circuit ------------------------------------------------

if [[ "$DRY_RUN" == "true" ]]; then
    show_header "Dry run — no writes to target"

    PROJECT_DETAILS_JSON="$WORKDIR/project-details.json"
    if ! curl_get_json "$SOURCE_URL$API_PREFIX/projects/$PROJECT_UUID" "$SOURCE_AUTH_FILE" >"$PROJECT_DETAILS_JSON"; then
        echo "ERROR: failed to fetch source project. Server response:" >&2
        print_response_body "$PROJECT_DETAILS_JSON"
        exit 1
    fi
    dry_project_name="$(jq -r '.name // "<unknown>"' "$PROJECT_DETAILS_JSON")"
    dry_flow_count="$(jq -r '(.flows // []) | length' "$PROJECT_DETAILS_JSON")"
    dry_nlu_count="$(jq -r '(.nlus // []) | length' "$PROJECT_DETAILS_JSON")"

    echo "Project        $PROJECT_UUID  ->  $TARGET_PROJECT_UUID" >&2
    echo "               \"$dry_project_name\"  (${dry_flow_count} flows, ${dry_nlu_count} NLUs)" >&2

    TARGET_PROBE_JSON="$WORKDIR/target-project-probe.json"
    target_status="$(curl_get_status_to_file "$TARGET_URL$API_PREFIX/projects/$TARGET_PROJECT_UUID" "$TARGET_AUTH_FILE" "$TARGET_PROBE_JSON")"
    case "$target_status" in
        200)
            existing_name="$(jq -r '.name // "<unknown>"' "$TARGET_PROBE_JSON" 2>/dev/null || echo "<unknown>")"
            echo "               ⚠️  target UUID already exists on $TARGET_URL (\"$existing_name\")" >&2
            echo "                   a real run would OVERWRITE it — pass --target-project-id for a new project" >&2
            ;;
        404)
            echo "               target UUID is free — a real run would create a new project" >&2
            ;;
        *)
            echo "               WARN: could not verify whether the target UUID exists (HTTP $target_status)" >&2
            ;;
    esac

    if [[ "$MIGRATE_GLOBAL_KB" == "true" ]]; then
        echo >&2
        echo "WARN: --migrate-global-kb is set — the ENTIRE org KB below would be" >&2
        echo "      written to the target, overwriting any existing KB there." >&2
        KB_LIST_JSON="$WORKDIR/kb-documents.json"
        if ! curl_get_json "$SOURCE_URL$API_PREFIX/knowledge/documents" "$SOURCE_AUTH_FILE" >"$KB_LIST_JSON"; then
            echo "ERROR: failed to list KB documents. Server response:" >&2
            print_response_body "$KB_LIST_JSON"
            exit 1
        fi
        if ! jq empty "$KB_LIST_JSON" 2>/dev/null; then
            echo "ERROR: KB documents response is not valid JSON. Body:" >&2
            cat "$KB_LIST_JSON" >&2
            exit 1
        fi
        dry_doc_count="$(jq 'length' "$KB_LIST_JSON")"
        echo "KB documents   ${dry_doc_count} would be migrated" >&2
        while IFS=$'\t' read -r doc_key src_cred_id; do
            [[ -z "$doc_key" ]] && continue
            if [[ -n "$src_cred_id" && "$src_cred_id" != "null" ]]; then
                cred_name="$(src_credential_name_for_id "$src_cred_id")"
                if [[ -z "$cred_name" ]]; then
                    echo "               - $doc_key  [cred id=$src_cred_id has no name on source → default]" >&2
                else
                    tgt_cred_id="$(target_credential_id_for_name "$cred_name")"
                    if [[ -z "$tgt_cred_id" ]]; then
                        echo "               - $doc_key  [cred \"$cred_name\" missing on target → default]" >&2
                    else
                        echo "               - $doc_key  [cred \"$cred_name\" mapped]" >&2
                    fi
                fi
            else
                echo "               - $doc_key  [no cred → target default]" >&2
            fi
        done < <(jq -r '.[] | [.key, (.embeddingsCredentialsId // "")] | @tsv' "$KB_LIST_JSON")

        KB_ITEMS_CSV="$WORKDIR/kb-items.csv"
        if ! curl_get_to_file "$SOURCE_URL$API_PREFIX/knowledge/items/export" "$SOURCE_AUTH_FILE" "$KB_ITEMS_CSV"; then
            echo "ERROR: failed to export KB items. Server response:" >&2
            print_response_body "$KB_ITEMS_CSV"
            exit 1
        fi
        dry_item_count=$(( $(wc -l <"$KB_ITEMS_CSV") - 1 ))
        (( dry_item_count < 0 )) && dry_item_count=0
        echo "KB items       ${dry_item_count} items" >&2
        if (( dry_item_count > 10000 )); then
            echo "               WARN: exceeds the server's 10 000-row import cap — see README for manual chunking." >&2
        fi
        SUMMARY_KB_DOCS_STATE="dry_run"
        SUMMARY_KB_ITEMS_STATE="dry_run"
    else
        echo "KB             skipped — pass --migrate-global-kb to include the org KB" >&2
        SUMMARY_KB_DOCS_STATE="skipped"
        SUMMARY_KB_ITEMS_STATE="skipped"
    fi

    SUMMARY_PROJECT_STATE="dry_run"
    SUMMARY_PROJECT_NAME="$dry_project_name"
    exit 0
fi

# --- stage 2: project export/import ---------------------------------------

show_header "Migrating project $PROJECT_UUID -> $TARGET_PROJECT_UUID"

PROJECT_ZIP="$WORKDIR/project.zip"
EXPORT_URL="$SOURCE_URL$API_PREFIX/projects/$PROJECT_UUID/export?includeNluModels=$INCLUDE_NLU_MODELS"
IMPORT_URL="$TARGET_URL$API_PREFIX/projects/$TARGET_PROJECT_UUID/import?publishFlows=$PUBLISH_FLOWS"

print_nlu_export_hint() {
    [[ "$INCLUDE_NLU_MODELS" == "true" ]] || return 0
    echo >&2
    echo "NOTE: --include-nlu-models was set. If any NLU in the project has no trained" >&2
    echo "data, the export endpoint aborts the download mid-stream (curl error 92 /" >&2
    echo "HTTP/2 INTERNAL_ERROR). Retry without --include-nlu-models, or train the" >&2
    echo "affected NLUs first." >&2
}

echo "Exporting from $SOURCE_URL ..." >&2
if ! curl_get_to_file "$EXPORT_URL" "$SOURCE_AUTH_FILE" "$PROJECT_ZIP"; then
    echo "ERROR: project export failed. Server response:" >&2
    print_response_body "$PROJECT_ZIP"
    print_nlu_export_hint
    SUMMARY_PROJECT_STATE="failed"
    exit 1
fi

if ! is_valid_zip "$PROJECT_ZIP"; then
    echo "ERROR: the exported project is not a valid ZIP — the export was truncated or" >&2
    echo "returned an error body instead of an archive. Contents:" >&2
    print_response_body "$PROJECT_ZIP"
    print_nlu_export_hint
    SUMMARY_PROJECT_STATE="failed"
    exit 1
fi

echo "Importing into $TARGET_URL ..." >&2
IMPORT_RESPONSE_FILE="$WORKDIR/import-response.json"
if ! curl_post_file "$IMPORT_URL" "$TARGET_AUTH_FILE" "$PROJECT_ZIP" >"$IMPORT_RESPONSE_FILE"; then
    echo "ERROR: project import failed. Server response:" >&2
    print_response_body "$IMPORT_RESPONSE_FILE"
    SUMMARY_PROJECT_STATE="failed"
    exit 1
fi
print_response_body "$IMPORT_RESPONSE_FILE"
SUMMARY_PROJECT_STATE="ok"
SUMMARY_PROJECT_NAME="$(jq -r '.projectDetails.name // empty' "$IMPORT_RESPONSE_FILE" 2>/dev/null || true)"

if [[ "$MIGRATE_GLOBAL_KB" != "true" ]]; then
    show_header "Skipping knowledge base" \
        "Project migrated. The org-wide KB was NOT touched." \
        "Pass --migrate-global-kb to also migrate KB documents + items."
    SUMMARY_KB_DOCS_STATE="skipped"
    SUMMARY_KB_ITEMS_STATE="skipped"
    exit 0
fi

# --- stage 3: KB documents ------------------------------------------------

show_header "Migrating knowledge base documents" \
    "WARNING: writing the ENTIRE org KB to $TARGET_URL"

KB_LIST_JSON="$WORKDIR/kb-documents.json"
if ! curl_get_json "$SOURCE_URL$API_PREFIX/knowledge/documents" "$SOURCE_AUTH_FILE" >"$KB_LIST_JSON"; then
    echo "ERROR: failed to list KB documents. Server response:" >&2
    print_response_body "$KB_LIST_JSON"
    exit 1
fi
if ! jq empty "$KB_LIST_JSON" 2>/dev/null; then
    echo "ERROR: KB documents response is not valid JSON. Body:" >&2
    cat "$KB_LIST_JSON" >&2
    exit 1
fi

mkdir -p "$WORKDIR/kb"

while IFS=$'\t' read -r doc_key source_cred_id; do
    [[ -z "$doc_key" ]] && continue
    echo "---" >&2
    echo "Document: $doc_key" >&2

    target_cred_id=""
    if [[ -n "$source_cred_id" && "$source_cred_id" != "null" ]]; then
        cred_name="$(src_credential_name_for_id "$source_cred_id")"
        if [[ -n "$cred_name" ]]; then
            target_cred_id="$(target_credential_id_for_name "$cred_name")"
            if [[ -z "$target_cred_id" ]]; then
                echo "WARN: credential \"$cred_name\" not found on target — falling back to default" >&2
            fi
        else
            echo "WARN: source credential id \"$source_cred_id\" has no name lookup — falling back to default" >&2
        fi
    fi

    if [[ -n "$target_cred_id" ]]; then
        params_json="$(jq -nc --arg id "$target_cred_id" '{embeddingsCredentialsId: $id}')"
    else
        params_json="{}"
    fi

    encoded_key="$(url_encode "$doc_key")"
    safe_name="$(slugify_filename "$doc_key")"
    xlsx_path="$WORKDIR/kb/${safe_name}.xlsx"
    put_response_file="$WORKDIR/kb/${safe_name}.upsert-response.json"

    if ! curl_get_to_file \
        "$SOURCE_URL$API_PREFIX/knowledge/documents/${encoded_key}/export/xlsx" \
        "$SOURCE_AUTH_FILE" \
        "$xlsx_path"; then
        echo "ERROR: failed to export document \"$doc_key\" from source. Server response:" >&2
        print_response_body "$xlsx_path"
        SUMMARY_KB_DOCS_FAILED+=("$doc_key")
        if [[ "$FAIL_FAST" == "true" ]]; then
            echo "Aborting after first failure (--fail-fast)" >&2
            break
        fi
        continue
    fi

    if ! curl_put_doc \
        "$TARGET_URL$API_PREFIX/knowledge/documents/${encoded_key}" \
        "$TARGET_AUTH_FILE" \
        "$xlsx_path" \
        "$params_json" >"$put_response_file"; then
        echo "ERROR: failed to upsert document \"$doc_key\" on target. Server response:" >&2
        print_response_body "$put_response_file"
        SUMMARY_KB_DOCS_FAILED+=("$doc_key")
        if [[ "$FAIL_FAST" == "true" ]]; then
            echo "Aborting after first failure (--fail-fast)" >&2
            break
        fi
        continue
    fi

    SUMMARY_KB_DOCS_OK+=("$doc_key")
    echo "OK: $doc_key" >&2
done < <(jq -r '.[] | [.key, (.embeddingsCredentialsId // "")] | @tsv' "$KB_LIST_JSON")
SUMMARY_KB_DOCS_STATE="completed"

# --- stage 4: KB items ----------------------------------------------------

show_header "Migrating knowledge base items"

KB_ITEMS_CSV="$WORKDIR/kb-items.csv"
if ! curl_get_to_file "$SOURCE_URL$API_PREFIX/knowledge/items/export" "$SOURCE_AUTH_FILE" "$KB_ITEMS_CSV"; then
    echo "ERROR: failed to export KB items from source. Server response:" >&2
    print_response_body "$KB_ITEMS_CSV"
    SUMMARY_KB_ITEMS_STATE="failed"
    exit 1
fi

ITEMS_RESPONSE_FILE="$WORKDIR/kb-items-response.json"
if ! curl_post_file \
    "$TARGET_URL$API_PREFIX/knowledge/items/import" \
    "$TARGET_AUTH_FILE" \
    "$KB_ITEMS_CSV" >"$ITEMS_RESPONSE_FILE"; then
    echo "ERROR: KB items import failed. Server response:" >&2
    print_response_body "$ITEMS_RESPONSE_FILE"
    echo >&2
    echo "If the error is 'FileTooBig', the server caps imports at 10 000 rows." >&2
    echo "Re-run with --workdir DIR to keep the exported CSV, then chunk and import" >&2
    echo "manually. See 'Manual chunking for KBs over 10 000 items' in tools/README.md." >&2
    SUMMARY_KB_ITEMS_STATE="failed"
    exit 1
fi
print_response_body "$ITEMS_RESPONSE_FILE"
SUMMARY_KB_ITEMS_STATE="ok"
SUMMARY_KB_ITEMS_ADDED="$(jq -r '.added // 0' "$ITEMS_RESPONSE_FILE" 2>/dev/null || echo 0)"
SUMMARY_KB_ITEMS_UPDATED="$(jq -r '.updated // 0' "$ITEMS_RESPONSE_FILE" 2>/dev/null || echo 0)"
SUMMARY_KB_ITEMS_UNCHANGED="$(jq -r '.unchanged // 0' "$ITEMS_RESPONSE_FILE" 2>/dev/null || echo 0)"

# --- exit code ------------------------------------------------------------

if (( ${#SUMMARY_KB_DOCS_FAILED[@]} > 0 )); then
    exit 1
fi

Before you start

You will need:

The project UUID — see Find the project UUID.
An API key for the source environment (the X-Api-Key) — see Get an API key.
An API key for the target environment. For a same-environment copy you only need one key (see Copy within one environment).
A machine with bash, curl, and jq — or Docker, which gives you a ready-made runtime. See Running with Docker.

API keys are organization-scoped. The source key must belong to the organization that owns the project. The target key must belong to the organization you want the project to land in.

Find the project UUID

In the admin panel, open the Projects list. Each project row has a Copy project ID icon — click it and the UUID is copied to your clipboard (you'll see a "Project ID copied to clipboard" confirmation). That UUID is the first argument to the script.

Alternatively, open the project and read the UUID from the browser address bar, e.g. https://app.example.com/project/5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10/....

Get an API key

The script authenticates with an organization API key (sent as the X-Api-Key header; these keys are prefixed at-). To create one in the admin panel:

Go to Organization → API Keys.
Click Create API key and confirm.
On the Your API key screen, use Copy the API key to clipboard — this is the only time the full key is shown, so store it somewhere safe.

Create a key in each organization you migrate between: one in the source organization (--api-key) and one in the target organization (--target-api-key).

Your first migration

The most common case: copy a project from one environment to another.

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY

What happens:

The script reads the project from staging.example.com.
It writes the project into prod.example.com under the same UUID.
It prints a summary showing what was migrated and how long it took.

By default only the project is migrated — flows, intents, settings, and configuration. The knowledge base is not touched (see Migrating the knowledge base).

A successful run looks like this:

##########################################################
Migrating project 5a54ca09-...  ->  5a54ca09-...
##########################################################

Exporting from https://staging.example.com ...
Importing into https://prod.example.com ...
{ "projectDetails": { "name": "My Bot", ... }, ... }

##########################################################
Skipping knowledge base

Project migrated. The org-wide KB was NOT touched.
Pass --migrate-global-kb to also migrate KB documents + items.
##########################################################

##########################################################
Summary
##########################################################

Project       5a54ca09-...  ->  5a54ca09-...   OK ("My Bot")
KB documents  skipped (global KB migration not requested; pass --migrate-global-kb)
KB items      skipped (global KB migration not requested; pass --migrate-global-kb)
Workdir       /tmp/migrate-between-envs-abc123 (auto-cleaned)
Elapsed       8s

##########################################################

Preview without making changes (`--dry-run`)

Before writing anything to the target, run with --dry-run. The script probes both environments and prints exactly what would be migrated — but makes no changes to the target.

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --dry-run

Example output:

Project        5a54ca09-...  ->  5a54ca09-...
               "My Bot"  (12 flows, 3 NLUs)
KB             skipped — pass --migrate-global-kb to include the org KB

This is the safest way to confirm you have the right project, the right target UUID, and the right credentials before committing.

Copy within one environment

To clone a project inside the same environment, omit the --target-* options — they fall back to the source URL and key.

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://app.example.com --api-key at-MY_KEY \
  --target-project-id 9f3b7c21-44de-4a18-9c02-1e5d6b8f0a33

Use --target-project-id to give the copy a new UUID so it doesn't collide with the original.

Handling a UUID that already exists on the target

The project keeps its UUID by default. What happens when that UUID is already used on the target depends on who owns it:

Another organization owns it → the import fails with a 404. (The API returns 404 rather than 403 so it doesn't reveal that another tenant holds the UUID.)
The same organization owns it → the import silently overwrites the existing project with the source content. There is no confirmation prompt. This is the documented behaviour of the public import API.

⚠️
An overwrite cannot be undone. Before a real run, use --dry-run:
it now probes the target and tells you whether the target UUID already exists (and the name of the project that would be overwritten). To migrate as a new project instead of overwriting, pass a fresh UUID with --target-project-id:

./tools/migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --target-project-id 9f3b7c21-44de-4a18-9c02-1e5d6b8f0a33

See Troubleshooting for the cross-organization 404.

Optional project settings

Option	What it does
`--include-nlu-models`	Also copies the trained binary NLU models. Without it, the target keeps NLU definitions but must retrain. Use it when you want the target to be immediately usable without retraining.
`--publish-flows`	Publishes all flows on the target right after import, so the bot is live without a manual publish step.

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --include-nlu-models --publish-flows

--include-nlu-models requires the NLUs to be trained. If the project has an NLU with no trained data, the export aborts mid-download (curl: (92) … HTTP/2 … INTERNAL_ERROR) and the script stops with a clear error. Train the NLUs first, or omit --include-nlu-models and retrain on the target.

Migrating the knowledge base

⚠️
The knowledge base is organization-wide, not per-project.
Migrating it copies every KB document and item in the source organization and overwrites the knowledge base in the target organization. This is hard to roll back. Never run it against a shared target that holds another tenant's knowledge base.

⚠️
If your bot uses the knowledge base, it will not answer correctly on the target until the KB is migrated. A plain project migration (without
--migrate-global-kb) copies flows and configuration but not the KB — so a bot that looks up KB documents will return empty answers on the target. Either migrate the KB with the flag below, or make sure the target organization already holds an equivalent KB.

Because of that blast radius, KB migration is off by default. Opt in explicitly with --migrate-global-kb:

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --migrate-global-kb

We recommend pairing it with --dry-run first to see exactly which documents and how many items would be written:

.migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --migrate-global-kb --dry-run

Related options

Option	What it does
`--migrate-global-kb`	Required to migrate the KB at all. Off by default.
`--fail-fast`	Stop at the first document that fails to migrate, instead of continuing through the rest. Only relevant alongside `--migrate-global-kb`.

AI credentials

The script does not copy AI credentials. There is no credentials export in the platform, and secrets (API keys, tokens) are never included in a project or KB export. You must create the credentials by hand on the target before migrating.

KB documents reference embedding credentials. The script links each migrated document to a target credential by display name — so the only thing it does automatically is match an already-existing target credential to the source one by name. It does not create anything.

What you do:

List the credential names the source uses by running with --dry-run --migrate-global-kb — the output shows the credential each document maps to.
In the admin panel on the target, go to Settings → AI Credentials → Add new and create each of those credentials, giving every one the exact same name it has on the source (the secret value itself can differ — only the name is matched).
Run the migration.

If a name is missing on the target at run time, that document falls back to the target's default credential and the script logs a WARN.

Knowledge bases over 10 000 items

The server limits each item import to 10 000 rows. If your KB is larger, the import fails with FileTooBig. The script does not auto-split (line-based splitting can corrupt rows that contain quoted newlines). To handle a large KB, re-run with --workdir to keep the exported CSV, then split and import the chunks manually — the operator README has step-by-step commands.

After migration: what to check

The script copies data; it doesn't validate that the bot is fully working on the target. After a run, confirm:

NLU is trained. If you did not pass --include-nlu-models, the target has the NLU definitions but no trained models — open the project and retrain before going live.
Flows are published. If you did not pass --publish-flows, publish the flows manually in the admin panel; otherwise the bot still runs the previous version (or nothing).
The knowledge base is present. If the bot uses the KB, confirm you ran with --migrate-global-kb (or that the target already had the KB) and that the documents show up under the knowledge base on the target.
AI credentials resolved. Re-check the run output for any WARN: credential "<name>" not found on target lines. For LLM blocks in flows (LLMSayState / LLMIntegrationState), credential IDs are not rewritten — open those blocks on the target and reselect the credential if the environments use different IDs.
Smoke-test a conversation on the target to confirm intents, flows, and KB answers behave as expected.

Running with Docker

If you don't want to install bash/curl/jq locally, use the bundled Dockerfile for a consistent runtime. Build the image once:

docker build -t automate-migrate ./tools

Then run it with the same flags as the script (the script is the image's entry point):

docker run --rm automate-migrate 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY

To keep staged files (for example, the exported CSV for manual KB chunking), mount a local directory and point --workdir at it:

docker run --rm -v "$PWD/migrate-data:/work" automate-migrate \
  5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --workdir /work

Reading the summary

Every run ends with a summary block (printed even on failure):

Project       5a54ca09-...  ->  5a54ca09-...   OK ("My Bot")
KB documents  2 total, 2 OK, 0 failed
KB items      added 5, updated 3, unchanged 142
Workdir       /tmp/migrate-between-envs-abc123 (auto-cleaned)
Elapsed       8s

Project — source UUID → target UUID, and the result.
KB documents / KB items — counts when migrated, otherwise skipped.
Workdir — where files were staged (auto-cleaned unless you set --workdir).
Elapsed — total run time.

Exit codes

Code	Meaning
`0`	Success.
`1`	An operational failure — network, authentication, a server-side rejection, or a failed KB document. Check the summary and the last `ERROR:` line.
`2`	Invalid command-line usage (e.g. a malformed UUID or a missing required option).

Troubleshooting

Symptom	Cause	What to do
`project import failed` → `404`	The UUID already exists on the target in a different organization.	Use the API key of the organization that owns the UUID on the target, or pass `--target-project-id <fresh-uuid>`, or delete the existing project on the target.
Existing target project replaced without warning	The UUID already exists in the same organization — the import overwrites it silently (documented API behaviour).	Run `--dry-run` first (it flags an existing target UUID), and pass `--target-project-id <fresh-uuid>` to migrate as a new project instead.
`curl: (92) … HTTP/2 … INTERNAL_ERROR` / `exported project is not a valid ZIP`	Used `--include-nlu-models` on a project whose NLU has no trained data; the export aborts mid-stream.	Train the NLUs first, or drop `--include-nlu-models` and retrain on the target.
`FileTooBig: Maximum allowed: 10000`	The knowledge base exceeds the per-import row cap.	See Knowledge bases over 10 000 items.
`curl: (7) Failed to connect ...`	Wrong URL, DNS, or firewall.	Verify the `--url` / `--target-url` values. Add `--debug` to see the exact requests.
`Invalid <project-uuid>` (exit 2)	The UUID is mistyped.	Re-copy it from the admin panel address bar.
The run hangs	Should not happen — requests have built-in timeouts.	Re-run with `--debug` and share the output with support.

Getting more detail

Add --debug to print every HTTP request to the terminal. API keys are masked in the output, but URLs are shown verbatim — review before pasting debug output into a ticket or chat.

./tools/migrate-project-between-envs.sh 5a54ca09-1c3e-4f9a-bb21-7d0e2f6c8a10 \
  --url https://staging.example.com  --api-key at-SOURCE_KEY \
  --target-url https://prod.example.com --target-api-key at-TARGET_KEY \
  --debug

Quick reference

.migrate-project-between-envs.sh <project-uuid> \
  --url URL --api-key KEY \
  [--target-url URL] [--target-api-key KEY] [--target-project-id UUID] \
  [--include-nlu-models] [--publish-flows] \
  [--migrate-global-kb] [--fail-fast] \
  [--dry-run] [--workdir DIR] [--debug]

Option	Default	Effect
`--url URL`	required	Source environment base URL.
`--api-key KEY`	required	Source API key (`X-Api-Key`).
`--target-url URL`	`--url`	Target environment base URL.
`--target-api-key KEY`	`--api-key`	Target API key.
`--target-project-id UUID`	source UUID	UUID for the project on the target.
`--include-nlu-models`	off	Include trained binary NLU models.
`--publish-flows`	off	Publish all flows on the target after import.
`--migrate-global-kb`	off	Also migrate the org-wide knowledge base. ⚠️ Overwrites the target org's entire KB.
`--fail-fast`	off	Stop at the first failed KB document.
`--dry-run`	off	Preview only; no writes to the target.
`--workdir DIR`	temp dir	Where to stage downloaded files.
`--debug`	off	Print every HTTP request (keys masked, URLs shown).

Migrating a project between environments

Before you start

Find the project UUID

Get an API key

Your first migration

Preview without making changes (`--dry-run`)

Copy within one environment

Handling a UUID that already exists on the target

An overwrite cannot be undone. Before a real run, use `--dry-run`:

Optional project settings

Migrating the knowledge base

The knowledge base is organization-wide, not per-project.

If your bot uses the knowledge base, it will not answer correctly on the target until the KB is migrated. A plain project migration (without

Related options

AI credentials

Knowledge bases over 10 000 items

After migration: what to check

Running with Docker

Reading the summary

Exit codes

Troubleshooting

Getting more detail

Quick reference

Before you start

Find the project UUID

Get an API key

Your first migration

Preview without making changes (--dry-run)

Copy within one environment

Handling a UUID that already exists on the target

An overwrite cannot be undone. Before a real run, use --dry-run:

Optional project settings

Migrating the knowledge base

The knowledge base is organization-wide, not per-project.

If your bot uses the knowledge base, it will not answer correctly on the target until the KB is migrated. A plain project migration (without

Related options

AI credentials

Knowledge bases over 10 000 items

After migration: what to check

Running with Docker

Reading the summary

Exit codes

Troubleshooting

Getting more detail

Quick reference

Preview without making changes (`--dry-run`)

An overwrite cannot be undone. Before a real run, use `--dry-run`: