---
title: Known Limitations
icon: phosphor-duotone:warning
summary: The honest gaps - what is proven against a real running app and what is not.
description: The honest gaps - what is proven against a real running app and what is not.
tags: [reference, limitations]
---

# Known Limitations

BX Agents is under active development. This page tracks the honest gaps - what's tested against a real running app, what still only runs against bx-ai's `"mock"` provider, and real upstream quirks this project ran into.

## Testing runs against the `mock` provider only

Every fast-lane spec (build pipeline, generators, CLI verbs) and the [ColdBox integration suite](#real-coldbox-integration-testing) exercise bx-ai's built-in `"mock"` provider - never a real network call to an LLM. This is deliberate (fast, free, deterministic CI), but it means no automated test currently proves a real provider (OpenAI, Anthropic, etc.) actually round-trips correctly end-to-end. Do at least one manual `chat`/`serve` run against a real provider and a throwaway API key before depending on this in production.

## Real ColdBox integration testing

`./gradlew testColdBoxIntegration` boots a **real** `boxlang-miniserver` process against a generated app's own `Application.bx`/`Bootstrap`, and makes a genuine HTTP request through a `toAi()`-registered route. This is the strongest proof point in the suite - it caught three real bugs in `ColdBoxAppGenerator.bx` during development (a missing WireBox `Binder` `extends`, wrong `Bootstrap` constructor argument order, and a bare `getInstance()` call that doesn't exist on `Binder`).

### The `toAi()` first-request race

The **very first** HTTP request to a freshly booted app's `toAi()` route can transiently fail with "Function [getInstance] not found" - a genuine ColdBox/WireBox lazy-injection race on the Router's own `getInstance` delegate, not a BX Agents bug. It succeeds reliably on every request after something else has already forced WireBox to build the `GeneratedAgent` singleton once (a health check, a `chat` session, another route). **Send a warm-up request** before relying on a freshly deployed `toAi()` route under load.

## Three integration checks - closed via a different route than originally planned

`tests/specs/integration/RuntimeStartupSmokeSpec.bx` still has three `xit()`s, since this file runs via the CLI runner (`runTests.bxs`/`testBx`), and BoxLang's CLI mode never gives a `cgi` scope - which ColdBox's RoutingService needs even to load the router at startup. That's a structural fact about CLI mode, not a gap: all three checks are now proven for real elsewhere, in `tests/specs/integration/coldbox/ColdBoxRuntimeSpec.bx` (which runs inside a real HTTP request served by a real `boxlang-miniserver` process):

- A real ColdBox-routed HTTP request reaching the generated agent end-to-end - proven by `ColdBoxRuntimeSpec.bx` plus `runColdBoxIntegrationTests.bxs`'s own `POST /api/chat/invoke` assertion.
- `schedules/*` actually registering with a **live** ColdBox `Scheduler` - proven via `SchedulerService.getSchedulers()["appScheduler@coldbox"].hasTask(...)` against a real boot, without waiting out an actual cron fire (the coarsest supported granularity is 1 minute, which would tax every CI run for marginal extra proof once the task is confirmed live and registered).
- `chat` and a `serve`d HTTP route never diverging - corrected from its original wording (`chat` deliberately never boots WireBox at all, so it can never share WireBox's singleton *object* by design) to what actually matters: instantiating `GeneratedAgentFactory` outside of WireBox produces a behaviorally-equivalent agent to WireBox's own singleton.

## Real OS-process CLI testing - and a real bug it found

`ModuleCliProcessTest.java` spawns genuine `java -jar <boxlang-jar> module:bxagents <verb> ...` child processes against a copy of the actual installable module structure (`build/modules/bxagents`), pointed at a real `modulesDirectory` - exactly how a real BoxLang installation loads this module. Every other CLI spec calls `ModuleConfig.main()`/each verb's `run()` in-process instead, which is faster but never proves the module resolves correctly once genuinely installed.

That gap was real: this test caught every CLI verb failing with "class not located" the moment it ran through an actual installed-module process, because internal cross-references used a bare `bxagents.models....` dotted path that only ever resolved thanks to this repo's own dev/test `boxlang.json` (a hand-declared `/bxagents` mapping) - a genuinely installed module never gets that mapping. Fixed by switching every nested class's internal references to the `Class@bxagents` module-relative suffix form (`ModuleConfig.bx` itself, sitting at the module root, resolves plain relative paths fine and needed no change) - see `BuildPipeline.bx`'s `init()` docblock for the full explanation. `build.gradle`'s module-structure output moved from `build/module` to `build/modules/bxagents` (folder name must equal the module name for `modulesDirectory` discovery to find it at all) and the dev/test `boxlang.json` now also loads it as a real module, so the whole existing suite exercises the same resolution path production does, not just the convenience mapping.

## A real, foundational bug found building the testing framework: agents never received their own tools

Building `BaseAgentSpec`'s `toHaveCalledTool` matcher (M15) surfaced a serious, previously-undiscovered bug: `ColdBoxAppGenerator`'s generated `aiAgent()` call never passed a `tools:` argument at all - confirmed against real bx-ai source (`AiAgent.bx` never references `aiToolRegistry()` internally). A project's `tools/` were copied into the build and made name-resolvable for MCP wiring, but **no agent BX Agents ever built - in any context: a real served app, `chat`, or a test spec - actually received its own declared tools.** This had gone undetected because no existing test ever asserted on a real tool invocation, only on a non-empty response.

Fixed in `ColdBoxAppGenerator.renderAgentFactory()`: every generated `GeneratedAgentFactory.bx` now loads its own `tools/` directory via a new `ToolRegistryLoader.bx` (using an ABSOLUTE path embedded at generation time, not a relative one depending on whatever "/" mapping happens to be in effect for the loading context - confirmed that `aiToolRegistry().scan("tools")`'s own relative-path resolution silently fails when called from a `DynamicClassLoader`-loaded context like `chat`), then passes `tools: aiToolRegistry().getAll()` to every `aiAgent()` call.

A third top-level-script pitfall, found the hard way in CI: **`var` cannot be used at the top level of a `.bxs` script.** `var` declares into the `local` scope, which only exists inside a function, so a top-level `var x = ...` throws `Scope [local] is not available in this context` at runtime - not at parse time, so it survives review and only fires on the code path that reaches it. It cost a CI cycle in `runColdBoxIntegrationTests.bxs`, where the offending line sat inside a failure-diagnostics branch and therefore crashed exactly when something else had already gone wrong, replacing the real failure with its own. Two rules follow: never write `var` outside a function in a `.bxs`, and wrap diagnostic output in its own try/catch so it can never mask the failure it is describing.

**The Application setting for a default datasource is `this.datasource`, NOT `this.defaultDatasource`.** The plural registration key really is `this.datasources[ "name" ] = { ... }`, so `this.defaultDatasource` is the name everyone reaches for - and BoxLang accepts it silently and does nothing with it. Verified directly against the runtime rather than inferred: with `this.defaultDatasource = "testds"` set, an unqualified `queryExecute()` still fails with `No default datasource defined in the application or globally or in the query options. Registered datasources are: [testds]`; changing the single line to `this.datasource = "testds"` resolves the datasource and the call proceeds to a real driver concern. There is no warning and no hint in the error that the setting you wrote was ignored - the message names the registered datasource you are trying to select, which reads as though the selection mechanism is broken rather than misspelled.

**qb's `moduleSettings.qb.defaultOptions` did not reach `QueryBuilder` in a real ColdBox boot.** The generated `config/ColdBox.bx` sets `moduleSettings.qb.defaultOptions = { datasource : "<name>" }`, and qb's own `ModuleConfig.cfc` maps `QueryBuilder@qb` with `.initArg( name = "defaultOptions", value = settings.defaultOptions )` inside `onLoad()` - which reads as correct, and is why the generated `ChatDb.query()` originally named no datasource. It did not work: every query failed with `No default datasource defined in the application or globally or in the query options. Registered datasources are: [<name>]`, i.e. the datasource was registered and the builder still had empty options. Why the module setting did not arrive was not run to ground - it stopped mattering, because naming the datasource explicitly is better than depending on that plumbing regardless. `ChatDb.query()` now calls `.mergeDefaultOptions( { datasource : static.DATASOURCE } )` on every builder, mirroring what `schemaOptions()` already had to do for `SchemaBuilder` (which qb never passes `defaultOptions` to at all). If you add a new qb call path to generated code, name the datasource on it; do not assume the module setting covers you.

**A generated app now declares `this.defaultDatasource`, not just `this.datasources`.** With one datasource, anything that runs a query without naming one - ColdBox itself, another module, a project's own code - should reach it instead of failing. Nothing asserted on the emitted datasource block before, which is how it shipped without a default; `ColdBoxAppGeneratorSpec` now covers both the web-UI case and the no-web-UI case (where neither line should appear).

**The generated `models/ChatDb.bx` did not compile, and no unit spec noticed.** `WebUiGenerator` builds BoxLang source out of BoxLang string literals, where a literal double quote is written as a doubled quote - so an empty-string literal needs FOUR quote characters, and writing the natural-looking two emits a single stray quote that opens a string and swallows the rest of the line. Two template lines carrying an elvis-to-empty-string (`?: ""`) emitted `?: "`, so every generated web-UI project shipped a `ChatDb.bx` that failed to parse. Every `WebUiGeneratorSpec` case asserted on substrings of the generated text, which all still matched; only a real ColdBox boot ever tried to *compile* it, and that path was itself broken for several cycles by the harness bugs described below, so the real bug stayed hidden behind them. There is now a `WebUiGeneratorSpec` case asserting that every line of generated source has an even number of double quotes - an invariant that catches the whole class rather than the two lines that happened to be wrong. The wider lesson: a generator spec that only greps its own output does not test that the output is valid code.

**`GET /chat/api/health` proves far less than it looks like it does.** The generated `health()` action renders a literal `{ status: "ok", success: true }` and touches nothing else - so a 200 proves ColdBox booted and routing reaches the generated `ChatUi` handler, and says nothing about WireBox, `ChatDb` or SQLite. The comment in `runColdBoxIntegrationTests.bxs` used to claim it exercised "ColdBox routing -> ChatUi -> WireBox -> ChatDb", which is false and gave real false confidence: one CI run returned a green probe while `models/ChatDb.bx` could not compile at all. Corrected in place; treat the integration specs, not the probe, as the store's coverage.

**TestBox's `JSONReporter` cannot report on specs that touch live framework singletons.** The ColdBox integration specs resolve real `controller`/WireBox/qb objects, and those object references end up in the TestBox result memento. `JSONReporter` serializes the memento wholesale, so BoxLang reflects over the live objects (`StructUtil.objectToStruct` -> `DynamicInteropService.getMethodNames`) and follows their cyclic references until the JVM throws `StackOverflowError`. The specs themselves had already run green when this fired, which is exactly what made it confusing: a passing suite reported as a failed run. `tests/runner-coldbox.bxm` therefore assembles its own report instead, forcing every value through a `toScalar()` helper so no object can reach the serializer regardless of what a spec leaves behind, and recording the pass/fail counts in the progress markers *before* the report is built - so a serialization failure can never again be mistaken for a test failure.

**In a BoxLang time mask, `nn` is NANOSECONDS, not minutes - use `mm`.** `dateTimeFormat( now(), "HH:nn:ss" )` silently produced timestamps like `02:777491298:32`, which read as corrupt output rather than a wrong mask. (Note the inverse trap in a date mask, where `mm` is the month: `yyyy-mm-dd` yields the minute, not the day.)

**Deriving an identity server-side is not the same as authorizing against it.** The generated `handlers/ChatUi.bx` took care to derive `userId` from the session on every action and never from the request body - and two actions still had an authorization hole, because they were addressed by `threadId` rather than by the caller. `/pending` and `/resume` loaded a checkpoint by id and acted on it; `/resume` even derived `decidedBy` from the session, which reads like a scoping check but only governs the *label on the decision*, not *whose run is being decided*. A visitor with someone else's `threadId` could read their pending tool calls and answer their approvals. Both now compare the caller against the `userId` the agent checkpointed in the run options. The general rule this is worth remembering for: when a route is keyed by an opaque id rather than by the caller, deriving identity buys you attribution, not access control - the two need separate thought, and a server-derived value sitting in the same function can easily be mistaken for both.

**`ProjectValidator`'s path checks must consider `..`, not just a leading separator.** `database.path` is spliced into an `expandPath()` call in the generated `Application.bx`, so it was checked for being absolute - and `../../var/lib/chat.db` escapes the app directory exactly as effectively while passing that check cleanly. Now rejected on both separators, matching whole path segments so legitimate names like `..hidden` or `a..b` still pass. Any future validator that guards a generated path needs both halves of this check.

**Running the test suite locally needs bx-ai overlaid from source, exactly as CI does.** `./gradlew downloadModules` fetches the *published* bx-ai snapshot, which lags its own development branch - it overwrites the vendored copy under `src/test/resources/modules/bxai` and the suite then fails ~40 specs with `Method 'isRunning' not found` and `The method aiGatewayRegistry does not exist`, both APIs that exist upstream but not in the published build. That is not a regression, and it is easy to misread as one. Rebuild and overlay before running `testBx`: `( cd <bx-ai checkout> && ./gradlew createModuleStructure )` then `cp -R <bx-ai>/build/module/. src/test/resources/modules/bxai/`. `tests/coldbox/`, `tests/testbox/` and `tests/qb/` come from `box install` (run it in `tests/`), and all three are gitignored.

**`chr()` does not exist in BoxLang - the BIF is `char()`.** This is not context-dependent, despite a comment in `ModuleConfig.bx` that used to claim it was unavailable only "in this module-CLI execution context" (now corrected). Verified directly against the runtime: `char( 10 )` returns the newline, `chr( 10 )` throws `Function [chr] not found`, in a plain CLI script and in a served template alike. It is worth knowing because the failure is a *runtime* one - `chr()` parses fine and survives review, then throws on whatever code path first reaches it. It cost several CI cycles here: `tests/runner-coldbox.bxm` called `chr( 10 )` in its very first progress marker, so every request 500'd on entry, and the retry loop described below turned that plain, one-line error into an opaque "HTTP 408, never responded".

A fourth lesson from the same CI harness, and the one that cost the most cycles: **instrumentation that a retry can overwrite records nothing.** `tests/runner-coldbox.bxm` opened by truncating its own progress file (`fileWrite( progressFile, "" )`), and `runColdBoxIntegrationTests.bxs` retried the runner request once a second until a deadline. So every retry wiped the markers the hung first attempt had written, and the failure diagnostics faithfully printed an empty file - which read as "the page never got anywhere" when the truth was "the evidence was deleted". Worse, the retries were actively harmful independent of the diagnostics: the runner page is neither cheap nor idempotent (it boots TestBox and runs every integration spec), so retrying it stacked concurrent full test runs onto a MiniServer worker pool of only a handful of threads - a way to cause a hang, not to recover from one. Both are fixed: the orchestrator clears the marker file once and makes exactly one request (the health probe ahead of it has already proven the server is up, so there is nothing for a retry to wait for), and the page only ever appends, tagging each line with a per-request UUID so overlapping attempts stay distinguishable. The general rule: a progress log must be append-only and must outlive the thing it is instrumenting, and anything that is expensive or stateful should not sit behind a retry loop at all.

A related, separate BoxLang pitfall found while diagnosing this: **`request` is a reserved built-in scope name.** A local/loop variable named `request` can silently shadow it - `for ( var request in someArray ) { request.someKey }` iterated the correct number of times, but every `request.someKey` access inside it silently read as the empty built-in scope instead of the loop variable, with no error at all. Fixed by renaming to `recordedRequest` in `BaseAgentSpec.bx`'s matchers - worth remembering for any future BoxLang code in this project that loops over anything sensibly named `request`.

## `serve`'s miniserver lookup is PATH-only

`serve` looks for `boxlang-miniserver` only on `PATH` (`MiniServerLauncher.findExecutable()`). There's no fallback to a configured path or a bundled binary - if it isn't installed and on `PATH`, `serve` fails with a clear, actionable error, but there's no alternate way to point it at one. `invoke --server` reuses `serve` internally, so it inherits the same PATH-only lookup - its own `InvokeSpec.bx` test for the real HTTP round-trip checks `MiniServerLauncher.findExecutable()` first and skips (rather than fails) when no real binary is on `PATH`, the same idiom `RuntimeStartupSmokeSpec.bx` already uses for its own jar-presence check. Do at least one manual `bxAgents invoke --message=... --server` run on a machine with `boxlang-miniserver` installed before depending on the real HTTP path in production - the same honest framing already used elsewhere in this page for gaps this suite can't close itself in every environment.

## The scoped BoxLang runtime home reaches `serve`/`invoke --server` unconditionally; the in-process verbs only if your BoxLang install loads `.env`

`serve` scopes the miniserver's own BoxLang runtime home to `.build/runtime` via `serverHome` (a real, confirmed `MiniServerConfig` field - `boxlang-web`'s own `MiniServer` CLI help text: `-s, --serverHome <PATH>  BoxLang server home directory (default: ~/.boxlang)`), so each project's compiled-class cache and any config overrides are isolated per-project rather than shared globally at `~/.boxlang`. `invoke --server` inherits this since it reuses `serve` internally. This part is unconditional - we write the launch config for that process ourselves.

`chat`, `build`, `test`, and default (in-process) `invoke` are different: they run inside the already-started `bxAgents` process, whose own BoxLang runtime - and therefore its home - was resolved before any of our BoxLang code, including `ModuleConfig.bx`'s own `main()`, ever got a chance to run (the engine has to exist to interpret that code at all). `BoxRuntime` is a JVM-wide singleton; its home is fixed at first initialization, so nothing a verb class does from inside that already-running process can change it retroactively.

There IS a real lever for this, though: `BoxRunner` (BoxLang's own core CLI entry point, not just the miniserver) reads a genuine `BOXLANG_HOME` environment variable before `BoxRuntime` initializes - confirmed directly against the real runtime jar (not just documentation): running it with `BOXLANG_HOME=<path>` set as a real OS environment variable (relative paths resolve against the CWD, same as an absolute path) populates the full home structure at that path instead of `~/.boxlang`, every time. `new` scaffolds a `.env` declaring `BOXLANG_HOME=.build/runtime` - the same path `serve` uses - for exactly this reason (mirroring `ortus-boxlang/bx-ai-intro`'s own `.env`/`BOXLANG_HOME` convention), at the **project root** - i.e. the directory a user actually runs `bxAgents <verb>` from for the in-process verbs, which is the right place for a CWD-based `.env` loader to find it.

**What we confirmed, and what we couldn't.** `boxlang-miniserver` (what `serve` launches) does have real, built-in `.env` auto-loading - confirmed by decompiling and then actually running `ortus.boxlang.web.MiniServer` directly: with no `envFile` configured, it resolves `.env` relative to the server's **webRoot** (not the project root) and, if found, loads it as Java `Properties`, applying each key via `System.setProperty()` - real values loaded this way ARE visible to BoxLang code via `getSystemSetting()`. The raw core runtime jar (`BoxRunner`, used by `chat`/`build`/`test`/default `invoke`) has no equivalent logic anywhere in it (confirmed by grepping every class in the jar for `.env`) - so those verbs only pick up `.env` if the actual `boxlang` CLI you have installed (a BVM-provided native launcher, not this raw jar) does its own `.env` loading before the JVM starts, the way `ortus-boxlang/bx-ai-intro` relies on. That's plausible and consistent with real-world usage of that project, but isn't something this sandbox has the real binary to verify directly.

One specific, confirmed gotcha even where `.env` loading does happen: **`BOXLANG_HOME` itself does not take effect unless it's a real OS environment variable** - not a JVM system property, and not a `-D` flag. Verified twice directly against the real jars: (1) running `boxlang-miniserver` against a webRoot whose `.env` declared `BOXLANG_HOME=customhome` loaded the file (confirmed via its own "Loaded environment variables from:" log line and via `getSystemSetting()` correctly returning other `.env` values), yet the server still logged `Logs Directory: /root/.boxlang/logs` - the default home, not `customhome`; (2) launching `BoxRunner` directly with `-DBOXLANG_HOME=<path>` (a JVM system property, no `.env` involved) also had no effect on the resolved home, even though `getSystemSetting( "BOXLANG_HOME" )` happily returned the flag's value from BoxLang code. So `BOXLANG_HOME` resolution specifically reads only the true OS environment variable - unlike most settings, `getSystemSetting()` returning a value for it doesn't mean the runtime home actually moved. If your `boxlang` CLI's `.env` loader works the same way MiniServer's does internally (`System.setProperty` after the JVM has already started) rather than exporting a real env var before launching the JVM, `BOXLANG_HOME` in `.env` won't reach the runtime home even though `.env` itself loaded successfully - everything else in it will still work. If your install doesn't give you this at all, source it yourself before running commands (e.g. `set -a; source .env; set +a`) to get the same isolation `serve` already gets unconditionally.

## `new`'s `box install` convenience step isn't exercised by an automated test

`new` runs `box install` inside the scaffolded `tests/` folder by default, so `bxAgents test` works immediately (see [CLI Reference](cli-reference.md)). This is a real network operation (CommandBox resolves `testbox` against ForgeBox) - confirmed to take ~25 seconds and fail with a certificate error in this development sandbox specifically, the same ForgeBox-unreachable constraint already noted elsewhere in this project's own tooling. `NewSpec.bx` only exercises the fast, deterministic `--skipInstall` path (and the OS-process `ModuleCliProcessTest.java`/in-process `ModuleConfigCliSpec.bx` invocations of `new` also pass `--skipInstall` for the same reason) - the default install-attempt path itself is verified by manual testing only, not CI.

## `chat` needs a real TTY

`chat` uses BoxLang's own `MiniConsole`, which shells out to `stty` to set up raw terminal mode - it can only run against a genuine interactive terminal. It doesn't work piped, redirected, or from a non-interactive process (a CI job, a script). There's no non-interactive fallback mode.

## Fixed: `chat` and default (non-`--server`) `invoke` used to fail for a class-based `Agent.bx`

Previously, both `chat` and default `invoke` threw `The requested class [agent.classes.agentClass] has not been located in any class resolver.` before ever reaching the agent. Root cause: both verbs load the generated `GeneratedAgentFactory.bx` in-process via `DynamicClassLoader.instantiate()` (a raw `RunnableLoader` call against an absolute path, no ColdBox container involved), and the generated factory used to instantiate a class-based `Agent.bx` via a **relative** dotted-path lookup, `new "agent.classes.agentClass"()` - which only resolves once something has registered a mapping making the app root resolvable, and nothing did outside a real ColdBox boot.

Registering one mid-script (`Configuration.registerMapping( "/", appDir )` right before `DynamicClassLoader.instantiate()`) does **not** fix it either - confirmed empirically by hand-rolling the exact same sequence in a standalone `.bxs` script. Same class of limitation already documented above for `TestRunnerLauncher`'s TestBox discovery: a mapping registered via `Configuration.registerMapping()` mid-script does not reliably propagate to a class's own relative-path lookups made within that same process.

**Fix:** `ColdBoxAppGenerator.copyAgentClass()` now returns the copied class's own absolute file path (instead of a dotted component path), and `renderClassBasedAgentStatement()` instantiates it via `DynamicClassLoader.instantiate( absolutePath, context )` - the exact same primitive `chat`/`invoke` already use to load `GeneratedAgentFactory.bx` itself - rather than a relative `new "..."()`. This sidesteps mapping resolution entirely, so it now works identically whether or not a real ColdBox container is booted. Confirmed against `examples/class-based-agent/`: `chat`, default `invoke`, `invoke --server`, and `serve` all now build and run the same agent correctly.

## No box.json `executable` install smoke test

`box.json` declares `"boxlang": { "executable": "bxAgents" }` so a real module install produces a native `bxAgents` command (see [Installation](getting-started/installation.md)). This wiring itself isn't exercised by an automated test - it relies on the BoxLang module installer's own documented behavior for generating executable wrappers, verified by reading its source, not by an install-and-run test in this repo's own CI.

## `schedules/Scheduler.bx` isn't validated at build time

Since it's real, hand-written ColdBox code passed through untouched (see [schedules/](conventions/schedules.md)), `build` can't meaningfully check it the way `{ cron, action }` config used to be checked - a syntax error, a typo in a `getInstance( "..." )` call, or referencing an agent name that doesn't exist all pass `build` cleanly and only surface when the generated app actually boots (`serve`), same as any other real BoxLang class this project doesn't own the contents of. `build` does catch one narrower, adjacent mistake: two agents (root or subagent, any depth) declaring the same `name` - that's a `config/WireBox.bx` binding collision this project generates itself, so it's checked at validation time same as everything else it generates.

## A real bug found building the `test` verb: resolving "the current BoxLang jar" is ambiguous once `boxlang-miniserver` is on the classpath

`TestRunnerService.bx` spawns a fresh child process to run a project's `tests/specs`, and needs to know which jar to launch it with. The first implementation used the standard "which jar was this class loaded from" trick (`BoxRuntime.class.getProtectionDomain().getCodeSource().getLocation()`) - this worked in isolated manual testing, but failed unpredictably once run as part of this project's own full `testBx` suite, which also needs `boxlang-miniserver-*.jar` on the classpath (for `serve`/`MiniServerLauncher`-related specs). Confirmed via direct inspection: **`boxlang-miniserver-*.jar` is a fat jar that bundles its own copy of `ortus.boxlang.runtime.BoxRuntime`** - with both jars on the classpath, the classloader can resolve `BoxRuntime.class` to the miniserver jar instead of the real runtime jar, silently launching `MiniServer.main` (which rejects `--bx-config` and exits 1 immediately, before any BoxLang script or test report is ever produced) instead of `BoxRunner.main`. This surfaced as `tests/specs/cli/TestSpec.bx`'s real-process cases failing with `exitCode=1` and an empty report - reproducible only when run through the full suite, not in isolation, which is what made it easy to miss.

Fixed by resolving the jar from `java.class.path` instead - scanning for a `boxlang-*.jar` entry that does **not** contain `miniserver`, falling back to the old codeSource trick only if no such entry is found.

## `deploy`'s `ssh`/`docker`/`digitalocean` targets aren't exercised by an automated real-process test

`SshTargetSpec.bx`/`DockerTargetSpec.bx` assert on the exact `scp`/`ssh`/`docker` command each target builds (a real `ProcessBuilder` argument array) without ever invoking the real binary - the same "capture, don't execute" approach used elsewhere in this suite for anything that needs a binary that might not be on `PATH` in CI (see `MiniServerLauncherTest`'s `assumeTrue` skips). `local`'s copy logic IS exercised for real (`LocalTargetSpec.bx`), including a regression test for a real latent bug this refactor fixed: the original `Deploy.bx` picked the "newest" `.bxa` via a lexical filename sort, which silently mis-picks once a project reaches double-digit versions (`v9.0.0` sorts after `v10.0.0`) - fixed by sorting on actual file modification time (`DistArtifactLocator`).

`DigitalOceanTargetSpec.bx` similarly only unit-tests the pure `buildAppSpec()`/`findExistingAppId()` logic - the real `GET/POST /v2/apps` calls are never exercised in CI, since that needs a live DigitalOcean account and API token. Do at least one manual `deploy --name=<ssh-entry>` against a real disposable VM, and one `deploy --name=<digitalocean-entry>` against a real DO account with a throwaway app, before depending on either in production - the same honest framing already used above for the mock-provider-only testing gap.

## `deploy`'s `ftp`/`sftp` targets: real connection handling proven, a real successful upload isn't

Unlike `ssh`/`docker` (which shell out to external binaries and so can only have their *command construction* tested without a real server), `ftp`/`sftp` call the real [`bx-ftp`](https://github.com/ortus-boxlang/bx-ftp) module's `bx:ftp` component **in-process** - there's no "capture the command, don't execute" option. `BaseFtpTargetSpec.bx` instead makes a genuine connection attempt against `127.0.0.1` on a port nothing listens on, and asserts that the real refused-connection error is caught and re-thrown as a clear `BxAgents.DeployFailed` (confirmed via bx-ftp's own source that every action throws on failure rather than returning a soft `succeeded: false`). This proves the real connect/error-wrap/cleanup path end-to-end - what it can't prove is a real successful upload.

That gap is specifically because **this development sandbox has no outbound raw TCP egress at all** - only HTTPS through the environment's proxy (confirmed directly: `curl ftp://test.rebex.net` and a raw `/dev/tcp` connect to a public FTP host both hung and timed out, and `docker info` shows no daemon running, so bx-ftp's own bundled Docker FTP/SFTP test servers couldn't be started here either). This is a sandbox constraint, not a code limitation - do at least one manual `deploy --name=<ftp-entry>` and one `deploy --name=<sftp-entry>` against a real reachable server (or bx-ftp's own `docker-compose up` test servers, from a machine that has Docker/network access) before depending on either in production, the same honest framing already used above for `ssh`/`digitalocean`.

## `build` doesn't roll back a partially-written `.build/app` on a mid-generation crash

`BuildPipeline.build()` deletes and recreates `.build/app` up front, then runs Phase 5's generators in sequence. Every input that Phase 3 (`ProjectValidator`) can check is checked before any of that happens, so a genuine mid-generation crash should be rare in practice - but if one still occurs (e.g. a `models/`/`schedules/`/`mcp/` entry that fails to load, or an environment/filesystem problem), `.build/app` is left on disk in a partially-written state rather than restored to its prior contents or cleaned up. A subsequent successful `build` overwrites it cleanly, so this isn't sticky, but anything that inspects `.build/app` between a failed build and the next one (a CI step, a manual `package` retry) can see a broken half-generated app. There is no rollback/temp-directory-then-swap step yet.

## An intermittent `StackOverflowError` was observed in `testBx`, unrelated to this project's own code

While investigating this milestone, `./gradlew testBx` occasionally (not every run) crashed the whole JVM with a `StackOverflowError` inside the BoxLang engine's own generic-object JSON serialization (`DynamicObjectSerializer`/`BoxStructSerializer` calling each other, alternating, until the stack is exhausted - confirmed to still happen with `-Xss16m`, so it's a genuine cycle, not merely a deep-but-finite structure). Isolated via `git stash` (the crash reproduced identically with none of this session's changes applied, against the exact same commit already on `development`) and via bisecting `tests/specs/**` into every subdirectory both individually and combined (every subset, and even every subset-but-one, ran clean) - only the single full run occasionally reproduces it, and re-running the identical full suite immediately after a crash sometimes passes clean. This points to a timing-dependent race (plausibly interacting with `ExampleScheduler`'s real background `everySecond()` task, which prints from its own thread pool concurrently with whichever spec happens to be running at that moment) rather than a bug in any one spec or in this project's own generated code. If `testBx` fails with a `StackOverflowError` and no other explanation, retry it before assuming a real regression - it is not currently reproducible on demand, so no automated regression test exists for it, and running it down further to a single root cause was out of scope for this round.

## Push-style gateways (Telegram) are tested against a mocked API/scheduler seam only - no live platform integration runs in CI

`TelegramGatewaySpec.bx` exercises `TelegramGateway`'s own logic (inbound normalization, outbound chunking at the 4096-char limit, HITL inline-keyboard building, scheduler task registration/removal) entirely against an injectable `apiCaller`/`setScheduler()` test seam - never a real Telegram Bot API call, and never a real ColdBox scheduler boot. This proves the gateway's own code is correct, not that it actually works end-to-end against Telegram's real API or a real running scheduler. Do at least one manual `bxAgents serve` against a project with a real `botTokenEnvVar`-backed Telegram bot before depending on it in production - the same honest framing already used above for the mock-provider-only/no-live-connection testing gaps elsewhere in this file. The same caveat will apply to every future push-style gateway (Slack, Discord, Email, WhatsApp) built the same way.

`SlackGatewaySpec.bx` carries the same gap, one level deeper: `SlackGateway`'s persistent websocket connection is tested via an injectable `setSocketOpener()` seam (a fake object standing in for the real `java.net.http.WebSocket`), so every frame-handling/reconnect-logic assertion in the spec runs with zero real network I/O. What genuinely was verified directly (not mocked): a standalone smoke test instantiated the real `SlackSocketListener(gateway)` (which `implements="java:java.net.http.WebSocket$Listener"` directly - BoxLang compiles it as a genuine JVM implementer, no proxy needed) and called the real `HttpClient.newWebSocketBuilder().buildAsync(...)` against an unreachable address, confirming the BoxLang-to-Java interop itself works correctly all the way to the network boundary (it failed with a plain `java.net.ConnectException`, not a casting/interop error) - but no test here has ever completed a real Socket Mode handshake against Slack's actual servers. Do at least one manual `bxAgents serve` against a project with real `botTokenEnvVar`/`appTokenEnvVar`-backed Slack app credentials before depending on it in production.

`DiscordGatewaySpec.bx` carries the identical gap, for the identical reason: `DiscordGateway`'s frame-handling/heartbeat/reconnect logic is exercised entirely against injectable `setApiCaller()`/`setSocketOpener()` seams, zero real network I/O. The same standalone smoke-test discipline was applied here too - `gateway.onConnect()` driven against a real `HttpClient.newWebSocketBuilder().buildAsync(...)` call to an unreachable address failed with a plain `java.net.ConnectException`, not a casting/interop error, confirming the interop chain works. What was NOT verified: a real Gateway handshake (`Hello` → `Identify` → `READY`) against Discord's actual servers, real heartbeat timing under Discord's own tolerance, or that the default `intents` value (`GUILDS`+`GUILD_MESSAGES`+`DIRECT_MESSAGES`+`MESSAGE_CONTENT` = `37377`) is actually enough to receive message content once `MESSAGE_CONTENT` is enabled/approved for a real bot in the Discord Developer Portal. Do at least one manual `bxAgents serve` against a project with a real `botTokenEnvVar`-backed Discord bot (with `MESSAGE_CONTENT` enabled) before depending on it in production.

`EmailGatewaySpec.bx` carries a larger version of the same gap. Inbound IMAP is tested entirely via an injectable `setImapPoller()` seam (canned normalized message structs, no real mailbox), and outbound is tested entirely via an injectable `setMailService()` seam (`FakeMailService`/`FakeMail`, standing in for `MailService@cbmailservices` - `EmailGateway` never goes through real WireBox at all in these specs, since there's no real ColdBox boot). What genuinely was verified directly this session (not mocked, not assumed): the real `jakarta.mail` API surface `fetchInboundMessages()` depends on (`Session.getDefaultInstance()`, `Flags`/`Flags.Flag`/`FlagTerm`, `Store.getStore("imaps")`, `Folder.READ_WRITE`, `MimeMultipart`, `InternetAddress`) against the real `jakarta.mail-api`/Angus Mail jars (downloaded standalone for this - they aren't vendored in this repo's own test classpath), confirming every class/method name used actually exists and resolves; a real `Store.connect()` against an unreachable address, driven through `EmailGateway.pollInbox()` itself (not a bypassed helper), failed with a plain connection-timeout error, not an interop/casting error, confirming the interop chain reaches the real network boundary correctly, same discipline as Slack/Discord's websocket smoke tests. What was explicitly NOT verified, and is a strictly bigger gap than the chat-platform gateways: no real IMAP handshake against a real mailbox, no real `cbmailservices`/`bx-mail` module installed anywhere in this repo or its test harness (neither is vendored the way `bx-ai`/TestBox are - see the snapshot-lag entry below for that same workaround applied to a different module), so the WireBox resolution path (`MailService@cbmailservices` really existing, `BXMail`'s `bx:mail` call really sending) has never been exercised at all, mocked or real, in this codebase. Do at least one manual `bxAgents serve` against a project with real IMAP credentials AND a real `cbmailservices`/`bx-mail` install (confirm `box install` succeeded and `moduleSettings.cbmailservices` resolves) before depending on this gateway in production - this is the least-verified of the four push-style gateways shipped so far.

`WhatsAppCloudGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: the signature-verification path is exercised with genuinely computed HMAC-SHA256 signatures (`javax.crypto.Mac`/`SecretKeySpec`, cross-checked independently this session against both `openssl dgst -hmac` and Python's own `hmac` module before trusting the BoxLang computation - a real reference-vector mismatch was caught and turned out to be a typo in the hand-copied expected value, not a bug, but only cross-verification caught that), the verify handshake, webhook dispatch/dedup, outbound send, and interactive button/list rendering are all driven through the gateway's real public methods with only the outbound Graph API HTTP call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/WhatsAppCloud.bx`'s own ColdBox request-context calls (`event.getHTTPContent()`/`event.getHTTPHeader()`/`event.renderData()`, `rc`'s URL-scope-merged dotted-key query param access for the GET handshake) against a real ColdBox boot - these ARE the documented, standard ColdBox REST-handler idioms (confirmed against ColdBox's own "Building REST APIs" recipe docs, not guessed), a meaningfully more trustworthy starting point than the undocumented `aiGatewayRegistry()` key assumption that turned out to be wrong elsewhere in this file - but "documented" isn't the same as "proven to work in this generated context," and extending the project's own real `runColdBoxIntegrationTests.bxs`/miniserver harness to cover it would have required either passing fake env vars into a separately-spawned miniserver subprocess (no existing mechanism for that) or risking real config-related boot failures in the shared `e2e-coldbox-route` fixture other passing tests depend on - deferred rather than risking that blast radius for what's a lower-confidence-of-actually-being-wrong gap than the registry-key bug was. Do at least one real `bxAgents serve` + a real Meta webhook test (or `curl`) against `/webhooks/whatsapp-cloud` before depending on this route in production. No real Graph API call has ever been made either - `deliver()`/`requestHumanInteraction()`'s HTTP layer is exercised only via the `apiCaller` test seam.

`TeamsGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: JWT verification is exercised against a genuinely generated 2048-bit RSA keypair (`java.security.KeyPairGenerator`) and hand-signed test JWTs built entirely inside the spec (no pre-computed fixtures, no external `openssl` dependency at test time) - a valid signature is accepted and dispatches, while a tampered signature, a wrong `aud`, a wrong `iss`, and an expired `exp` are each independently confirmed to reject with 401. The invoke-activity (Adaptive Card button click) path, message dispatch, personal-scope-only filtering, threading via `replyToId`, chunking, and Adaptive Card rendering are all driven through the gateway's real public methods with only the outbound Connector REST call itself stubbed (`setApiCaller()`) and the JWKS/OAuth2 token fetches stubbed (`setJwksFetcher()`/`setTokenFetcher()`). What was NOT verified: the generated `handlers/Teams.bx`'s own ColdBox request-context calls against a real ColdBox boot (same category of gap as WhatsApp Cloud's own handler, and deferred for the identical reason - see that entry above); no real OAuth2 token fetch or Connector REST call has ever been made against Microsoft's actual endpoints; and the JWKS-cached-for-instance-lifetime tradeoff (see `docs/conventions/gateways.md`'s Teams section) means a real key-rotation scenario has never been exercised either. Do at least one real `bxAgents serve` + a real Teams app registration (App ID/password from the Azure/Bot Framework portal) + a real Teams client sending a DM before depending on this gateway in production.

`TwilioGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: the `X-Twilio-Signature` HMAC-SHA1/base64 verification path is exercised with a genuinely computed signature built inline in the spec, cross-checked independently this session against Python's own `hmac`/`hashlib` modules before trusting the BoxLang implementation (same discipline as WhatsApp Cloud's own HMAC-SHA256 cross-check) - a real reference signature was computed in Python for a known auth token/URL/params combination and confirmed to match BoxLang's output exactly. Form-body parsing (including a literal `+` correctly round-tripping through `%2B` percent-encoding), the `publicUrl` override for proxy/tunnel deployments, the dual-path TwiML-ack-then-async-REST-reply model, outbound chunking, and the phone-number-keyed HITL reply correlation are all driven through the gateway's real public methods with only the outbound Messages API HTTP call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/Twilio.bx`'s own `event.getUrl()` call against a real ColdBox boot (same category of gap as WhatsApp Cloud/Teams's own handlers, deferred for the identical reason) - `event.getUrl()` is a documented ColdBox Routable/Request Context method (confirmed via the ColdBox docs MCP), not a guess, but "documented" isn't "proven in this generated context." No real Twilio Messages API call has ever been made either. Do at least one real `bxAgents serve` + a real Twilio phone number webhook test before depending on this route in production - and note the phone-number-keyed HITL correlation (see `docs/conventions/gateways.md`'s Twilio section) has a real, documented limitation: a second HITL request to the same phone number before the first is answered would overwrite the first's `pendingApprovals` entry, silently orphaning it. No allowlist/rate-limiting is built for inbound SMS either - unlike Eve, which documents (but doesn't enforce in code) that its own `allowFrom` config is "mandatory," this port has no equivalent gate at all; any phone number can message a deployed Twilio number and reach the agent.

`GitHubGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked, and this gateway's core logic was additionally driven through a standalone real-BoxLang smoke test during development (not just the permanent spec) - which is how a genuine bug was caught before it ever reached the test suite: the mention-extraction helper's substring logic called `left( body, 0 )` whenever the `@mention` occurred at the very start of a comment (a very common case), and BoxLang's `left()` throws `"Count cannot be zero"` rather than returning an empty string for a zero count - confirmed by triggering it with a real comment body, then fixed by explicitly branching around the zero-length case rather than assuming `left()`/`mid()` tolerate it. The `X-Hub-Signature-256` verification, `@mention` regex-lookahead gating (confirmed via a dedicated smoke test that a bot named `mybot` does NOT fire on `@mybot2`), bot-loop guards, delivery-id dedup, issue-vs-review-thread conversation identity, `deliver()`, and the `@mention`-to-reply HITL correlation are all driven through the gateway's real public methods with only the outbound GitHub REST call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/GitHub.bx` against a real ColdBox boot (same category of gap as every other webhook gateway's own handler in this project, deferred for the identical reason). No real GitHub API call has ever been made, and no real GitHub App/PAT has ever been used against a real repository - do at least one real `bxAgents serve` + a real GitHub webhook configured against a test repository before depending on this gateway in production.

`SignalGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: `handleSseEvent()`'s JSON-RPC/SSE parsing (blank-line/invalid-JSON handling, group-message filtering, quote-threading, `sourceUuid` fallback when a display name is absent), `deliver()`'s send shape and chunking, and HITL decision correlation/matching are all driven through the gateway's real public methods with only the outbound `rpcCaller`/`connector` I/O calls stubbed, the same seam-testing discipline as every other gateway in this project. Two BoxLang-level findings surfaced during development, both resolved and worth recording as general landmines rather than gateway-specific bugs: (1) a throwaway smoke-test script that named a local variable `request` was silently interacting with BoxLang's own reserved `request` scope instead of creating a plain variable, producing misleading "method not found"/"argument type mismatch" errors from `HttpClient.send()` that looked exactly like a genuine Java-interop limitation but disappeared entirely once the variable was renamed - `SignalGateway.bx` itself never had a bug; (2) a `try/catch` placed directly at the top level of a standalone `.bxs` smoke-test script (not inside a function) triggered a `java.lang.VerifyError: Inconsistent stackmap frames`, a real bytecode-verification limitation of BoxLang's top-level-script compiler, traced via the stack trace to the test script's own generated class, not `SignalGateway.bx` - fixed by wrapping the try/catch inside a named function instead. What was NOT verified, and is the largest gap among all the push-style gateways shipped so far: no real `signal-cli` daemon has ever been available in this environment, so the entire async SSE connection lifecycle - opening the stream via `HttpClient.sendAsync()`+`BodyHandlers.ofLines()`, the exponential reconnect-backoff loop against a genuinely flaky connection, the 30s/120s idle watchdog forcing a reconnect, and a live JSON-RPC round trip - has never been exercised end-to-end, only smoke-tested at the interop-plumbing level (a standalone test reached a real `java.net.ConnectException` against an unreachable address, proving the chain is sound, not that it works against a live daemon). Do at least one manual `bxAgents serve` against a project with a real, running `signal-cli` daemon and a real linked Signal account before depending on this gateway in production - this is a genuinely new transport architecture in this codebase (the only SSE-based gateway among Telegram/Slack/Discord/Email/Signal), not just a new platform on an already-proven transport shape.

## WhatsApp Personal (unofficial personal-account bridge) - researched, not built

The original plan (matching Hermes Agent's own architecture) called for a `WhatsAppPersonalGateway` built by spawning a Node.js subprocess running `@whiskeysockets/baileys` (the multi-device WhatsApp Web protocol client Hermes itself uses - MIT licensed, its full `bridge.js` was read directly from Hermes's real source, not summarized). That approach was set aside mid-session on direct instruction to prefer a native BoxLang/JVM integration over a subprocess bridge, and to use a native Java library only if it's open source and neither GPL nor LGPL.

That search found **Cobalt** (`com.github.auties00:cobalt`, formerly WhatsappWeb4j) - a real, MIT-licensed, actively maintained (900+ stars) Java implementation of WhatsApp's multi-device "Linked client" protocol, with a documented fluent API (`WhatsAppClient.builder().linkedApi().webClient()...`, `addNewMessageListener()`, `sendMessage()`) that would coerce cleanly from a BoxLang closure via BoxLang's own documented Java-SAM coercion (confirmed via the BoxLang docs MCP - no `createDynamicProxy()` needed for a single-abstract-method listener). Two real blockers surfaced during verification, not guessed:

1. **The first pom.xml read (Cobalt's `master` branch, an in-progress multi-module rewrite) requires Java 25** - two majors ahead of BoxLang's own documented baseline (Java 21+, confirmed via the BoxLang docs MCP and matching this project's own `21.0.10` JDK). Re-checking against the artifact actually published to Maven Central (`cobalt:0.0.10`, the real thing a `<dependency>` would resolve today, not the unreleased rewrite) showed `<java.version>21</java.version>` - so the Java-25 finding was a false alarm caused by reading the wrong branch, not a real blocker. Worth recording as a caution: a GitHub repo's default-branch `pom.xml` is not necessarily what's on Maven Central.
2. **The real, published `cobalt:0.0.10` pulls in `com.aspose:aspose-words` as a hard compile-time dependency** (used internally for generating link-preview thumbnails from Word documents) - Aspose.Words for Java is commercial/proprietary-licensed, not open source, so bundling it would violate the same license constraint Cobalt itself was chosen to satisfy. The full dependency graph (~15 jars: zxing, qr-terminal, curve25519, protobuf-base, jackson or fastjson2 depending on version, libphonenumber, dd-plist, apk-parser, link-preview, jaffree, ez-vcard, slf4j, plus Aspose) would all need to be manually downloaded and bundled under this module's `libs/` folder - BoxLang modules have no Maven-style dependency resolution of their own (confirmed via the BoxLang docs MCP: third-party jars are bundled directly into a module's `libs/` folder, loaded by a per-module classloader - there's no `javaLibraries` key in `box.json` that would resolve a dependency tree automatically).

Given the license contamination via Aspose and the manual-fat-jar-assembly effort with no dependency-resolution tooling available to verify the result actually loads, **WhatsApp Personal was descoped** rather than shipped as either a real gateway or a stub. `ProjectValidator`'s `validGatewayTypes` and `GatewayGenerator`'s `TYPE_CLASS_MAP` do not include a `whatsapp-personal` entry - a project that tries to declare one gets the existing "unknown gateway type" validation error, same as any other unsupported type, rather than a misleading half-built stub. Revisiting this is reasonable if Cobalt ever drops the Aspose dependency (it's a narrow feature - link-preview thumbnailing of Word docs - not core to messaging), or if a future session decides the Node/Baileys subprocess-bridge approach (rejected this round on architecture preference, not on a technical blocker) is preferable after all.

## `GatewaySession` is project-wide and root-agent-only (v1)

A project with at least one push-style gateway entry gets exactly one generated `GatewaySession`, bundling every push-style gateway and always bound to the project's root agent - matching the existing precedent that `exposes: "agent"` HTTP exposure is also always root-agent-only (see [gateways/](conventions/gateways.md#3-push-style-gateways-type-telegram--slack--discord--email--whatsapp-cloud--teams--twilio--github--signal-and-friends)). A project with subagents cannot yet route different gateways to different subagents (e.g. "Telegram talks to SupportBot, Slack talks to ResearchBot"). A future per-gateway `targetAgent: "SubagentName"` key, consumed by a per-agent-node `GatewaySession` instead of one project-wide session, is the natural extension point - not built yet.

## Fixed: `GatewaySessionBootstrap.bx` looked gateways up by the wrong `aiGatewayRegistry()` key (shipped broken across all four push-style gateways, caught during the WhatsApp research pass)

A real, previously-shipped bug: the generated interceptor's `aiGatewayRegistry().get(...)` call used the discovered `gateways/*` entry's own filename (e.g. `"telegramChannel"`, from `gateways/telegramChannel.bx`), but bx-ai's real `GatewayRegistry.register()` always keys by the gateway CLASS's own fixed `getName()` (e.g. `"telegram"`, set once in `TelegramGateway.init()`) - never anything caller-supplied. Confirmed both by reading bx-ai source directly and empirically (registering a real gateway, then calling `.get()` with its discovered entry name, threw `"No item found in registry"`). This meant `GatewaySession` construction would throw at ColdBox boot (`afterConfigurationLoad`) for **every** generated project with a push-style gateway - Telegram, Slack, Discord, and Email all shipped with this bug; it went undetected because the only prior test coverage asserted on the generated file's raw STRING CONTENT, never against a live registry.

Fixed in `GatewayGenerator.generate()`: the interceptor now looks gateways up by their TYPE string (which is always identical to the registered name, for every push-style gateway built so far), deduplicated. `GatewayGeneratorSpec.bx` gained a permanent regression test that registers a REAL gateway instance and proves the EXACT key the generator just emitted resolves it via a live `aiGatewayRegistry()` - closing the exact gap that let this ship undetected the first time.

**A real, permanent consequence this fix surfaces (not new behavior, just now correctly reachable)**: because the registry is keyed by type, not by entry, **two `gateways/*` entries of the same push-style type collide on the same registry slot project-wide** - e.g. two `type: "telegram"` entries (two different bot tokens) would silently have the second registration overwrite the first, and `GatewaySession` would only ever see one of them. There's no per-entry alias/registration-name override today. One instance per push-style type, per project, is the actual v1 ceiling - not documented as such before this fix made it visible.

## `./gradlew downloadModules` may fetch a bx-ai snapshot that's temporarily behind `GatewayGenerator`'s `aiGatewayRegistry()` codegen

bx-ai renamed `gatewayRegistry()` to `aiGatewayRegistry()` on its `development` branch (confirmed directly - `bifs/gatewayRegistry.bx` was deleted outright, no back-compat alias) - `GatewayGenerator` was updated to match, since bx-ai hasn't cut a release yet and this project's own instruction was to follow it straight, not shim around it. The catch: `downloadModules` fetches a fixed, continuously-republished snapshot artifact (`bx-ai@3.4.0-snapshot`) from `downloads.ortussolutions.com`, and that published zip can lag behind bx-ai's own git `development` HEAD by some amount of time (confirmed directly this session: right after this rename landed upstream, the downloadable snapshot still had the OLD `gatewayRegistry.bx`). If a fresh `downloadModules` pulls a snapshot that predates this rename, any project with a channel-adapter `gateways/*` entry will fail to boot with `Function 'aiGatewayRegistry' not found`, since the generated code now calls the new name but the fetched module still only has the old one. This isn't fixable from BX Agents' side - it resolves itself on its own once ForgeBox republishes the snapshot from current bx-ai `development`. Verified this project's own generated code is correct against bx-ai's true HEAD by building a local module structure directly from its git source (`ortus-boxlang/bx-ai`) rather than relying on the possibly-stale downloaded zip.

Reconfirmed the same lag independently while building the push-style gateway/`GatewaySession` work: the downloaded `bx-ai@3.4.0-snapshot` at the time still had no `GatewaySession.bx`, no `aiGatewaySession()`/`aiGatewayRegistry()` BIFs, and a `BaseGateway.bx`/`IGateway.bx` with no `onMessage()`/`onError()` at all - a snapshot from well before this same rename. `testBx` was run in this state by replacing `src/test/resources/modules/bxai`'s `bifs/`/`models/`/`public/`/`ModuleConfig.bx` with a fresh copy from bx-ai's own git source (its `libs/`/`box.json` left untouched), the same workaround as above - not something `build`/`serve` end users need to do themselves once ForgeBox catches up, but necessary for this session's own CI-less verification.

**CI now works around this automatically, because it had to.** The first real GitHub Actions run of this suite proved the lag is not cosmetic: the published `bx-ai@3.4.0-snapshot` still ships `bifs/gatewayRegistry.bx` (renamed to `aiGatewayRegistry` upstream long ago) and contains **no `models/gateway/BaseGateway.bx` whatsoever** - the class all nine push-style gateways in this module extend. Tested against it, 41 specs fail for reasons unrelated to this module's own code (`The method aiGatewayRegistry does not exist`, then every gateway spec cascading off a base class that is not there). `.github/workflows/tests.yml` therefore clones bx-ai's `development` branch, runs its `createModuleStructure`, and overlays the result over whatever `downloadModules` fetched - the same manual workaround described above, automated. `bx-ftp` and `bx-sqlite` are stable releases and still come from `downloadModules` untouched. Delete the overlay step once a published snapshot catches up; until then, note that CI is testing against bx-ai's branch HEAD rather than a pinned artifact, so an upstream breakage surfaces here as a bx-agents failure.

Hit a third time while wiring up `/compact`: bx-ai scoped `IAiMemory.summarize()` by `userId`/`conversationId` on `development` (commit `f9ac7bd`), but a fresh `downloadModules` still fetched a snapshot carrying the old single-argument `summarize( struct config = {} )`. Same workaround - the module was rebuilt from bx-ai's own git source (`./gradlew createModuleStructure` in that repo, copied over `src/test/resources/modules/bxai`) and the scoped behaviour verified against that build directly. Unlike the earlier two, this one has a runtime consequence an end user can hit: `/compact` calls `mem.summarize( config, userId, conversationId )`, and on a bx-ai predating that commit the extra arguments are simply ignored, so compaction would summarize the memory instance's *default* scope rather than the caller's conversation. A generated app therefore needs bx-ai at or past `f9ac7bd` for `/compact` specifically; every other route is unaffected. Nothing in this repo's own test suite regresses on an older snapshot either, since the web UI specs assert on generated source text rather than executing it.

## The v1 web chat UI (`exposes: "webui"`) - what's real, and what could only be checked against docs, not a live server

`WebUiGeneratorSpec.bx` and a `BuildPipelineSpec.bx` end-to-end test both drive the real `WebUiGenerator`/`BuildPipeline` classes against real fixtures: the static `<path>/index.html` shell is confirmed to be written and correctly templated (`__API_BASE__`/`__APP_TITLE__` placeholders substituted, never left in the output), the optional `interceptors/WebUiAuthGate.bx` is confirmed to be generated only when `apiKeyEnvVar` is configured, to gate exactly `<path>/api/*` and never the bare `<path>` shell itself, and to be correctly registered into `config/ColdBox.bx`'s `interceptors:[...]` list end-to-end through the real `BuildPipeline`. The generated interceptor was also confirmed this session to compile and instantiate cleanly via a standalone smoke test (loaded through the same `DynamicClassLoader` primitive the build pipeline itself uses).

What was NOT verified - and couldn't be, in this dev environment: a real `bxAgents serve` + real browser test of the page actually loading, streaming a reply, and the `X-API-Key` gate actually rejecting/accepting requests over real HTTP. This project's own `runColdBoxIntegrationTests.bxs`/`tests/coldbox` harness (the same one that proved `toAi()`'s `/invoke` route works end-to-end for `http-gateway-agent`) requires `tests/coldbox` to be present via a real `box install` inside `tests/` - CommandBox and ForgeBox network access were both unavailable in this session's sandbox, so that harness could not be exercised for the web UI (or re-confirmed for anything else) this session. Two direct consequences:

- The exact `/invoke` JSON response shape used by this project's own generated pages/docs (`{"input": "..."}` in, a response containing `"success": true`) is empirically confirmed **only** via `runColdBoxIntegrationTests.bxs`'s own prior, already-passing assertion (a substring check, not a full shape assertion) - not re-verified this session.
- The `/stream` SSE wire format the web UI's own JS parses (`data: {"token":"..."}` lines, terminated by `data: [DONE]`) is taken directly from ColdBox's own official "AI Routing" documentation, not independently re-confirmed against a live server this session - unlike almost every other wire-format claim in this project's docs, which were cross-checked against real running code wherever possible (see e.g. the HMAC-SHA256/SHA1 signature schemes' independent Python/openssl cross-checks elsewhere in this file).

Do at least one real `bxAgents serve` + a real browser test (message sent, reply streams in, the `X-API-Key` gate actually 401s a request missing the key) before depending on this feature in production - the same standing advice already given for every generated webhook handler's own untested-against-a-real-boot gap elsewhere in this file.

## The web UI's SQLite store - verified at the library level, not through a real ColdBox boot

The qb + bx-sqlite stack underneath `models/ChatDb.bx` was verified directly this session against the real jars, not inferred from docs: qb 13.1.0's `.cfc` sources compile and run natively on BoxLang 1.16 with no `bx-compat-cfml`; `SQLiteGrammar` + `SchemaBuilder` really do create the v1 tables and indexes against a real SQLite file; a second migration pass is a clean no-op; `QueryBuilder` round-trips inserts, filtered/ordered reads and deletes; and the `preferences` composite primary key really does reject a duplicate `(userId, prefKey)`. Two constraints were found this way rather than assumed - qb requires a **named** datasource (its own `appendSqlComments()` types that argument as `string`, so an inline struct throws before any SQL runs), and `SchemaBuilder@qb` is mapped with its `grammar` argument only, never receiving `moduleSettings.qb.defaultOptions` - and the generated code is shaped around both.

What was **not** verified, for the same reason as the rest of the web UI: none of this has run inside a real ColdBox boot. `tests/coldbox` requires a real `box install`, and CommandBox/ForgeBox network access was unavailable in this session's sandbox. So the migration logic is proven, but three wiring assumptions are not exercised end to end: that `getInstance( "ChatDb" )` resolves through the generated app's own WireBox, that `SchemaBuilder@qb`/`QueryBuilder@qb` resolve once qb is installed as a real ColdBox module (qb is a `box.json` dependency, not vendored here - the same honest gap `cbmailservices` already carries), and that `this.datasources` in the generated `Application.bx` is picked up as expected. Do one `bxAgents serve` against a webui project with `qb` and `bx-sqlite` genuinely installed before relying on the store in production; the failure mode if any of those is missing is loud at boot (an unresolvable WireBox mapping or an unknown JDBC driver), not silent.

## The web UI over real HTTP: proven by the integration runner's probe

`runColdBoxIntegrationTests.bxs` fetches `GET /chat/api/health` from outside the server, against the generated fixture app booted by a real `boxlang-miniserver`, and fails the build if it is not a 200. On CI it returns:

```
+ App probe GET /chat/api/health -> status=200
  body: {"success":true,"status":"ok"}
```

That single request is the end-to-end proof for the web UI's server side: ColdBox routing reaching `handlers/ChatUi.bx`, WireBox resolving the handler and `ChatDb`, `this.datasources` giving bx-sqlite a usable SQLite file, and qb being a genuinely activated ColdBox module - none of which any source-text assertion can establish. `WebUiRuntimeSpec.bx` then covers the store's behaviour (migrations, scoped conversation CRUD, the cross-user guard, preference upserts) by reading real objects out of WireBox inside that same boot.

**What is still not covered:** the other webui routes over HTTP. Only `/health` is fetched from outside the process. Driving the rest from `WebUiRuntimeSpec` is not possible as structured, because that spec runs *inside* a request served by the same single miniserver, so a loopback call would compete for the worker pool the runner itself occupies. Covering them needs either a second server process or a larger miniserver thread pool, and is a natural follow-up rather than something to fake.

Note that this paragraph previously cited `runner-coldbox.bxm never responded: HTTP 408` as evidence of that starvation. **That attribution was wrong**, and is worth recording because it misled this project for several CI cycles: the 408 had nothing to do with loopback calls. The runner page called `chr( 10 )` - a BIF that does not exist - so it returned 500 on entry to every request, and the orchestrator's retry loop ground through 90 seconds of those 500s before reporting the final timeout. Both are fixed. The loopback-starvation concern above is a genuine structural reason not to drive more routes from inside that spec, but it remains a well-founded expectation rather than something that has actually been observed failing here.

## The web UI's own front-end: driven in a real browser, but against a mocked API

The shipped page's JavaScript was exercised for real this session, not just asserted against as source text: the generated `index.html` was loaded in headless Chromium and driven end to end with every `<path>/api/*` route intercepted and answered with realistic payloads. Confirmed working that way - the conversation sidebar rendering from `GET /conversations` and switching/renaming/deleting through its own routes; `GET /info` shaping the toolbar (Compact appearing only when `capabilities.compact` is true, the model name landing in the header); the theme arriving from server-side `preferences` and applying; `/history` rehydrating the transcript; a real SSE turn streaming content, reasoning and tool-call chips out of the bx-ai envelope; markdown rendering; and **New chat** creating a conversation server-side and opening it. Zero JavaScript errors, zero unreplaced `__TOKEN__` placeholders, and the narrow-screen layout was screenshotted at 390px.

What that does **not** prove: the API was a Playwright mock, not the generated `handlers/ChatUi.bx` running under a real ColdBox boot against a real SQLite store. The request and response shapes were taken from the generated handler's own source, so a drift between the two would not be caught by this. Everything in the known-limitations entry above about the store applies here too - one manual `bxAgents serve` against a webui project with `qb` and `bx-sqlite` genuinely installed remains the honest gate before production use.