Anna LLM sampling is unstable

Developers
1

Hi anna team:

The same app, the same executable—sometimes sampling can be successfully invoked, sometimes not, and even after three retries, it still fails.

Two types of errors have occurred：

[-32001] Sampling token missing — host did not authorize sampling for this invoke

[-32001] invalid sampling token: Signature has expired.

The same code has never encountered any errors with sampling in the local dev environment.

It’s difficult for me to determine the cause myself.

2

Hi @zhaopy2121 :waving_hand:

Thanks so much for the detailed report — this was super helpful, and you nailed it: these were two real, distinct bugs on the platform side (not anything wrong with your app), which is exactly why local dev never reproduced them. :raising_hands:

We’ve shipped fixes for both error cases:

1. Sampling token missing :admission_tickets:
The Anna App iframe invoke path wasn’t minting a sampling token the way the chat-agent path did. It now mints tokens on both paths, so authorized sampling calls go through consistently.

2. invalid sampling token: Signature has expired :hourglass_not_done:
Sampling tokens were issued once per invoke with a short lifetime, so long-running or multi-call invokes could outlive the token. We added end-to-end token renewal — the host now proactively refreshes before expiry and retries seamlessly, so your retries won’t keep hitting an already-expired token. :counterclockwise_arrows_button:

Give it a spin :rocket:
The easiest way to validate in a dev environment is the LLM demo plus a push:

anna-app apps push

Reference app: [anna-executa-examples/examples/anna-app-llm-demo at main · whtcjdtc2007/anna-executa-examples · GitHub]

If you still see -32001 after this :magnifying_glass_tilted_left:
At that point it’s almost certainly an authorization step rather than the token mechanism itself. Two quick things to check:

  1. :white_check_mark: Your executa’s manifest declares host_capabilities: ["llm.sample"] and the capability has propagated (a re-publish / rediscover refreshes it).
  2. :white_check_mark: Sampling is enabled for your executa via the per-user grant (the Permissions toggle) — this is a separate consent from the app-level grants.

Give it a try and let us know how it goes — happy to dig in further if anything still feels off. Thanks again for helping make Anna better! :purple_heart: