Should `anna.llm.complete` expose a timeout parameter? Current SDK-level timeout may cause the app to time out while the LLM request still completes

[HappyLight]

Hi Anna team,

While developing an Anna App, I ran into a question about timeout handling for anna.llm.complete, and I would like to confirm whether the current behavior is expected or whether the docs / SDK contract could be clarified.

I checked the current official developer documentation, especially:

• App-Side LLM & Agent API
• App UI Host API
• Local Dev with --llm

The documented anna.llm.complete(req) example is roughly:

const reply = await window.anna.llm.complete({
  messages: [
    { role: "user", content: { type: "text", text: "Say hi" } },
  ],
  maxTokens: 256,
});

From the docs, it looks like anna.llm.complete(input) itself does not publicly accept a timeout, timeoutMs, signal, or similar timeout/cancellation parameter.

However, when I inspected the local @anna-ai/app-runtime implementation, I found that the lower-level generic RPC method runtime.call(ns, method, args, options) does support timeoutMs, for example:

await runtime.call("llm", "complete", input, {
  timeoutMs: 120_000,
});

So if an app wants to avoid the default SDK wait timeout used by anna.llm.complete(input), it currently seems necessary to bypass the higher-level helper and call the lower-level generic RPC method directly.

There is also a semantic concern here: this timeoutMs appears to be a local iframe SDK / runtime RPC wait timeout, not a timeout for the actual backend LLM request. In other words, the following situation seems possible:

1. The app calls runtime.call("llm", "complete", input, { timeoutMs: 5000 })
2. After 5 seconds, the app-side Promise rejects with RPC llm.complete timed out
3. But the LLM request in the dev harness / host / network layer is not cancelled
4. Later, the LLM request may still complete successfully in logs or at the network layer
5. The app runtime has already removed the pending request, so the late result no longer resolves the original caller

This can lead to a few practical issues:

• The app treats the request as failed, while the backend request may still be running
• If the app retries after timeout, it may create duplicate LLM requests
• This may cause duplicate quota usage or duplicate generation work
• It is not obvious whether timeoutMs means “local SDK wait timeout” or “LLM request timeout”
• The higher-level anna.llm.complete API and the lower-level runtime.call capability feel inconsistent

I would like to ask:

1. Is anna.llm.complete(input) expected to officially support timeoutMs or AbortSignal in the future?
2. Is the current meaning of runtime.call(..., { timeoutMs }) only “local SDK wait timeout”, without cancelling the host / backend / provider-side request?
3. If so, could the docs explicitly distinguish between:
   • app runtime RPC timeout
   • host/backend LLM request timeout
   • provider/model timeout
4. What is the recommended best practice for long-running LLM completions? For example, should developers:
   • use a longer runtime.call timeout
   • avoid automatic retries after timeout
   • use agent sessions / streaming instead
   • or wait for future cancellable anna.llm.complete support

I think it would be helpful to add a small section to the docs clarifying the current request schema supported by anna.llm.complete, and how runtime.call’s timeoutMs relates to the actual LLM request lifecycle. This would help developers avoid misunderstandings where the app times out but the backend request still completes, potentially causing duplicate requests or quota usage.

[hunter]

Hi @HappyLight — thank you for such a careful, well-researched write-up! This is exactly the kind of report that helps us sharpen the platform. You’re right on the important parts, and the good news is most of what you need already works today. Let me clarify and share where we’re headed.

1. You don’t need to drop down to runtime.call

The higher-level helper already forwards a per-call options object, so this is fully supported:

const reply = await window.anna.llm.complete(
  {
    messages: [{ role: "user", content: { type: "text", text: "Say hi" } }],
    maxTokens: 256,
  },
  { timeoutMs: 120_000 } // 👈 second arg, same as runtime.call
);

[anna.llm.complete(input, { timeoutMs })] and [runtime.call(“llm”, “complete”, input, { timeoutMs })] are equivalent — the helper just passes the options straight through. So you can stay on the ergonomic API.

Also worth noting: the default [llm] timeout is already 180s (not 30s) to comfortably cover long completions and large context. You only need to pass [timeoutMs] when you want something longer or shorter. This was under-documented in the examples — we’re fixing that.

2. Your semantic concern is accurate — and it’s the right thing to flag

You nailed it: today [timeoutMs] is a client-side / SDK wait timeout, not a backend request timeout. If the SDK timer fires first, the app-side promise rejects, but the in-flight host/provider request is not actively cancelled. That means a late result can arrive after the pending entry is gone, and a naive retry could create a duplicate request. Great catch.

To make the three layers explicit (we’ll add this to the docs):

• App runtime RPC timeout → [timeoutMs] (what you control from the iframe)
• Host/backend LLM request timeout → enforced server-side independently
• Provider/model timeout → upstream provider P99 bounds

3. Recommended best practices for now

• Use [anna.llm.complete(input, { timeoutMs })] directly — no need to bypass the helper.
• Set a generous [timeoutMs] for long completions rather than a tight one.
• Avoid automatic retries on timeout for now — since the original request may still be completing, a retry can double up work/quota.
• For long-running or cancellable workloads, prefer [anna.agent.session] — it supports streaming and [session.cancel].

4. What we’re shipping

• Docs update clarifying the [timeoutMs] semantics and the three timeout layers above.
• Bringing [llm.complete] in line with our cancel-capable surfaces so a client timeout can signal the backend and avoid orphaned/duplicate requests.

Thanks again for the thoughtful report and for digging into the runtime internals — this kind of feedback genuinely makes Anna better for every developer. Please keep them coming, and let us know if anything above doesn’t match what you’re seeing!

Happy building!