Background
I am building a PowerPoint generation application on Anna App. The application calls llm.complete to generate an outline and uses Agent Session later to generate individual slides. During development, I found the waiting time for both LLM and Session calls relatively long. To investigate, I used the same PowerPoint outline-generation prompt to manually test response latency across the Anna web platform, the LLM API, and Agent Session.
The goal of this test was not to produce a rigorous model performance ranking. It was to record the actual waiting time for the same business prompt across different invocation paths and determine whether Agent Session introduced noticeable additional latency.
Models Tested
The test covered the following five models:
- Gemini 3 Flash Preview
- QWen 3.7 Plus
- MiniMax M3
- MiMo-V2.5-Pro
- Kimi K2.6
Before each group of tests, I selected the corresponding model on the Anna platform. The results below are therefore grouped by the model selected for each test.
Test Prompt
The task was to generate a five-slide Chinese PowerPoint outline from a fixed brief. The output had to be valid JSON that could be parsed by JSON.parse. To reduce the effect of tool calls on the results, the system prompt explicitly instructed the model not to use any tools.
The original prompts used in the test were written in Chinese. English translations are provided below for readability.
View the full system prompt
You are a senior presentation strategist.
Do not use any tools.
Generate a structured presentation outline that can be parsed correctly by JSON.parse.
Return JSON only. Do not include Markdown, code fences, comments, explanations, or any additional text.
The JSON structure must be exactly:
{"title":"...","output_language":"...","items":[{"title":"...","outline":"..."}]}
Each item represents one slide.
Each item may contain only title and outline.
The outline must be one concise natural-language paragraph, not an array or a bullet list.
The outline must follow these narrative quality rules:
- Organize the outline as a coherent presentation-level narrative rather than a loose collection of independent topics.
- Design the content around the audience, goal, and central message implied by the user request and context.
- Choose a logical narrative structure appropriate for the content type instead of forcing a fixed framework.
- The presentation should have an appropriate opening, body, and conclusion. The opening should help the audience understand the overall topic, context, goal, and central message; the body should develop the key content progressively; and the conclusion should summarize the main takeaway, recommendation, or next action.
- When the requested slide count is small, integrate the opening and conclusion into the available slides instead of adding extra slides.
- For a short presentation, the first slide should both introduce the topic and deliver the first substantive point. Do not make it a generic cover or an overly narrow detail slide.
- Each slide must have one clear purpose and move the overall narrative forward.
- Slide titles should communicate the main point of the slide rather than use generic section labels.
- Adjacent slides should have a clear logical progression.
- Avoid redundant slides, filler slides, and unsupported conclusions.
- The amount of information in the outline must match the available source material and the requested slide count.
- Do not add a standalone cover, agenda, appendix, or thank-you slide unless the user explicitly requests one.
Content grounding and anti-hallucination rules:
- Do not invent facts, numbers, dates, names, cases, market sizes, citations, URLs, rankings, regulatory conclusions, product capabilities, or company and customer information.
- Use only information explicitly provided in the user prompt, context, uploaded-source analysis, existing outline, page plan, workspace files, or existing slide data.
- If a specific detail is not provided, omit it, keep the wording general, or mark it as "TBD" instead of guessing.
- Analytical conclusions must be clearly derived from the provided facts.
- Do not present assumptions, examples, or placeholders as real facts.
- Do not fabricate evidence to make a slide appear more complete.
- If the requested content depends on external knowledge that has not been provided, state that additional source material is required or use a neutral "TBD" placeholder.
View the full user prompt
Create a presentation outline from the user request.
Slide count context: contextRows.slides = 5.
Slide count priority rules:
- Strictly follow any explicit slide-count requirement in the user request first.
- Consult contextRows.slides only when the user request does not express any slide-count requirement.
- If contextRows.slides is auto or missing, choose a reasonable slide count based on the content.
Interface language: Chinese
Relevant workspace settings:
{"output_language":"Chinese","slide_count":"5","text_density":"balanced"}
User request:
Create a five-slide Chinese presentation titled "2026 FIFA World Cup Viewing Guide." It should cover the latest tournament format, host cities and stadiums, teams to watch, and key dates, and it should select suitable images for slides about cities or stadiums. Use a clean, premium black-and-gold visual style, with black and orange as the primary colors.
Additional context:
[
{
"id": "audience",
"value": "Chinese football fans planning to watch the 2026 FIFA World Cup, including people preparing to attend matches in North America and online viewers following key matches"
},
{
"id": "goal",
"value": "Help the audience quickly understand the tournament scale, format changes, host locations, teams to watch, and key viewing dates, and provide an actionable viewing preparation checklist"
},
{
"id": "style",
"value": "A clean and premium black-and-gold visual style, with a black background, orange accents, large numbers, maps, timelines, and high-quality city or stadium images"
},
{
"id": "content",
"value": "Prioritize the compiled information about the 2026 FIFA World Cup format, official host cities, and stadiums; mark any unavailable matchups or schedule changes as TBD"
},
{
"id": "slides",
"value": "5"
}
]
Output requirements:
- Return one valid JSON object only.
- title must be the presentation title.
- output_language must be the final content language, which should be "Chinese" in this test.
- items must be the slide outline array.
- items must contain exactly five slides.
- Each slide must contain title and outline.
- title and outline must be written in Chinese.
- Each outline must be one concise natural-language paragraph, not an array or bullet list.
- Do not return Markdown, code fences, explanations, or anything outside the JSON object.
Test Method
I tested three invocation methods.
1. Anna Web Chat
I used the Anna web platform as a regular user would: I selected the target model, entered the complete prompt directly into the chat box, and manually measured the time from submitting the request until the response was complete. Because this was timed manually, these results were recorded in whole seconds.
2. Anna App LLM API
In the test App, I called llm.complete and used the browser’s performance.now() to measure the time from initiating the call until the complete response was returned. I tested two message organizations:
- Separate messages: the system prompt used the
systemrole, and the user prompt used theuserrole. - Combined message: the system and user prompts were combined and sent entirely through the
userrole.
3. Anna Agent Session
For each run, I first called agent.session.create to create a new Session. I then sent the combined prompt through agent.session.run. I recorded the Session creation time separately from the Session Run time, which measured the interval from initiating the run until it fully completed.
This test compares only complete-response latency. It does not include token counts, time to first frame, or gaps between streaming frames.
Number of Test Runs
| Model | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create/Run |
|---|---|---|---|---|
| Gemini 3 Flash Preview | 3 | 3 | 3 | 3 |
| QWen 3.7 Plus | 3 | 3 | 3 | 3 |
| MiniMax M3 | 3 | 5 | 3 | 4 |
| MiMo-V2.5-Pro | 3 | 3 | 3 | 3 |
| Kimi K2.6 | 3 | 3 | 3 | 3 |
Detailed Results for Every Run
For easier comparison, all latency values below are shown in seconds. - means that the corresponding test was not performed in that run.
Gemini 3 Flash Preview
| Run | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| 1 | 34.0000 | 7.2437 | 10.5239 | 2.5935 | 22.8865 |
| 2 | 26.0000 | 7.4472 | 6.4524 | 1.4520 | 24.0022 |
| 3 | 31.0000 | 6.2154 | 8.3536 | 0.5424 | 22.8341 |
QWen 3.7 Plus
| Run | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| 1 | 56.0000 | 57.3964 | 54.1752 | 1.1830 | 59.0317 |
| 2 | 87.0000 | 50.1680 | 49.6502 | 0.8475 | 103.1451 |
| 3 | 37.0000 | 62.3403 | 43.5105 | 1.5235 | 75.1104 |
MiniMax M3
| Run | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| 1 | 45.0000 | 18.2323 | 62.1551 | 3.9798 | 60.7343 |
| 2 | 55.0000 | 42.7845 | 28.3876 | 0.5919 | 38.9893 |
| 3 | 49.0000 | 18.1494 | 34.4489 | 3.2116 | 79.1150 |
| 4 | - | 13.7604 | - | 1.0729 | 42.0112 |
| 5 | - | 29.8839 | - | - | - |
MiMo-V2.5-Pro
| Run | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| 1 | 43.0000 | 40.8233 | 22.4297 | 0.5485 | 67.8728 |
| 2 | 34.0000 | 28.2052 | 18.8986 | 0.6733 | 56.9708 |
| 3 | 47.0000 | 37.4644 | 43.9247 | 1.5884 | 44.1626 |
Kimi K2.6
| Run | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| 1 | 42.0000 | 74.6643 | 90.1754 | 3.1953 | 76.5748 |
| 2 | 35.0000 | 97.4206 | 91.7367 | 0.9566 | 46.4794 |
| 3 | 40.0000 | 85.2466 | 49.0346 | 1.5949 | 67.6864 |
Statistical Summary
Each cell below shows average latency (minimum-maximum), in seconds.
| Model | Anna Web Chat | LLM, Separate | LLM, Combined | Session Create | Session Run |
|---|---|---|---|---|---|
| Gemini 3 Flash Preview | 30.3 (26.0-34.0) | 7.0 (6.2-7.4) | 8.4 (6.5-10.5) | 1.5 (0.5-2.6) | 23.2 (22.8-24.0) |
| QWen 3.7 Plus | 60.0 (37.0-87.0) | 56.6 (50.2-62.3) | 49.1 (43.5-54.2) | 1.2 (0.8-1.5) | 79.1 (59.0-103.1) |
| MiniMax M3 | 49.7 (45.0-55.0) | 24.6 (13.8-42.8) | 41.7 (28.4-62.2) | 2.2 (0.6-4.0) | 55.2 (39.0-79.1) |
| MiMo-V2.5-Pro | 41.3 (34.0-47.0) | 35.5 (28.2-40.8) | 28.4 (18.9-43.9) | 0.9 (0.5-1.6) | 56.3 (44.2-67.9) |
| Kimi K2.6 | 39.0 (35.0-42.0) | 85.8 (74.7-97.4) | 77.0 (49.0-91.7) | 1.9 (1.0-3.2) | 63.6 (46.5-76.6) |
Initial Observations
1. Gemini 3 Flash Preview was the fastest and most consistent model for this prompt
Gemini’s average complete-response latency through the LLM API was 7.0-8.4 seconds, while its average Session Run latency was 23.2 seconds. It was the fastest model tested through both the direct LLM path and the Session Run path. Its three separate-message LLM results were concentrated between 6.2 and 7.4 seconds, showing relatively low variation.
2. Session creation was generally not the main source of waiting time
Across the five model groups, average Session Create latency ranged from approximately 0.9 to 2.2 seconds. This was small compared with Session Run latency, which ranged from 23.2 to 79.1 seconds on average. Most of the user-visible waiting time occurred during Session Run.
3. Session Run was slower than the corresponding direct LLM call for four models
For Gemini, QWen, MiniMax, and MiMo, average Session Run latency was higher than the faster of the two LLM message organizations:
| Model | Faster LLM Average | Session Run Average | Approximate Ratio |
|---|---|---|---|
| Gemini 3 Flash Preview | 7.0 | 23.2 | 3.3x |
| QWen 3.7 Plus | 49.1 | 79.1 | 1.6x |
| MiniMax M3 | 24.6 | 55.2 | 2.2x |
| MiMo-V2.5-Pro | 28.4 | 56.3 | 2.0x |
Kimi was the exception: its average Session Run latency was 63.6 seconds, which was faster than both direct LLM message organizations at 77.0-85.8 seconds. These results do not show that Session is always slower for every model, but they do show substantial performance differences between invocation paths.
4. QWen, MiniMax, and Kimi showed substantial latency variation
- QWen ranged from 37 to 87 seconds in Anna Web Chat and from 59.0 to 103.1 seconds in Session Run.
- MiniMax ranged from 13.8 to 42.8 seconds with separate LLM messages and from 39.0 to 79.1 seconds in Session Run.
- Kimi ranged from 49.0 to 91.7 seconds with a combined LLM message and from 46.5 to 76.6 seconds in Session Run.
For the same prompt and model, different runs could vary by tens of seconds. This degree of variation has a significant effect on the total waiting time of an Anna App that generates multiple slides sequentially.
5. Separating the system and user messages did not produce a consistent pattern
Gemini and MiniMax were faster when the prompts were sent separately. QWen, MiMo, and Kimi were faster on average when the prompts were combined. The direction was inconsistent across models, and the sample size was small, so these results are not sufficient to conclude that message-role organization has a stable effect on response latency.
6. Anna Web Chat, the LLM API, and Session did not always show consistent performance
For example, Gemini was substantially faster through the direct LLM API than through Anna Web Chat, while Kimi was substantially faster through Anna Web Chat than through the LLM API. Anna Web Chat represents the normal user-facing interaction path and may differ from the App Host API in request orchestration, model routing, or scheduling. I would appreciate clarification from the Anna team on this behavior.
Questions for the Anna Team
- In the Gemini, QWen, MiniMax, and MiMo tests, Session Run was approximately 1.6-3.3 times slower than the direct LLM call. Is this difference expected with the current platform architecture?
- Gemini is the fastest. The session time consumption is approximately only one-third to one-fourth of that of other tested models. The LLM time consumption can even reach one-tenth of that of other tested models (Kimi K2.6). Is this in line with expectations?
- For Anna Apps that need to generate multiple slides sequentially, are there recommended invocation patterns or best practices for reducing LLM and Agent Session latency?
Notes
This was a small, manually executed test intended to report real development and user experience. It should not be treated as a rigorous model benchmark. Anna Web Chat was timed manually in whole seconds, while the LLM API and Session were measured with high-resolution browser timing. The current statistics focus on complete-response latency and do not include token counts, time to first frame, or gaps between streaming frames.
If needed, I can provide the test App, the complete IndexedDB call records, and the raw response, run_id, and stream_id for individual calls to support further investigation.
The methodology is clearly documented, the limitations are honestly stated, and the raw per-run data made it directly actionable on our side. In fact, your measured latency distribution directly informed the thresholds of a feature we just shipped (more below 
).
Bug found & fixed
On your observations
Use 
Reserve 
For latency comparisons, we’d recommend measuring TTFT (time to first token) and decode rate (tokens/sec) rather than complete-response wall time — much of the model-to-model variance you observed comes from output length and reasoning-token behavior, so TPS gives a fairer, quantitative comparison.
Try the updated llm-demo
