Qaf · Infra test · Oct 27, 2026Gemini Flex: half price, but only for work nobody is waiting on
We ran 60 real Qaf questions through gemini-3.7-flash on Standard and Flex, on both Vertex (our prod path) and the Gemini API. The answers came out the same; the difference is latency and reliability.
Vertex Flex works: 59/59 requests succeeded at 50% off. Median time to finish was about 31 s vs 21 s on Standard, but 1 in 10 took over 2.8 min, and the slowest took 5.6 min.
Keep chat on Standard; move catalog generation and evals to Vertex Flex. Skip Gemini API Flex: 22 of 60 requests were rejected with 503 and the rest took 1.3–7 min (median ≈4 min).
Results · second run, 30 questions per row
| Path | Price | Succeeded | First text p50 | Done p50 | Done p90 | Slowest |
|---|---|---|---|---|---|---|
| Vertex · StandardProd today | list | 30/30 | 12 s | 21 s | 29 s | 32 s |
| Vertex · FlexFlex PayGo | 50% off | 30/30 | 22 s | 31 s | 2.8 min | 5.6 min |
| Gemini API · Standard | list | 30/30 | 15 s | 28 s | 34 s | 45 s |
| Gemini API · Flexservice_tier | 50% off | 19/30 | 3.5 min | 3.7 min | 6.5 min | 7.2 min |
Every request, time to finish
Each dot is one answer (log scale); the black tick marks the median. Blue = Standard, green = Flex.
What this means for Qaf
- Chat: no. On Vertex Flex, first text took 2.5+ min for 1 in 10 users, and Flex 503s don't fall back to Standard on their own. Chat is most of our Gemini spend, so most of the bill stays at list price.
- Catalog generation: yes.
catalogLlmalready runs on Vertexglobal, the only endpoint Flex supports. A full ar+en catalog run measured ≈$1.1k at list price, so Flex saves ≈$550 per run, and ≈$1.1k once Gemini's introductory pricing ends on Jan 1. - Evals and LLM judges: yes. Bench harnesses and judge passes don't need to be fast. It's the same change.
- Change needed: add
sharedRequestType: "flex", requestType: "shared"inside the existinggoogleVertexoptions (merge into that key, don't add a new one, orthinkingConfigis silently dropped), allow requests to run ≥10 min, and retry 503/429 with backoff, falling back to Standard as a last resort. To confirm Flex was used, checkusageMetadata.trafficType: it should readON_DEMAND_FLEX.
How we tested
Oct 27, 2026, 09:55–11:00 UTC, in two runs of 30 real user questions from prod search logs (Arabic and English). Each question went to all four paths at the same time: streaming, thinking set to medium, no SDK retries, a unique system-prompt prefix so no request was served from cache, and about 2.4k output tokens per answer. The table uses the second run only. In the first run, Bun's built-in 300 s fetch timeout cut off 10 Flex requests early (9 Gemini API, 1 Vertex). The 503 count includes both runs. Every response reported the tier it was served on: ON_DEMAND_FLEX on Vertex, serviceTier: flex on the Gemini API. Two caveats: this covers about an hour on one weekday, and Flex speed depends on how busy Google is overall. The 50% price is from Google's docs; we haven't checked it against our credits invoice yet. Vertex allows 3,000 Flex requests per minute per model.