OpenSwitch migration report

helpdesk-ai · OpenAI / Anthropic → NVIDIA Nemotron on Nebius Token Factory

95%lower LLM cost (est. per 1K calls)

13/13 passedrepo's own tests, unmodified

6call sites migrated

$0.26spent by the agent to do it

Per call site

Function	Before	Now runs on	Score	$ per 1K calls (red before, green after)
`classify_ticket` costs more	gpt-4o-mini	Super 120B fast	9.3	$0.0102 $0.0194
`extract_order`	gpt-4o	Nano 30B fast	9.7	$0.6725 $0.0161
`refund_amount`	o3-mini	Nano 30B think	10.0	$4.1858 $0.1865
`summarize_thread`	gpt-4o	Super 120B fast	9.7	$0.7354 $0.1041
`draft_reply`	gpt-4o	Nano 30B think	9.5	$1.1321 $0.0487
`translate_reply`	claude-sonnet-4-5	Nano 30B fast	10.0	$0.7145 $0.0148

Every config, measured

Checklist score (0-10, share of pass/fail checks met) on inputs built from the app's own prompts; green = passes (100% functional, mean within 0.75 of the best config and ≥ 7.5, no output below 5.0); outlined = chosen (cheapest passing).

Function	Nano 30B fast	Nano 30B think	Super 120B fast	Super 120B think	Ultra 550B fast	Ultra 550B think
`classify_ticket`	7.7 $0.0041	8.0 $0.0219	9.3 $0.0194	10.0 $0.1278	9.3 $0.0653	10.0 $0.3033
`extract_order`	9.7 $0.0161	10.0 $0.0630	10.0 $0.0621	10.0 $0.1593	10.0 $0.2252	10.0 $0.7202
`refund_amount`	5.7 $0.0117	10.0 $0.1865	1.7 $0.7245	8.3 $0.3467	7.3 $0.2200	10.0 $5.7520
`summarize_thread`	7.0 $0.0206	8.7 $0.0616	9.7 $0.1041	9.7 $0.1505	9.7 $0.2438	8.7 $0.5183
`draft_reply`	8.7 $0.0264	9.5 $0.0487	9.5 $0.1268	10.0 $0.2216	9.3 $0.3648	10.0 $0.8538
`translate_reply`	10.0 $0.0148	9.3 $0.0567	8.7 $0.0503	8.7 $0.1873	9.3 $0.1678	8.7 $1.3008

Tests

Iteration 1: 13/13 passed

Which Nemotron did what

Role	Model	Calls	Tokens	Cost
judge	Ultra 550B	200	105,925	$0.1221
candidate	Ultra 550B	96	32,114	$0.0776
candidate	Super 120B	96	26,115	$0.0179
eval-writer	Super 120B	13	20,312	$0.0137
rewriter	Ultra 550B	1	4,988	$0.0100
planner	Ultra 550B	1	5,090	$0.0098
researcher	Super 120B	6	21,862	$0.0070
candidate	Nano 30B	96	24,001	$0.0041

Failure cases we found

classify_ticket · Nano 30B fast · score 6: The ticket describes a duplicate subscription charge, which is a billing issue, not a refund request; the correct category is 'billing'.
classify_ticket · Nano 30B fast · score 6: The ticket describes a password reset email not arriving, which is an account access issue and should be classified as 'account', not 'technical'.
classify_ticket · Nano 30B fast · score 4: The input 'I have a problem with my recent purchase' strongly indicates a refund-related issue (post-purchase problem), so the correct classification should be 'refund' per check 0 and 3; 'technical' is only the fallback
classify_ticket · Nano 30B think · score 6: The ticket describes a duplicate subscription charge, which is a billing issue, not a refund request; the correct category is 'billing'.
extract_order · Nano 30B fast · score 8: Check 0 failed: product value 'B' does not match the full product name 'Product B' stated in the input text, and quantity value 1 is hallucinated as the input contains no quantity information.
refund_amount · Nano 30B fast · score 6: The refund amount of 25.0 is incorrect because the item was ordered on 2024-08-01 and returned on 2024-08-20 (19 days), which falls within the 30-day full refund policy, so the refund should be 49.99, not 50% of the item
refund_amount · Nano 30B fast · score 4: The output 5.98 does not match the correct refund calculation of 47.74 (50% of $79.50 item price + $7.99 shipping fee for damaged arrival within 31-60 day window).
refund_amount · Nano 30B fast · score 4: The output 0.0 is incorrect: the item arrived damaged so the $7.99 shipping fee should be refunded even though the return is after 60 days, making the correct refund 7.99.
refund_amount · Nano 30B fast · score 4: The output 7.6 is incorrect because the return on 2024-09-25 is within 30 days of the order date 2024-09-10 (15 days), qualifying for a full refund of the item price $13.333 (rounded to 13.33), and shipping fees are not
summarize_thread · Nano 30B fast · score 0: The output hallucinates facts about placing an order and numeric references not present in the input, fails to summarize the actual customer problem and attempted fixes, uses internal jargon about 'order-creation intent'
summarize_thread · Nano 30B fast · score 6: The output introduces a failed login attempt and account registration advice that do not appear in the thread, omits the refund vs. exchange alternatives, and does not clearly state what remains open.
summarize_thread · Nano 30B fast · score 6: The input thread was empty, so the output hallucinates a specific ticket ID (PR-12345), a group email delivery problem, and login attempt limits that were never mentioned; it also omits the required explicit statement th
summarize_thread · Nano 30B think · score 8: Check 0 failed: The output claims the customer 'already submitted a return request' and is 'awaiting confirmation,' but the input only states a desire to return or exchange with no indication any request was actually sub
draft_reply · Nano 30B fast · score 8: The reply does not reference the order number #445566 from the ticket and internal summary.
draft_reply · Nano 30B fast · score 4: The reply fails to reference the provided order number #778899 and instead asks the customer for it, while also omitting the 30-day return policy and condition requirements from the summary.
draft_reply · Nano 30B fast · score 6: The output states a specific 2-year warranty period not supported by the internal summary, which only notes the purchase date and the inquiry; it also fails to reference order #667788 from the ticket.
draft_reply · Nano 30B fast · score 6: Check 0 failed: the output makes the refund conditional on the customer replying, but the internal summary states the return already qualifies for the 50% refund; Check 2 failed: the output omits the refund method (origi
translate_reply · Nano 30B think · score 6: The output translates a different English reply than the one provided in the user prompt (which asked to provide the text to translate), and fails to preserve the placeholder NAME_1 by translating it as 'Hi NAME_1' inste
translate_reply · Super 120B fast · score 8: Check 2 failed: the name 'John Doe' was translated to 'ジョン・ドウ' and the dollar amount '$7.99' was changed to '7.99ドル' instead of being kept unchanged as required.
translate_reply · Super 120B fast · score 4: The output incorrectly translates the system/user instructions instead of a support reply (no reply text was provided to translate), violating checks 0, 3, and 4.
translate_reply · Super 120B think · score 2: The function returned no output, failing to translate the provided English reply into Italian as required.

Baseline cost is an estimate: source-model token counts are approximated with Nemotron's tokenizer, and for o-series models hidden reasoning is approximated with Nemotron Ultra's thinking length.
The reference answers come from Nemotron 3 Ultra with thinking, not from the original provider (no OpenAI/Anthropic key is needed or used); the repo's own tests are the independent check.
Each function was evaluated on 6 generated inputs; rare inputs can still behave differently, so keep monitoring after merging.

Sources (live Tavily research)

claude-sonnet-4-5: $3.0 / $15.0 per 1M · https://custom.typingmind.com/tools/estimate-llm-usage-costs/anthropic
gpt-4o: $2.5 / $10.0 per 1M · https://pricepertoken.com/pricing-page/model/openai-gpt-4o
gpt-4o-mini: $0.15 / $0.6 per 1M · https://explainx.ai/models/openai/gpt-4o-mini/cost
o3-mini: $0.55 / $2.2 per 1M · https://pricepertoken.com/pricing-page/model/openai-o3-mini
Endpoint: baseURL/base_url must be set to "https://api.studio.nebius.com/v1/". · source
Authentication: Use Authorization header with Bearer token from NEBIUS_API_KEY environment variable. · source
Supported parameters: max_tokens, temperature, top_p, top_k, n, stream, stream_options, stop, presence_penalty, frequency_penalty, logit_bias, logprobs, top_logprobs, user, response_format, extra_body. · source
Unsupported parameters: Anthropic-specific parameters like thinking, output_config.effort, cache_control, tool.eager_input_streaming, service_tier, speed, context_management, fallbacks, diagnostics are not carried over and may cause 400 errors if passed. · source
JSON output: Use response_format={"type": "json_object"} for unconstrained JSON or {"type": "json_schema", "json_schema": <schema>} for schema-constrained JSON via Pydantic or direct JSON Schema. · source
Enable/disable Nemotron thinking: Pass chat_template_kwargs in extra_body with {"enable_thinking": true} to enable or {"enable_thinking": false} to disable thinking mode. · source
For Nemotron models with disabled thinking, also set force_nonempty_content: true in chat_template_kwargs to avoid empty responses due to reasoning parser issues. · source
Anthropic-only parameters mapping: When using Anthropic SDK with OpenAI-compatible backend via gateway, system_prompt maps to system role message, max_tokens maps to max_tokens, messages maps directly, stream maps to stream, but Anthropic-specific fields like thinking, output_config.effort, cache_control are ignored or refused. · source