TL;DR

Groq, Together AI, Fireworks and Replicate rate cards for open-weight models, worked token bills at four volumes, and the math for per-second GPU billing.

Serving the same 26 million tokens a month on GPT-OSS 120B costs $6.60 on Groq, $6.60 on Fireworks, $6.60 on Together AI — and $4,008 for a dedicated H100 on Replicate that runs around the clock. That is a 600x spread between platforms hosting the same open-weight model, and it exists because three of them bill in tokens while the fourth bills in seconds. The open-weight market has flattened LLM inference pricing to a few tenths of a dollar per million tokens, so the real differences now hide in caching discounts, batch modes, latency guarantees, and what happens when your model sits idle between requests. Frontier model API pricing — GPT-6, Claude, Gemini — gets the headlines, but the open-weight hosting layer underneath is where most production budgets actually live.

LLM Inference Pricing at a Glance

Four platforms, four meters. Groq bills tokens on LPU hardware with a single rate card and no subscription. Together bills tokens across the deepest open-model catalog, then layers dedicated GPUs and raw clusters on top. Fireworks bills tokens through a mode system — the same model carries five price points depending on urgency. Replicate bills seconds of GPU time: the only option here for image, video, and audio workloads, and the only one where your bill depends on your own engineering.

PlatformBilling modelFree tierMain discount levers
GroqPer token, pay-as-you-goYes — 30 req/min, 6K tokens/min, 14.4K req/dayBatch (50% off), prompt cache (~50% off cached input)
Together AIPer token; dedicated GPUs and clusters priced hourlyOne catalog model (Ternary Bonsai 27B) listed freeCached input (as low as 2% of input rate), raw GPU clusters from $1.99/hr
FireworksPer token across five modesNo flat free tier — prepaid credits, no commitmentBatch (50% off), Spot (30% off), cached input at $0.015 on GPT-OSS
ReplicatePer second of GPU time (some models per output)None published — cheapest hardware is CPU at $0.36/hrPublic models bill only while processing; volume discounts by contract

Normalized rate cards, per 1M tokens in / out, as of November 2026:

Model classGroqFireworksTogether AIReplicate
GPT-OSS 120B$0.15 / $0.60$0.15 / $0.60$0.15 / $0.60— (no per-token listing)
DeepSeekR1 Distill 70B $0.75 / $0.99V4.1 Flash $0.30 / $1.20V4.1 Flash $0.30 / $1.20R1 $3.75 / $10.00
KimiK2 $1.00 / $3.00K3 $3.00 / $15.00K3 $3.00 / $15.00—
GLM-5.3—$1.40 / $4.40$1.40 / $4.40—
Llama 4 Scout$0.11 / $0.34—fine-tune hosting only—
Cheapest text tier8B Instant $0.05 / $0.08Lightning 30B $0.05 / $0.20Bonsai 27B freeCPU $0.36/hr

Groq Pricing: One Rate Card and an LPU

Groq pricing is the simplest of the four: pure pay-as-you-go token rates, no monthly subscription, and a rate card that has stayed stable since August 2026. The headline model is GPT-OSS 120B at $0.15 per million input tokens and $0.60 per million output, with cache hits billed at $0.075. Around it: GPT-OSS 20B at $0.075/$0.30, Llama 4 Scout at $0.11/$0.34, Llama 4 Maverick at $0.20/$0.60, Llama 3.3 70B at $0.59/$0.79, Qwen3 32B at $0.29/$0.59, Kimi K2 at $1.00/$3.00 with cached input at $0.50, and Llama 3.1 8B Instant at $0.05/$0.08 — the cheapest text rate of any major hosted platform. A DeepSeek R1 distill on Llama 70B runs $0.75/$0.99.

The pitch is latency, not price. Groq's custom LPU hardware pushes hundreds of output tokens per second on a single stream, which matters when the model response is the product experience — voice agents, live coding assistants, interactive demos. The trade is a narrower catalog than Together or Fireworks and rate limits that scale with your tier: the free tier caps at 30 requests per minute and 14,400 requests per day, which is enough to build and test, not enough to ship on. Batch jobs take 50% off the standard rate, and prompt caching discounts cached input roughly in half with no fee to write the cache.

Together AI Pricing: The Deepest Open-Weight Catalog

Together AI pricing spans four product layers: serverless tokens, provisioned throughput, dedicated instances, and raw GPU clusters. On the serverless layer, GPT-OSS 120B runs the same $0.15/$0.60 as Groq and Fireworks; DeepSeek V4.1 Flash runs $0.30/$1.20 with cached input at just $0.006 — a 98% discount on repeat context; DeepSeek V4 Pro 0813 runs $1.32/$3.96, matching DeepSeek's own API peak rates; GLM-5.3 runs $1.40/$4.40; Kimi K3 runs $3.00/$15.00; MiniMax M3.1 Flash runs $0.15/$0.60; and Qwen2.5 7B Instruct Turbo runs $0.30/$0.30. One catalog model, Ternary Bonsai 27B, is listed free. Everything on the list can be geo-pinned to US regions at roughly a 25% premium.

The dedicated layers are where the platform starts to resemble a cloud provider. Dedicated inference instances run $5.49 per GPU-hour on H100, $5.99 on H200, $9.29 on B200, $11.49 on B300, and $15.79 on a GB300 NVL72 rack. Raw clusters undercut that: an H100 on-demand costs $3.99 per hour, or $1.99 preemptible. Fine-tuning is priced per training token — Llama 3.3 70B supervised fine-tuning costs $2.03 per million tokens with a $4 minimum, Llama 4 Scout $2.10. If you want a model nobody serverless-hosts, Together is usually where it appears first; the catalog holds dozens of open-weight releases the other two token platforms never picked up. Pair it with a self-hosted vector database and the whole open RAG stack runs on one invoice. The cost is complexity: four billing meters, and the difference between $3.99 and $5.49 for nominally the same GPU depends on which product page you walked in through.

Fireworks AI Pricing: One Model, Five Price Points

Fireworks AI pricing collapses into one idea: pick a mode, get a price. Standard is the default. Priority raises rates about 20% — GPT-OSS 120B moves from $0.15/$0.60 to $0.18/$0.72 — to jump the queue at peak load. Fast guarantees 100+ tokens per second for interactive apps. Batch takes 50% off input and output for jobs that can wait. Spot takes 30% off Standard on best-effort capacity that can throttle to zero. No other platform here prices urgency this explicitly: throw background summarization at Batch, reserve Standard for user-facing turns, push re-indexing to Spot.

The base rates sit exactly on the market floor — GPT-OSS 120B at $0.15/$0.60, DeepSeek V4.1 Flash at $0.30/$1.20, GLM-5.3 at $1.40/$4.40, Kimi K3 at $3.00/$15.00, all matching Together to the cent. The caching story is where Fireworks pulls ahead: cached input bills at half price "or less, with no write fee," and on GPT-OSS 120B the cached rate is $0.015 per million tokens — 90% off input, four times cheaper than Groq's cache rate. Session affinity keeps an agent's context on one server, and the docs claim cache hit rates up to 99.6% on long runs. Below the token layer sit on-demand GPUs at $8.00 per hour for H100 or H200, $13 for B200, $15 for B300, with idle deployments scaling to zero after one hour and region-restricted deployments costing 1.5x. Two enterprise details close the pitch: Fireworks is a first-party provider inside Microsoft Foundry, so usage counts toward Azure commit, and zero data retention is the default with SOC 2 Type II and ISO 27001 on the trust page.

Replicate Pricing: The Meter Runs in Seconds

Replicate pricing works on a different axis entirely: most public models bill by the time they take to run, on the hardware you pick. The hardware card runs from CPU at $0.36 per hour, through a T4 at $0.81, an L40S at $3.51, an A100 80GB at $5.04, up to an H100 or H200 at $5.49 per hour — the H200 reserved for committed-spend contracts. An 8x H100 box runs $43.92 per hour. A handful of models break the pattern with fixed output pricing: DeepSeek R1 bills $3.75 per million input tokens plus $0.01 per thousand output tokens — $10 per million output — while FLUX 1.1 Pro bills $0.04 per image, FLUX Schnell $3 per thousand images, and Wan 2.1 video-to-image runs $0.09 per second of 480p output or $0.25 at 720p.

The meter that matters is the private one. Your own models — anything deployed with Cog, including fine-tunes — run on dedicated hardware, and you pay for every minute they are online: setup, idle, and active. The exception is fast-booting fine-tunes, which bill only while processing a request. Public models never charge idle time; a bursty app that hits FLUX a hundred times a day for fifteen seconds each pays about $0.34 a month on a T4. That asymmetry is the whole Replicate decision: for media workloads and models nobody else hosts, it is the only shelf in the store. For pure text serving, the per-token platforms are structurally cheaper, and the crossover math below shows by how much.

Worked Bills: The Same Tokens on Four Meters

GPT-OSS 120B pricing happens to be identical on all three token platforms — $0.15 input, $0.60 output — which turns the comparison into a clean control experiment. Fix the workload, move the volume, and the bills track perfectly:

Monthly workload (in / out)GroqFireworksTogether AIReplicate
2M / 0.6M tokens (side project)$0.66$0.66$0.66n/a — no per-token GPT-OSS
20M / 6M tokens (small app)$6.60$6.60$6.60$135 on DeepSeek R1 ($3.75/$10)
200M / 60M tokens (growing product)$66.00$66.00$66.00$4,008 dedicated H100, 24/7
2B / 600M tokens (scale)$660.00$660.00$660.00$4,008 dedicated H100, still one box

Two things hide in that flat line. First, caching. A chat app where 80% of input tokens hit cache pays $5.40 on Groq, $4.44 on Fireworks, and — switching to DeepSeek V4.1 Flash to get a published cached rate — $8.50 on Together for a workload that costs $13.20 uncached. Same base rates, 18%, 33%, and 36% discounts respectively, because each platform prices the cache differently. Second, Replicate's per-token alternative is not a rounding error: its hosted DeepSeek R1 costs $135 for the workload the token platforms serve for $6.60 to $13.20 — and that R1 rate is per-token; DeepSeek V4's own API rates run lower still, as tracked on the DeepSeek V4 tool page. This is the part of LLM inference pricing that sticker rates hide — the billing unit, not the rate, decides the bill.

Per Token vs Per Second Billing: Where the Crossover Sits

Per token vs per second billing comes down to one question: can you keep the hardware busy? A dedicated H100 on Replicate costs $5.49 per hour, or about $4,008 a month running flat out. Against the $0.285 blended rate of the token platforms — a 70/30 input/output mix on GPT-OSS 120B — that H100 breaks even at roughly 14 billion total tokens per month. That volume exceeds what a single H100 can realistically serve on a 120B model, so for text serving, per-second dedicated billing loses to per-token at every scale a small team actually runs. Replicate GPU pricing is not a competitive threat to the token platforms; it is a different product aimed at workloads they do not sell.

Where the per-second model genuinely wins: image and video generation, which none of the token platforms serve; long-tail fine-tunes that Together would require you to host on dedicated instances anyway; and bursty public-model traffic, where Replicate charges nothing between requests. If the workload is raw GPU rental rather than managed inference, the GPU cloud providers — RunPod, Lambda, Vast.ai — sit below everyone here, with H100s around $2 to $3 per hour bare; what you give up is the API layer, autoscaling, and the person who pages you at 3 a.m. when the node dies.

Free Tiers and the Cheapest Start

The cheapest LLM inference to start with is free: Groq's tier gives the full open-weight catalog at 30 requests per minute and 14,400 requests per day, no card required to prototype, which makes it the default sandbox for open source LLM hosting experiments. Together's free Ternary Bonsai 27B is a real hosted model, not a trial, though a 27B unknown-quantity model is a niche pick. Fireworks starts you on prepaid credits with no commitment — the floor is a $0.05/$0.20 catalog model, Nemotron 3.5 Lightning 30B. Replicate publishes no free tier; the cheapest hardware is a $0.36-per-hour CPU, and GPU experiments start at $0.81 on a T4. For production traffic on a budget: Groq Llama 3.1 8B Instant at $0.05/$0.08 is the cheapest capable text rate, Fireworks Batch mode halves it again for async work, and Together's DeepSeek V4.1 Flash at $0.30/$1.20 with a $0.006 cache rate is the budget pick when context is long and repetitive.

Frequently Asked Questions

How much does Groq cost per million tokens?

GPT-OSS 120B runs $0.15 input / $0.60 output, with cache hits at $0.075. Llama 4 Scout is $0.11/$0.34, Maverick $0.20/$0.60, Llama 3.3 70B $0.59/$0.79, and Llama 3.1 8B Instant $0.05/$0.08. Batch jobs take 50% off. Groq pricing stays pure pay-as-you-go with no subscription, and the rate card has not moved since August 2026.

Is Fireworks AI cheaper than Together AI?

On headline rates they are identical — GPT-OSS 120B $0.15/$0.60, DeepSeek V4.1 Flash $0.30/$1.20, Kimi K3 $3.00/$15.00 on both. The gap opens in the levers: Fireworks sells Batch (50% off), Spot (30% off), and Priority (+20%) on the same endpoint, with GPT-OSS cached input at $0.015; Together sells US geo-pinning at +25%, dedicated instances from $5.49 per hour, and a published $0.006 cache rate on DeepSeek Flash. Neither is cheaper in general — the modes you actually use decide it.

How much does it cost to run an open-source LLM on a dedicated GPU?

A single H100 costs $5.49 per hour on both Replicate and Together's dedicated tier, about $4,008 a month running 24/7, with Fireworks at $8.00. Together's raw cluster layer undercuts that at $3.99 on-demand or $1.99 preemptible — no management API, no SLA. Against token billing at $0.285 blended, the break-even is roughly 14 billion tokens per month, which is past what one H100 serves on a 120B model. Dedicated GPUs pay off for data residency, custom weights, or guaranteed throughput — not for saving money on tokens.

Does Replicate charge when my model is idle?

It depends which shelf the model sits on. Public models bill only for the time each request takes to process — idle costs nothing, so a bursty app pays pennies. Private models deployed with Cog run on dedicated hardware and bill for every minute online, including setup and idle; the exception is fast-booting fine-tunes, which bill only while active. If your private model serves traffic twice a day, that idle meter is the bill.

What is the cheapest way to serve an open-source model in 2026?

Stack the discounts: pick the smallest model that passes your evals (Llama 3.1 8B Instant at $0.05/$0.08 on Groq, Nemotron 3.5 Lightning at $0.05/$0.20 on Fireworks), enable prompt caching where input repeats, and route anything non-urgent through Batch modes at 50% off. LLM inference cost per million tokens lands between $0.10 and $0.30 blended for most chat workloads that way. Self-hosting only beats that past multi-billion-token volumes, and only against raw GPU clusters rather than managed platforms.

Bottom Line

LLM inference pricing has converged on the headline numbers — three platforms selling the same model at the same $0.15/$0.60 within a cent of each other — and diverged everywhere that matters. Groq: buy it when response latency is the product, and treat the free tier as the best sandbox in the category. Together AI: buy it for catalog depth and the dedicated-GPU ladder when a model you need exists nowhere else serverless. Fireworks: buy it for the mode system and the sharpest caching deal in the group — a 33% effective discount on a realistic chat workload, before Batch mode halves async jobs. Replicate: buy it when the output is images, video, audio, or a fine-tune with no per-token home; skip it for text serving, where its own DeepSeek R1 rate runs 20 times the token platforms' equivalent. Check the meter before the rate — the platform that bills the way your workload behaves is the one that comes in cheap.

About the author: This article was written by the AI Tool Lab Editorial Team, with 5+ years of paid AI tool testing experience and $200+ monthly subscription spend. All reviews are based on real paid long-term use.

Data statement: All data in this article cites its source and is verifiable. Found an error? Report it via our contact page, we verify within 48 hours.