Handling 429 and 503 errors from an LLM API

Every LLM API fails sometimes. The two errors you will meet most often in production are 429, you sent too much, and 503, there is nothing to serve you right now. Both are temporary. Retrying the wrong errors, or retrying too fast, turns a short blip into an outage. Here is what to retry, how long to wait, and when to switch models instead.

Retry, or fix the request?

Status	Retry?	What to do
400	No	The request is malformed. Fix it.
401, 403	No	Wrong, revoked, or wrong-type key.
402	No	Out of balance. Top up.
404	No	Unknown model id. Check the model list.
408	Yes	Timed out. Retry with backoff.
429	Yes	Rate limited. Wait for `Retry-After`, then retry.
500, 502	Yes, a few times	Server or upstream error. Back off.
503	Yes, or fall back	No capacity. Retry later or use another model.

The rule of thumb: 4xx means change the request, except 408 and 429, which mean wait. 5xx means wait.

429: slow down

A 429 means your key went over its rate limit. Good APIs tell you how long to wait in a Retry-After header, in seconds. Respect it. Retrying sooner only burns attempts against the same limit.

On AIKeys, the default limit is 60 requests a minute per key, counted in one-minute windows. A 429 comes back as rate_limit_exceeded with Retry-After: 60. If you hit it often, spread requests out with a client-side queue, or use more than one key for separate workloads.

503: no capacity

A 503 means the service could not place the request. On a marketplace like AIKeys it has a specific meaning: no publisher is serving that model at or below the price cap right now. The code is no_capacity, and you are not charged.

Before you see a 503, the proxy has already tried. If a publisher’s upstream times out, rate-limits, or is overloaded, the proxy retries once on a different publisher before it answers. So a 503 is worth handling in your code: wait and retry, or try a different model id.

What the SDKs already do

The official OpenAI and Anthropic SDKs retry connection errors, 408, 429, and 5xx responses twice by default, with exponential backoff. They read Retry-After when it is present. Set max_retries when you create the client to change that. For many apps, the defaults plus a fallback list is all you need.

A fallback list in Python

import os
from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.aikeys.co/v1",
    api_key=os.environ["AIKEYS_CO_KEY"],
    max_retries=1,  # retry each model once, then move on
)

# Same job, in order of preference.
MODELS = ["cc/claude-sonnet-4-6", "ag/claude-sonnet-4-6", "cx/gpt-5.6-terra"]

def complete(messages):
    last_error = None
    for model in MODELS:
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except APIStatusError as error:
            if error.status_code in (503, 404):  # no capacity, or id not listed
                last_error = error
                continue
            raise  # 400, 401, 402, 403: retrying will not help
    raise last_error

reply = complete([{"role": "user", "content": "Summarize this ticket in one line: ..."}])
print(reply.choices[0].message.content)

Two details matter. The list mixes upstreams: the same Claude Sonnet model is listed as both cc/claude-sonnet-4-6 and ag/claude-sonnet-4-6, so a capacity gap on one may not exist on the other. And it re-raises the errors that a retry cannot fix, so a bad key or an empty balance fails fast instead of looping.

Reading the error body

AIKeys answers in the shape of the endpoint you called. Chat Completions and Responses use the OpenAI shape:

{ "error": { "message": "No serving offer at or below the cap.", "type": "server_error", "code": "no_capacity" } }

Messages uses the Anthropic shape:

{ "type": "error", "error": { "type": "no_capacity", "message": "No serving offer at or below the cap." } }

Branch on the status code first and the error code second. Both SDKs expose them on the exception.

Checklist

Retry 408, 429, and 5xx. Never retry 400, 401, 402, 403, or 404.
Honor Retry-After. Add jitter to your own backoff so clients do not retry in lockstep.
Cap total attempts, and cap total time for anything a user is waiting on.
Keep a fallback model list for 503.
Log the status, the error code, and the model id, so you can see which failures are yours and which are capacity.

For setup, see how to call Claude, GPT and Gemini with one key. Every error code is in the API reference.