Async requests

Model Gate async mode stores an inference job in the durable MariaDB queue and immediately returns a request ID. Independent sub2proxy workers claim queued jobs with leases, and abandoned leases can be recovered after a process restart. The completed result is read from the Partner API or delivered to the callback configured in Profile.

Async mode is supported for:

• POST /v1/chat/completions
• POST /v1/messages
• POST /v1/responses
• POST /v1/images/generations

Async requests cannot use stream: true.

Queue request

Add the Model Gate extension "async": true to an otherwise valid request.

Request

curl https://api.model-gate.com/v1/responses \
  -H "Authorization: Bearer mg_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Write a short release summary.",
    "async": true,
    "stream": false
  }'

Response — 202

{
  "request_id": "01KZ...",
  "status": "queued",
  "result_url": "https://p-api.model-gate.com/api/v1/requests/01KZ..."
}

The async field is removed before the request is sent upstream. A model alias is resolved to its canonical model before the job is queued. The worker re-validates the key and executes through the same Model Gate request/accounting core as synchronous inference. Queue rows contain no API-key secret. Native async pricing is unchanged and does not use the batch request price coefficient.

Before a worker claims an item it checks the current account balance and the current resettable key/group spend usage in MariaDB. If the account balance is not positive, or a configured key/group spend limit is already exhausted, the item remains queued and does not consume an execution attempt. A later top-up, usage reset, or limit increase makes the item eligible automatically. Model Gate does not reserve an estimated request cost: work that has already passed admission is allowed to finish and settle its complete actual cost, even when concurrent work makes the final balance negative or slightly exceeds a spend limit. New executions are then blocked until the account is eligible again.

Recovery semantics

The durable queue provides at-least-once recovery, not exactly-once execution. If a worker process stops after an upstream provider accepted a request but before Model Gate durably stores the async result, the lease eventually expires and another worker may retry the same Model Gate request ID. Settlement is guarded against double-debit, but applications that cause external side effects through model tools should make those side effects idempotent.

Retrieve result

Use a Partner API key belonging to the same Model Gate account.

Request

curl https://p-api.model-gate.com/api/v1/requests/01KZ... \
  -H "Authorization: Bearer mg_partner_..."

Response — processing

{
  "data": {
    "request_id": "01KZ...",
    "status": "processing",
    "created_at": "2026-08-05 10:00:00",
    "started_at": "2026-08-05 10:00:01",
    "completed_at": null,
    "expires_at": "2026-08-06 10:00:00"
  }
}

Response — completed

{
  "data": {
    "request_id": "01KZ...",
    "status": "completed",
    "response_status": 200,
    "response_headers": {
      "content-type": ["application/json"]
    },
    "response": {
      "id": "resp_...",
      "object": "response",
      "status": "completed"
    },
    "created_at": "2026-08-05 10:00:00",
    "started_at": "2026-08-05 10:00:01",
    "completed_at": "2026-08-05 10:00:03",
    "expires_at": "2026-08-06 10:00:00"
  }
}

The result endpoint returns the complete stored upstream body until expires_at. A foreign, expired, or unknown request ID returns 404.

Errors

{
  "error": {
    "type": "invalid_request_error",
    "code": "invalid_async_stream_combination",
    "message": "Async requests cannot be streamed."
  }
}

Unsupported endpoints return async_not_supported.

Callbacks

Completed async results are durably enqueued for delivery to the callback URL configured in Profile. Callback delivery has its own worker/retry queue and therefore does not wait for another Model API request. Maintenance also reconciles terminal async rows into request history and recovers a missing completion callback after a worker crash. See Callbacks for signature verification, retries, and idempotency guidance.