OpenAI-compatible Batch API

Model Gate implements the OpenAI Files + Batch workflow on https://api.model-gate.com/v1. This is a compatibility layer: each JSONL item is executed through the normal Model Gate inference path. Model Gate does not submit an upstream provider-native OpenAI batch.

Files/Batch read/control endpoints are a control plane: an otherwise valid active API credential may list/read/download/cancel/delete existing resources even when the account balance is currently zero or a resettable spend limit is exhausted. Storage-producing operations are different: POST /v1/files and POST /v1/batches require fresh positive-balance/spend admission before Model Gate inserts file/job/item data. A zero-balance key therefore cannot upload JSONL or create new batch storage. Actual batch items still re-check admission at worker claim time and remain queued if funds are later exhausted. JSONL input is read incrementally and validated/inserted line by line; Model Gate does not retain the complete 200 MB input plus all request bodies in process memory while creating a batch. Per-user stored-byte/file/active-job/queued-item quotas provide an independent database-abuse boundary.

Supported batch endpoints in this release are:

• /v1/responses
• /v1/chat/completions
• /v1/embeddings
• /v1/images/generations

Every executed item is marked request_mode=batch, batch_protocol=openai, with its batch_job_public_id and custom_id.

1. Upload a JSONL input file

Each non-empty line contains custom_id, method, url, and body. The URL must equal the endpoint later supplied to /v1/batches.

Example batch.jsonl:

{"custom_id":"request-1","method":"POST","url":"/v1/responses","body":{"model":"gpt-5.4","input":"Summarize this text."}}
{"custom_id":"request-2","method":"POST","url":"/v1/responses","body":{"model":"gpt-5.4","input":"Classify this text."}}

Request

curl https://api.model-gate.com/v1/files \
  -H "Authorization: Bearer mg_live_..." \
  -F "purpose=batch" \
  -F "[email protected]"

Response — 200

{
  "id": "file-01K...",
  "object": "file",
  "bytes": 322,
  "created_at": 1786610000,
  "filename": "batch.jsonl",
  "purpose": "batch",
  "status": "processed",
  "status_details": null
}

2. Create the batch

completion_window must be 24h.

Request

curl https://api.model-gate.com/v1/batches \
  -H "Authorization: Bearer mg_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id":"file-01K...",
    "endpoint":"/v1/responses",
    "completion_window":"24h",
    "metadata":{"job":"nightly-evaluation"}
  }'

Response — 200

{
  "id": "batch_01K...",
  "object": "batch",
  "endpoint": "/v1/responses",
  "input_file_id": "file-01K...",
  "completion_window": "24h",
  "status": "in_progress",
  "output_file_id": null,
  "error_file_id": null,
  "request_counts": {
    "total": 2,
    "completed": 0,
    "failed": 0
  },
  "metadata": {
    "job": "nightly-evaluation"
  }
}

3. Retrieve a batch

Request

curl https://api.model-gate.com/v1/batches/batch_01K... \
  -H "Authorization: Bearer mg_live_..."

Response — completed

{
  "id": "batch_01K...",
  "object": "batch",
  "endpoint": "/v1/responses",
  "status": "completed",
  "output_file_id": "file-01KOUTPUT...",
  "error_file_id": null,
  "request_counts": {
    "total": 2,
    "completed": 2,
    "failed": 0
  },
  "usage": {
    "input_tokens": 240,
    "input_tokens_details": {"cached_tokens": 0},
    "output_tokens": 90,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 330
  }
}

4. Download results

Request

curl https://api.model-gate.com/v1/files/file-01KOUTPUT.../content \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{"id":"batch_req_01K...","custom_id":"request-1","response":{"status_code":200,"request_id":"01K...","body":{"id":"resp_...","status":"completed"}},"error":null}

Failed, canceled, or expired items are written to error_file_id as JSONL records with response:null and an error object.

5. List batches

limit defaults to 20 and must be from 1 to 100. Use after for cursor pagination.

Request

curl "https://api.model-gate.com/v1/batches?limit=20&after=batch_01K..." \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{
  "object": "list",
  "data": [],
  "first_id": null,
  "last_id": null,
  "has_more": false
}

Completed batch objects may include an aggregate usage object when settled Model Gate request accounting is available.

6. Cancel a batch

Cancel prevents queued items from starting; an item already processing may finish.

Request

curl -X POST https://api.model-gate.com/v1/batches/batch_01K.../cancel \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{
  "id": "batch_01K...",
  "object": "batch",
  "status": "cancelling",
  "request_counts": {
    "total": 2,
    "completed": 0,
    "failed": 0
  }
}

7. File metadata, listing, and deletion

Retrieve metadata:

curl https://api.model-gate.com/v1/files/file-01K... \
  -H "Authorization: Bearer mg_live_..."

{
  "id": "file-01K...",
  "object": "file",
  "bytes": 322,
  "filename": "batch.jsonl",
  "purpose": "batch",
  "status": "processed",
  "expires_at": 1789202000
}

List files with optional purpose, after, and order=asc|desc; limit defaults to 10000 and must be from 1 to 10000:

curl "https://api.model-gate.com/v1/files?purpose=batch&order=desc&limit=100" \
  -H "Authorization: Bearer mg_live_..."

{
  "object": "list",
  "data": [],
  "first_id": null,
  "last_id": null,
  "has_more": false
}

Delete an unreferenced/expired-compatible file:

curl -X DELETE https://api.model-gate.com/v1/files/file-01K... \
  -H "Authorization: Bearer mg_live_..."

{
  "id": "file-01K...",
  "object": "file",
  "deleted": true
}

Limits and recovery semantics

Model Gate accepts up to 50,000 JSONL items and limits the uploaded file to the configured OPENAI_BATCH_MAX_FILE_BYTES value (200 MiB by default). Each individual batch item must also fit Model Gate's normal MAX_REQUEST_BODY_BYTES limit. custom_id values must be unique. Batch items cannot use stream:true or nested async:true. Large JSONL files are persisted internally as database chunks rather than one oversized SQL value.

Execution recovery is at-least-once, not exactly-once. If a worker stops after an upstream request was accepted but before its result is durably recorded, an expired lease can cause the same Model Gate request ID to be retried. Settlement remains idempotent for already-finished request rows, but tool/external side effects initiated by a model should themselves be idempotent.

Each JSONL item is admitted immediately before a worker claims it. Otherwise-valid items stay queued while the current account balance is non-positive or the key/group resettable spend limit is already exhausted. This waiting state does not consume an attempt or create an error record; a later top-up, usage reset, or limit increase automatically makes the remaining items eligible. Model Gate does not reserve a theoretical maximum batch cost. Items already admitted may therefore settle in full even when concurrent work makes the final balance negative or produces a small spend-limit overshoot, after which new items remain queued until the account is eligible again.

Pricing and accounting

The batch adapter does not use provider-native batch execution. Every item goes through normal Model Gate model routing and settlement, then receives the pricing template's Batch request price coefficient. Default: 1.

If the coefficient is 0.5, an item whose normal Model Gate cost is 0.02 is debited as 0.01. Official reference pricing remains unchanged. The applied coefficient is snapshotted and stored on the request for audit.

Errors

{
  "error": {
    "message": "line 2 url must match batch endpoint /v1/responses",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_batch_file"
  }
}