Partner LLM API létrehozása a nyers szolgáltatói kulcsok felfedése nélkül

A Partner API for LLM access should not be a thin wrapper around provider keys. If partners, resellers, or customer applications need delegated access to language models, the platform needs its own control plane, gateway, policy layer, metering pipeline, and operational runbooks. Otherwise, provider-level billing, provider-level limits, and shared credentials become the only controls between one partner integration and a very expensive incident.

The goal is straightforward: let external parties consume LLM capabilities through your platform while you keep upstream provider credentials private, enforce partner-specific entitlements, measure billable usage, and give partners enough transparency to build reliable clients.

Reference Architecture

A practical Partner API has two major parts: a control plane and a request gateway.

The control plane manages partner organizations, applications, environments, delegated API keys, model entitlements, rate-limit tiers, spend caps, billing terms, webhook endpoints, and audit records. It is where commercial and operational rules become machine-readable policy.

The request gateway receives partner traffic, authenticates the delegated key, evaluates policy, applies quota, routes to the selected model provider, records usage, and returns partner-visible response metadata. In a mature enterprise LLM API, this gateway is also where model routing, retries, failover, content controls, and observability are enforced consistently.

The important boundary is this: partners receive only your platform-issued keys. They never receive raw provider credentials. Upstream provider keys stay in your platform secret store and are used only by the gateway or backend workers.

Fact Pattern to Build Around

The following are facts supported by public platform and cloud documentation, not assumptions about any one deployment:

• Some LLM providers support project-scoped keys, service accounts, project-level model access, and project-level rate limits.
• Cloud guidance for multi-tenant SaaS APIs commonly separates policy administration, policy decision, and policy enforcement to reduce accidental cross-tenant access.
• API gateways and SaaS control planes can apply tenant-specific throttling and quota controls, including rate, burst, and quota tiers.
• Usage-based SaaS billing depends on collecting tenant activity, aggregating it, and reconciling it against billing records.
• Idempotency keys are a documented pattern for safely retrying state-changing API requests without creating duplicate side effects.
• Standardized rate-limit response headers can help clients understand limits, remaining allowance, reset timing, and retry behavior.

The recommendation is to combine these patterns into a partner-facing LLM access layer that you own, instead of asking every partner to integrate directly with each upstream model provider.

Step 1: Define the Tenant and Identity Model

Do not identify partner traffic with only one API key string. A useful Partner API needs separate identifiers because billing, abuse review, quota debugging, and support investigations often ask different questions.

At minimum, create identifiers for:

• Partner organization: the contracted external company.
• Partner application: a specific integration, product, plugin, or reseller application.
• Environment: production, staging, sandbox, or development.
• End customer: the partner's customer, workspace, account, or tenant, when contractually allowed and technically available.
• API key: the delegated credential used to authenticate requests.
• Request ID: a unique identifier for tracing one request across your gateway, provider calls, usage events, and support logs.

This structure avoids the common failure mode where one compromised or noisy integration can only be described as “partner X traffic.” It also lets you suspend one application or environment without disabling the partner's entire business.

Step 2: Issue Delegated Partner Keys

Partner keys should be first-class platform objects, not static tokens in a spreadsheet. Each key should have metadata and policy bindings:

• Partner organization ID
• Application ID
• Environment
• Allowed API surfaces, such as chat, embeddings, batch, or image generation
• Allowed model aliases
• Region or data-residency constraints where applicable
• Rate-limit tier
• Spend cap or prepaid balance rule
• Creation time, last-used time, owner, and rotation status

Store only hashed key material where possible, expose full secrets only once at creation, and support rotation without downtime. A practical rotation model allows two active keys during a short transition window, with audit logs showing which key was used for each request.

Step 3: Separate Entitlements From Routing

A partner's right to use a model should not be implied by whatever routing code currently supports. Keep entitlement checks explicit and separate from provider selection.

For example, a request for model: premium-reasoning should first pass a policy decision:

• Is this partner allowed to use the model alias?
• Is the partner application allowed to use it in this environment?
• Is the request inside the allowed region or data boundary?
• Is the contract active and in good standing?
• Is the partner inside spend, quota, and abuse thresholds?

Only after that decision should routing choose the upstream provider, provider project, deployment, or fallback path. This matters because model routing changes are operationally frequent. Entitlement changes should be deliberate and auditable.

Step 4: Publish a Stable Partner API Surface

Partners need stable behavior even when your upstream providers change. Use your own model aliases and capability descriptions instead of exposing every provider model name directly.

A small example:

POST /v1/chat/completions
Authorization: Bearer partner_key_...
Idempotency-Key: job_01J7...
X-Partner-Customer-Id: customer_123

{
  "model": "standard-chat",
  "messages": [
    {"role": "user", "content": "Summarize this support thread."}
  ],
  "metadata": {
    "ticket_id": "T-44821",
    "workflow": "support_summary"
  }
}

The partner asks for standard-chat. Your gateway maps that alias to the current approved provider and model, subject to entitlement, availability, price, latency, region, and quality rules.

Avoid hiding all provider differences, though. If two model aliases have different context windows, structured-output support, tool-calling behavior, latency profiles, or regional availability, document those differences clearly. Abstraction should simplify integration, not create surprise behavior.

Step 5: Enforce Quotas Before Provider Calls

Provider limits are useful guardrails, but partner-level enforcement belongs in your platform. One upstream provider project may serve many partners, or one partner may have several applications with different contracts. Relying only on provider-level rate limits cannot express those distinctions cleanly.

Apply quota checks before making upstream calls:

• Requests per minute by partner key
• Tokens per minute by partner organization
• Concurrent requests by application
• Daily or monthly billable unit quota
• Prepaid balance or credit threshold
• Per-end-customer caps where required

For synchronous requests, reject clearly when the request cannot be accepted. For asynchronous and batch workloads, consider queue admission separately from execution quota. A batch upload may be accepted, but individual jobs should still be metered and stopped when partner limits are reached.

Step 6: Return Predictable Rate-Limit Signals

Opaque 429 Too Many Requests responses create avoidable support tickets. Return partner-visible quota signals on successful and throttled responses.

A useful response can include:

RateLimit-Limit: 600;w=60
RateLimit-Remaining: 42
RateLimit-Reset: 17
Retry-After: 17
X-Request-Id: req_01J8...

Recommendations:

• Document whether limits apply per key, application, organization, model, or endpoint.
• Return reset timing that clients can act on.
• Use Retry-After for throttled responses.
• Keep error bodies stable and machine-readable.
• Do not expose upstream provider internals unless partners need them to debug behavior.

This makes partner clients easier to build and reduces retry storms during traffic spikes.

Step 7: Use Idempotency for Billable Work

Any endpoint that creates state or billable work should support idempotency keys. This is especially important for async generation jobs, batch submissions, credit purchases, webhook-triggered retries, and long-running tasks.

The implementation rule is simple: when a partner repeats the same request with the same idempotency key inside the retention window, return the original result instead of creating duplicate work.

Make the edge cases explicit:

• How long idempotency records are retained
• Which endpoint and partner key the idempotency key is scoped to
• What happens if the same key is reused with different request parameters
• Whether failed requests are cached
• How idempotency applies to partially completed batch jobs

For billable LLM work, store enough request fingerprint data to detect parameter mismatches without retaining unnecessary sensitive payload content.

Step 8: Emit Immutable Usage Events

Do not treat provider invoices as the source of truth for partner billing. Provider invoices usually do not know your partner contracts, markup rules, reseller tiers, end-customer IDs, or dispute workflow. Instead, emit an immutable usage event when each request reaches a terminal state.

A usage event should include:

• Partner organization ID
• Application ID and environment
• API key ID, not the raw key
• End-customer ID if provided and allowed
• Request ID and provider correlation ID
• Model alias and upstream provider model
• Input tokens, output tokens, cached tokens, or other billable units
• Provider cost basis when available
• Partner billing unit, markup, credit drawdown, or contract rate
• Status, error class, latency, and retry count
• Timestamp and schema version

These events feed partner dashboards, billing exports, anomaly detection, quota counters, and reconciliation. Keep the event stream append-only. Corrections should be adjustment events, not silent edits.

Step 9: Reconcile Daily

Metering is only useful if it reconciles. Build a daily process that compares your usage events against upstream provider usage, provider invoices, credit balances, and partner-facing billing records.

The reconciliation process should identify:

• Requests accepted by your gateway but missing terminal usage events
• Provider usage that cannot be mapped to a partner request
• Token count differences between provider reports and gateway records
• Retries that were billed incorrectly
• Manual credits, refunds, and billing corrections
• Partners nearing committed spend, prepaid exhaustion, or unusual growth

This is also where finance, support, and engineering need the same IDs. If the dashboard, invoice line item, log event, and provider record cannot be tied together, disputes become manual investigations.

Step 10: Build Operational Runbooks

A Partner API is an external product surface, so operational behavior has to be planned before incidents happen. At minimum, write runbooks for these situations:

• Compromised partner key: revoke or suspend one key, notify the partner, preserve audit logs, and provide a rotation path.
• Runaway usage: apply emergency caps, pause an application, or require approval for further usage.
• Provider outage: fail over to an approved provider or return a stable error with retry guidance.
• Quota exhaustion: show whether the partner needs to wait, upgrade, purchase credits, or reduce traffic.
• Webhook replay: verify signatures, use idempotency, and track event delivery attempts.
• Billing correction: create adjustment events and preserve the original usage records.

Runbooks should define who can take each action. Suspending one key is different from disabling a partner organization. Raising a quota is different from changing a contract. Emergency controls need audit trails.

Trade-Offs

A Partner API gateway adds latency and operational responsibility. That is the cost of controlling credentials, quotas, routing, billing, and visibility at the partner-contract level.

Strict quotas protect margins and reduce abuse, but they can create unnecessary failures if burst behavior and retry guidance are poorly designed. A useful quota system has both hard limits and documented burst behavior.

Detailed metering improves billing transparency and support outcomes, but it increases retention, privacy, and reconciliation obligations. Collect the identifiers needed for operations and billing, but avoid storing raw prompt and response content in billing records unless there is a clear contractual and compliance basis.

Model abstraction makes partner integrations simpler, but excessive abstraction can hide material differences. Partners do not need to know every provider implementation detail, but they do need to know when a model alias changes context limits, output formats, latency expectations, or regional availability.

Implementation Checklist

• Issue platform-owned partner keys instead of sharing provider credentials.
• Map every key to partner, application, environment, entitlement, quota, and billing records.
• Keep authorization and entitlement checks separate from routing logic.
• Use stable model aliases with documented capabilities and limits.
• Apply quota checks before upstream provider calls.
• Return rate-limit headers and machine-readable errors.
• Require idempotency keys for endpoints that create state or billable work.
• Emit immutable usage events at request completion.
• Reconcile usage against provider records and partner invoices.
• Maintain runbooks for key compromise, runaway usage, provider outage, quota exhaustion, webhook replay, and billing correction.

Actionable Conclusion

The safest Partner API for LLM access is not a proxy that forwards requests with hidden credentials. It is a delegated access system with its own identity model, policy layer, quota enforcement, usage meter, reconciliation process, and partner-facing transparency.

Start with the control boundaries: partner keys, explicit entitlements, pre-call quota checks, idempotent state-changing endpoints, and immutable usage events. Then add operational maturity: dashboards, reconciliation, rate-limit semantics, and incident runbooks. This architecture lets partners build on LLM capabilities without receiving raw provider keys, while giving the platform enough control to manage cost, abuse, reliability, and billing disputes.

Related reading

• designing governed API keys for teams and partner integrations