Cum să faci ieșirile LLM structurate portabile în OpenAI, Anthropic și Gemini

Structured LLM outputs become fragile when teams treat provider support as interchangeable. OpenAI, Anthropic, and Gemini can all produce structured data, but they do not expose the same contract. Schema syntax, tool-call envelopes, strictness guarantees, refusal behavior, streaming behavior, and validation failure modes differ enough that a parser that works with one model can silently fail with another.

The practical fix is to stop treating provider-specific structured output features as the business contract. Define one canonical output contract for each workflow, compile it into each provider's supported request format, validate every response against the canonical schema, and test each model before it is allowed to serve production traffic.

What Is Portable Structured Output?

Portable structured output means the application can rely on the same business-level response shape even when the underlying model provider changes. It does not mean every provider receives the exact same JSON Schema. It also does not mean provider-side validation is enough.

A portable contract has five layers:

• Canonical schema: the workflow's provider-neutral data shape.
• Portability linting: checks that the schema avoids constructs that are unsupported, weakly enforced, or risky across providers.
• Provider compilation: translation from the canonical contract into OpenAI, Anthropic, Gemini, or another provider's native structured-output surface.
• Runtime validation: application-side validation of the returned object against the canonical schema.
• Contract tests: repeatable tests that prove each model handles required fields, enums, refusals, truncation, malformed requests, and semantic edge cases before production routing changes.

This pattern is especially important for an enterprise LLM API where several teams, tenants, or partner workflows depend on predictable output contracts.

Facts: Provider Structured Output Is Not One Standard

Before designing the portability layer, separate documented provider behavior from assumptions.

OpenAI

OpenAI documents Structured Outputs as distinct from JSON mode. JSON mode is intended to ensure valid JSON, while Structured Outputs are intended to match a supplied JSON Schema. OpenAI supports structured output through response formatting and through function calling with strict schema adherence, but the documented behavior includes model and version constraints and a supported schema subset. Unsupported schemas in strict mode can produce API errors, and applications still need to handle edge cases.

Anthropic

Anthropic documents Claude tool use as structured tool calls based on developer-provided tools with an input_schema. The model returns a tool_use block, and the application executes the requested tool. This resembles function calling, but it is not the same contract as an OpenAI response-format JSON schema response. Anthropic's engineering guidance also notes that JSON Schema can define structure, required fields, and enums, but cannot express usage patterns such as when optional parameters should be included or which field combinations make business sense.

Gemini

Google's Gemini API documentation says structured output can be configured for JSON responses and that Gemini structured output mode supports a subset of the JSON Schema specification. That matters because a schema that is valid JSON Schema may still be unsuitable as a portable provider request schema.

Observability

OpenTelemetry GenAI semantic conventions include attributes for provider, requested model, response model, output type, finish reasons, token usage, tool definitions, tool call arguments, and related GenAI request and response fields. These fields can support cross-provider debugging when structured-output failures are hard to distinguish from prompt, schema, model, or provider issues.

Step 1: Define the Canonical Business Schema

Start with the business workflow, not the provider API. A support-triage workflow, invoice extraction workflow, approval recommendation, or partner-facing automation endpoint should each have a canonical schema that represents what the application needs downstream.

For example, a support-triage workflow might use this canonical shape:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["category", "priority", "summary", "requires_human_review"],
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "technical", "security", "account", "other"]
    },
    "priority": {
      "type": "string",
      "enum": ["low", "normal", "high", "urgent"]
    },
    "summary": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500
    },
    "requires_human_review": {
      "type": "boolean"
    },
    "customer_risk_signals": {
      "type": "array",
      "items": { "type": "string" },
      "maxItems": 5
    }
  }
}

This schema is the contract the rest of the application trusts. Provider-specific request formats are generated from it; they are not the source of truth.

Use explicit required fields, bounded arrays, bounded strings, closed enums, and additionalProperties: false where your validator supports it. Avoid accepting arbitrary nested objects unless the downstream system is designed to handle arbitrary keys.

Step 2: Add a Schema Portability Lint Step

A portability linter should reject schemas that are technically valid JSON Schema but risky for multi-model use. The goal is not full JSON Schema expressiveness. The goal is a stable output contract that can be compiled into several provider surfaces without weakening the application guarantee.

Useful lint checks include:

• Reject unsupported composition keywords unless your compiler has explicit provider mappings for them.
• Reject unconstrained object fields that allow arbitrary properties.
• Require enums for classification fields that drive routing, billing, approvals, or partner-visible state.
• Require maxItems for arrays used in automation or UI display.
• Require maxLength for text fields that enter databases, queues, or partner APIs.
• Flag optional fields whose absence changes business meaning.
• Reject schemas whose nesting or size exceeds known provider limits in your capability matrix.

The linter should run in CI and before schema deployment. Do not wait until a production request hits a provider error because a schema keyword was unsupported.

Step 3: Compile Into Provider-Specific Request Formats

Provider compilation translates the canonical schema into the API surface each model actually supports. The compiler should be deterministic and versioned. It should also leave an audit trail showing which canonical schema version produced which provider request payload.

OpenAI compilation

For OpenAI, the compiler may target Structured Outputs through response formatting for extraction and classification tasks, or strict function calling for action-oriented workflows. The compiled payload should include the supported subset of the schema and provider-specific strictness settings when available.

Anthropic compilation

For Anthropic, the compiler will often represent the desired output as a tool with an input_schema. The application then expects a tool-use block and validates the tool input against the canonical schema. This means the adapter must normalize the response envelope, not just parse JSON.

Gemini compilation

For Gemini, the compiler should target its structured JSON response configuration using only the supported schema subset. If a canonical schema uses constraints that Gemini does not enforce natively, the compiler should record those constraints as requiring application-side validation.

The adapter output should be normalized into one internal result type:

type StructuredResult = {
  provider: "openai" | "anthropic" | "gemini";
  model: string;
  schema_version: string;
  status: "ok" | "refusal" | "truncated" | "provider_error" | "invalid";
  raw_response: unknown;
  parsed_output?: unknown;
  finish_reason?: string;
};

This normalization keeps downstream workflow code from depending on provider envelopes.

Step 4: Validate Every Response Against the Canonical Schema

Provider-side structure is the first line of defense, not the final correctness check. Always validate the parsed output against the canonical schema after the model responds.

Runtime validation should produce structured errors such as:

• missing_required_field
• invalid_enum
• unexpected_property
• type_mismatch
• array_too_long
• string_too_long
• semantic_rule_failed

Keep structural validation separate from semantic validation. JSON Schema can check shape, required properties, enums, and basic type constraints. It cannot fully express business rules such as valid state transitions, approval thresholds, customer-specific policy, or mutually dependent field combinations.

For example, this is a semantic validator, not a schema rule:

if (output.priority === "urgent" && output.requires_human_review !== true) {
  throw new SemanticValidationError(
    "urgent tickets must require human review"
  );
}

Put those checks in application code where they can use customer configuration, workflow state, authorization context, and policy versioning.

Step 5: Use Bounded Repair, Then Fail Closed

A repair path can recover from small output defects, but it can also hide systematic schema and prompt problems. Use one targeted repair attempt with the validation error included. If the repaired response still fails validation, fail closed or route to review.

A practical repair policy looks like this:

• Attempt repair only for eligible failure types, such as missing required fields, invalid enums, or malformed JSON.
• Do not repair provider refusals, safety blocks, authorization failures, or policy-sensitive outputs.
• Include the canonical schema version and validation error in the repair prompt.
• Allow at most one repair attempt for customer-visible or action-triggering workflows.
• Log repair count, repair success, token usage, and final status.

For low-risk internal extraction jobs, a second repair attempt may be acceptable if latency is not important. For workflows that trigger payments, access changes, partner API responses, or compliance actions, a fail-closed policy is usually safer.

Step 6: Build Contract Tests for Each Production Model

Contract tests decide whether a model is eligible for a workflow. They should run before initial launch, before model upgrades, and before routing policy changes.

At minimum, test each model and provider adapter for:

• Valid happy-path output.
• Missing required fields.
• Invalid enum values.
• Extra properties when the schema is closed.
• Too many array items.
• Overlong strings.
• Refusal or inability to comply.
• Truncated output.
• Tool-call mismatch or missing tool call.
• Structurally valid but semantically wrong output.

Use adversarial prompts that pressure the model to violate the contract. For example, ask it to add an extra explanatory field, return a priority outside the enum, or omit a required boolean. The test passes only if the provider plus adapter plus validator produce the expected application status.

For each workflow, maintain a simple compatibility record:

{
  "workflow": "support_triage",
  "schema_version": "2026-09-24",
  "model": "provider-model-name",
  "supports_strict_schema": true,
  "supports_tool_calling": true,
  "supports_streaming_structured_output": false,
  "refusal_signal_normalized": true,
  "contract_tests_passed": true,
  "known_limitations": ["max schema depth 4"]
}

This capability matrix belongs in the control plane, not in scattered application constants.

Step 7: Instrument Structured-Output Reliability

Without telemetry, teams cannot tell whether failures come from provider incompatibility, model drift, schema defects, prompt defects, truncation, or downstream validators.

Log these fields for every structured-output request:

• Workflow name and schema version.
• Provider, requested model, and response model.
• Output mode, such as response schema, tool call, or JSON mode.
• Validation status and failure type.
• Repair count and repair result.
• Finish reason and refusal status.
• Prompt tokens, completion tokens, total cost, and latency.
• Adapter version and compiler version.

Use this data to compare models before changing routing. A cheaper model is not cheaper if it increases repair attempts, manual review, partner API failures, or downstream incident handling.

Trade-Offs to Decide Explicitly

Lowest-common-denominator schemas improve portability but weaken constraints. If the provider request schema must be simplified, preserve stricter checks in canonical validation and semantic validators.

Provider-specific compilation preserves native strengths but adds maintenance. This is usually worth it for production workflows, but only if compilers are versioned and covered by contract tests.

Strict schemas reduce parser failures but require better task design. If the task is underspecified, strictness exposes the ambiguity sooner. That is useful, but it can slow initial development.

Repair improves completion rates but adds cost and latency. Bounded repair is a recovery mechanism, not a substitute for schema design.

Tool calling and response-format JSON serve different workflows. Tool calling is often a better fit for action-oriented workflows. Response-format JSON is simpler for extraction, classification, and summarization. A multi-model platform can support both, but the workflow policy must be explicit.

Predictions: Where Structured Output Portability Is Going

Provider support for structured output will likely continue improving, but full JSON Schema portability across models should not be assumed. Providers have different model architectures, API histories, tool-use patterns, and safety behavior. Even when schema support becomes stronger, applications will still need canonical validation because business correctness depends on more than JSON shape.

Teams should expect structured-output routing decisions to become more data-driven. Reliability, repair rate, refusal handling, latency, and cost will matter alongside model quality. The operational question will shift from "does this model support JSON?" to "does this model satisfy this workflow's contract at an acceptable cost and failure rate?"

Implementation Checklist

• Define one canonical schema per workflow.
• Version schemas and keep provider request payloads reproducible.
• Add CI linting for unsupported or risky schema features.
• Compile canonical schemas into provider-specific formats.
• Normalize provider envelopes into one internal result type.
• Validate every parsed response against the canonical schema.
• Add semantic validators for business rules.
• Allow only bounded repair attempts.
• Fail closed or route to review for high-risk workflows.
• Run contract tests before adding or upgrading production models.
• Maintain a model capability matrix in the control plane.
• Log validation, repair, cost, latency, provider, and model fields for every request.

Actionable Conclusion

Structured output portability is an API contract problem, not a prompt formatting problem. The reliable pattern is to define a canonical business schema, lint it for portability, compile it into each provider's native surface, validate every response after the provider returns it, and use contract tests to decide which models are production-eligible.

This design adds some upfront engineering work, but it prevents the more expensive failure mode: downstream systems assuming a stable contract while model routing, provider behavior, or schema enforcement quietly changes underneath them.

Related reading

• fallback routing that protects structured output contracts