Requirement for the AI integration: your AI provider must support strict structured outputs (response_format: json_schema)

Background

In preeco | data protection and preeco | information security you store the AI provider yourself: under Settings → Integrations → AI provider you choose between OpenAI, Langdock and Custom (OpenAI-compatible) and enter the API key, the model and – for the custom provider – the API endpoint. This way the contractual relationship, the choice of model and the data flow remain with you.

For this to work, it is not enough for an endpoint to be "OpenAI-compatible". It must support a specific capability: strict structured outputs via response_format: json_schema ("Strict Structured Outputs"). If the endpoint or the model does not support this, the AI integration fails – already when saving the settings.

The technical requirement

preeco calls exclusively the chat completions path of your endpoint, that is {your base URL}/chat/completions. Every single AI request – without exception – contains a response format with a strict JSON schema:

{
  "model": "…",
  "messages": [ … ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "…",
      "schema": { … },
      "strict": true
    }
  }
}

The endpoint must accept this field and the model must actually produce the response in conformity with the schema. Both are required:

• The endpoint must not reject response_format and must not silently ignore it either.
• The model must master strict structured outputs. A plain "JSON mode" (response_format: {"type": "json_object"}) is not sufficient – it only guarantees valid JSON, not the prescribed structure.
• A model that merely "reproduces" the schema by way of an instruction in the prompt is not sufficient either.

Why preeco requires this

In preeco, AI results do not end up as running text in a chat window but directly in structured fields – for example in risk analyses, protective measures, technical and organizational measures, guidelines, privacy policies and the assessment of data breaches. Each of these fields has a fixed structure.

That is why preeco sends the expected schema with the request and afterwards checks the response once again on the server side against the same schema. If the response deviates, it is discarded instead of writing incomplete or incorrectly assigned content into your documentation. A provider without strict schema support fails at this second check at the latest.

Checking before setting up

Whether the endpoint you have in mind is suitable can be determined with a single call. Replace the base URL, API key and model name and run the following command:

curl -s https://your-endpoint.example.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "YOUR_MODEL",
    "messages": [{"role": "user", "content": "Return the status."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "status_check",
        "schema": {
          "type": "object",
          "properties": {"status": {"type": "string", "enum": ["ok"]}},
          "required": ["status"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
  }'

The endpoint is suitable if the response contains exactly {"status":"ok"} as its content.

It is not suitable if an error such as "response_format is not supported", "unknown parameter" or "strict is not supported" is returned instead – or if the model answers with explanatory running text instead of the JSON object. In these cases the AI integration in preeco will not work; this cannot be worked around on the application side.

Further requirements for a custom endpoint

• HTTPS is mandatory. HTTP endpoints are rejected.
• No internal or private network addresses. The endpoint must be publicly reachable.
• Enter only the base URL, for example https://api.example.com/v1/. If you accidentally include the full path with /chat/completions, preeco removes it automatically and points this out – the application appends the path itself.
• The model name must correspond exactly to the model ID of the provider, including spelling and version. A display name from a user interface usually does not work.
• The API key needs write permissions for chat completions. A key that may only list models is not sufficient.

What happens if the requirement is not met

When saving the AI settings, preeco performs a real test call against your endpoint. The settings are only saved if this test was successful – so a non-functioning configuration cannot even be stored in the first place.

If the model does not respond in conformity with the schema, the following message appears directly in the form: "The model did not respond in the expected format. Please make sure that the model supports structured JSON outputs. You may need to adjust the configuration or the model."

If a provider that has already been configured later changes its behaviour – for example after the provider switches models – the affected AI actions fail at runtime. You can recognize this in the AI activities section on the same settings page: every AI call is logged there with model, status, duration, token consumption and error type.

Error patterns and their causes

• "The model did not respond in the expected format …" – the endpoint or the model does not support strict structured outputs. Choose a different model or a provider with support for structured outputs.
• "The endpoint was not found." – there is no /chat/completions path under the specified base URL. Check whether the version identifier (for example /v1/) is missing.
• "The endpoint is not reachable." – wrong host name, DNS problem or a firewall is blocking the connection.
• "The API endpoint must use HTTPS." or "… must not point to internal or private network addresses." – the endpoint does not meet the security requirements.
• Authentication errors – the API key is invalid, has expired or does not have the necessary permissions.
• Rate limit errors – the provider's quota is exhausted. preeco repeats such calls automatically; a permanently exhausted quota, however, has to be increased with the provider.

Recommendation

Clarify the support for response_format: json_schema with strict: true with your provider or your IT operations before procuring or approving an AI endpoint – ideally with the test call shown above. This is the only hard exclusion criterion: everything else, such as model size, region or price, is your free decision, but this one capability is a prerequisite.

A practical note for operations: set up an API key with restricted permissions and a monthly budget limit with the provider. That way you keep costs under control without restricting the function.