What is Structured Output?

Structured Output

• LLM answers with structured data (JSON, XML), not Markdown
  • App will not pass on LLM's answer as is, but process it using code
• In this talk, we focus on OpenAI
  • Other providers offer similar functionality (e.g. Anthropic 🔗)
• If you need XPlat, consider...
  • ... libs like Instructor 🔗
  • ... proxies like LiteLLM 🔗
• Areas of application
  • Request structured response
  • Function calling arguments

Why is Structured Output relevant?

Typical Use Cases

• Data extraction and parsing
  • E.g. from documents (like PDF, JPEG, Word, PPT) for RAG
• Create semi-structured reports
  • E.g. summaries, incident reports, etc.
• Form filling
  • E.g. pre-fill form in UI from email
• Content classification
  • E.g. For content moderation, structured customer feedback
• Train of thought
  • Show reasoning process step-by-step
• Function calling arguments

Structured Outputs
in OpenAI

Full JSON Response

• Controlled using text parameter 🔗
  • Note: In this talk, we only focus on the new Responses API 
• Legacy format { "type": "json_object" }
  • Only ensures JSON, not a specific schema
• Recommended: { "type": "json_schema" }
  • ​Ensures specific JSON Schema 🔗
  • Create JSON schema manually or with libs like Pydantic 🔗 or zod 🔗
• Subset of JSON schema is supported 🔗. Important:
  • strict: true to ensure reliable adhering to schema, recommended
  • additionalProperties: false
  • All fields must be required, define "undefined" values in field desc

Demo

• Scenario
  • Extract structured data from PDF
• Goal
  • Define exact schema of data to extract
  • Includes different data types, nested objects, enums, arrays

Function Calling

• Basis for agentic behavior ➡️ important!
  • Read more at 🔗
• JSON Schema for function arguments
  • Output vs. input 😜
  • Schema is defined in parameters field

Demo

• Scenario
  • Bot for generating session proposals
  • Proposals are stored in a "database" (here: in memory)
• Goal
  • Define exact schema for functions' input arguments