Skip to main content

​
Overview

OkraPDF’s chat completions endpoint is OpenAI-compatible. Pass tools in the request and the LLM can call them — your tools execute client-side, builtin tools (SQL queries, metadata lookups) execute server-side automatically.
Drop-in compatible. Works with the OpenAI SDK, Vercel AI SDK, LangChain, or raw HTTP. No adapter code needed.

​
How it works

Builtin tools like query_sql, get_job_metadata, get_live_status, and query_document run server-side in the document’s Durable Object — you never see them. Your tools are returned to the client for execution.

​
Setup

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: `https://api.okrapdf.com/document/${docId}`,
  apiKey: 'okra_YOUR_KEY',
});

​
Custom system prompt

System messages are appended to OkraPDF’s built-in document context prompt. Use them to steer tone, format, language, or domain behavior. 
const res = await client.chat.completions.create({
  model: 'kimi-k2p5',
  messages: [
    {
      role: 'system',
      content: `You are a financial analyst. Always respond in bullet points.
Format monetary values with $ and 2 decimal places. Cite page numbers.`,
    },
    { role: 'user', content: 'What are the key revenue figures?' },
  ],
});
The built-in prompt already includes document metadata, page content, and tool instructions. Your system message adds to it — you don’t need to repeat context about the document.

​
Basic tool calling

Define tools in standard OpenAI format. The LLM decides when to call them. 
const res = await client.chat.completions.create({
  model: 'kimi-k2p5',
  messages: [
    { role: 'user', content: 'Summarize and email to bob@example.com' },
  ],
  tools: [
    {
      type: 'function',
      function: {
        name: 'send_email',
        description: 'Send an email to a recipient',
        parameters: {
          type: 'object',
          properties: {
            to: { type: 'string', description: 'Recipient email' },
            subject: { type: 'string' },
            body: { type: 'string', description: 'Email body (markdown)' },
          },
          required: ['to', 'body'],
        },
      },
    },
  ],
});

if (res.choices[0].finish_reason === 'tool_calls') {
  const calls = res.choices[0].message.tool_calls!;
  console.log(calls[0].function.name);      // "send_email"
  console.log(calls[0].function.arguments); // {"to":"bob@example.com","body":"..."}
}
Response: 
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "functions.send_email:0",
        "type": "function",
        "function": {
          "name": "send_email",
          "arguments": "{\"to\":\"bob@example.com\",\"subject\":\"Summary\",\"body\":\"...\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

​
Multi-turn tool execution

After receiving tool_calls, execute the tool client-side, then send the result back to continue the conversation. 
// Turn 1: LLM requests tool call
const turn1 = await client.chat.completions.create({
  model: 'kimi-k2p5',
  messages: [
    { role: 'user', content: 'Post a summary to #general on Slack' },
  ],
  tools,
});

const assistantMsg = turn1.choices[0].message;
const toolCall = assistantMsg.tool_calls![0];

// Execute tool client-side
const result = await postToSlack(
  JSON.parse(toolCall.function.arguments),
);

// Turn 2: Send result back
const turn2 = await client.chat.completions.create({
  model: 'kimi-k2p5',
  messages: [
    { role: 'user', content: 'Post a summary to #general on Slack' },
    assistantMsg, // includes tool_calls
    {
      role: 'tool',
      tool_call_id: toolCall.id,
      content: JSON.stringify(result),
    },
  ],
  tools,
});

console.log(turn2.choices[0].message.content);
// "Done! Posted the summary to #general."

​
Streaming with tools

Works with stream: true. Tool calls arrive as delta chunks, finish reason is "tool_calls". 
const stream = await client.chat.completions.create({
  model: 'kimi-k2p5',
  stream: true,
  messages: [
    { role: 'user', content: 'Save risk factors to a spreadsheet' },
  ],
  tools: [
    {
      type: 'function',
      function: {
        name: 'save_to_spreadsheet',
        description: 'Save data to Google Sheets',
        parameters: {
          type: 'object',
          properties: {
            title: { type: 'string' },
            rows: { type: 'array', items: { type: 'object' } },
          },
          required: ['title', 'rows'],
        },
      },
    },
  ],
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (delta?.content) process.stdout.write(delta.content);
  if (delta?.tool_calls) {
    // tool_calls arrive as deltas
    for (const tc of delta.tool_calls) {
      console.log(tc.function?.name, tc.function?.arguments);
    }
  }
  if (chunk.choices[0]?.finish_reason === 'tool_calls') {
    console.log('Tool calls received — execute and send results back');
  }
}

​
Vercel AI SDK

The /ai-stream endpoint emits AI SDK v6 events. Use useChat with onToolCall for client-side tool execution. 
'use client';
import { useChat } from '@ai-sdk/react';

export function Chat({ docId }: { docId: string }) {
  const { messages, input, handleSubmit, handleInputChange, addToolResult } = useChat({
    api: `https://api.okrapdf.com/document/${docId}/ai-stream`,
    headers: { Authorization: `Bearer ${apiKey}` },
    maxSteps: 5,
    async onToolCall({ toolCall }) {
      // Execute your tool client-side
      if (toolCall.toolName === 'send_email') {
        const result = await sendEmail(toolCall.args);
        return result;
      }
    },
  });

  return (
    <form onSubmit={handleSubmit}>
      {messages.map(m => <div key={m.id}>{m.content}</div>)}
      <input value={input} onChange={handleInputChange} />
    </form>
  );
}

​
System prompt + tools combined

Combine custom instructions with tools for domain-specific workflows. 
const res = await client.chat.completions.create({
  model: 'kimi-k2p5',
  messages: [
    {
      role: 'system',
      content: `You are a compliance assistant. When you find regulatory issues,
always use create_ticket to log them. Be thorough.`,
    },
    { role: 'user', content: 'Audit this document for SEC compliance issues' },
  ],
  tools: [
    {
      type: 'function',
      function: {
        name: 'create_ticket',
        description: 'Create a compliance ticket',
        parameters: {
          type: 'object',
          properties: {
            title: { type: 'string' },
            severity: { type: 'string', enum: ['low', 'medium', 'high', 'critical'] },
            description: { type: 'string' },
            page_references: { type: 'array', items: { type: 'number' } },
          },
          required: ['title', 'severity', 'description'],
        },
      },
    },
  ],
});

​
Edge cases

ScenarioBehavior
No tools in requestAll tools are builtin, auto-execute, finish_reason: "stop" always
Empty tools: []Same as no tools
Only builtin calls by LLMAuto-execute server-side, never returns tool_calls
Only user tool callsReturns all as tool_calls
Mixed builtin + user callsBuiltins execute silently, user calls returned
Tool name collides with builtinBuiltin wins
Malformed tool definition400 with invalid_request_error

​
Builtin tools reference

These execute automatically server-side — you don’t need to define or handle them:
ToolDescription
query_sqlRun read-only SQL against the document’s extracted data
get_job_metadataFile name, page count, processing status
get_live_statusReal-time phase, pending tasks, activity logs
query_documentAsk a question grounded in OCR page content

Structured Extraction

Extract typed data with JSON Schema or Zod.

Build a ChatPDF App

Full chat app with page rendering.