AI chat

Stream multi-turn chat with persistent history and built-in tool use over your records. The scaffold registers four HTTP endpoints, persists every chat to a Durable Object, and streams responses via Vercel AI SDK v5. Use the bundled ChatPanel component for a turnkey UI, or call the streaming endpoint directly and decode it with the wire helpers.

Install the chat feature#

Install the bundled feature:

npx deepspace add ai-chat

This installs five files:

• src/components/ChatPanel.tsx - the chat surface composing the message list, model picker, composer, and abort/retry controls.
• src/components/ChatPanel.messages.tsx - memoized message, Markdown, tool-status, empty, and thinking renderers.
• src/components/ChatPanel.stream.ts - the stream transport and pending-overlay reducer; owns auto-create, abort, retry, SSE errors, and the tool-call lifecycle.
• src/pages/(app)/(protected)/assistant.tsx - a protected full-page assistant with a chat history rail.
• src/schemas/ai-chat-schema.ts - exports aiChatSchemas (an array of [AI_CHATS_SCHEMA, AI_MESSAGES_SCHEMA]) for spreading into src/schemas.ts.

The feature also adds react-markdown, remark-gfm, remark-breaks, rehype-highlight, and highlight.js to package.json. Pass --install to have the installer run your package manager, or run npm install yourself before building or deploying - the build fails on the missing packages otherwise.

Embedding ChatPanel#

ChatPanel is designed to be embedded - a sidebar, a modal, a dock - not only used on the assistant page. The parent owns the chat lifecycle:

• Pass chatId={null} to auto-create a chat on first send, and onChatCreated to capture the new id. onChatCreated is required in practice whenever chatId starts null - without it the parent never learns which chat the panel created.
• Pass disabled while a parent-owned create is in flight so the panel doesn't kick off its own duplicate auto-create.

<ChatPanel
  chatId={activeChatId}
  onChatCreated={(id) => setActiveChatId(id)}
  disabled={isCreatingChat}
/>

Persisted messages arrive live through useQuery('ai-messages'); the stream hook overlays the pending user and assistant turns immediately, then removes each overlay entry as the matching persisted recordId arrives. The X-Asst-Id response header ties the pending assistant to its eventual row without relying on client clocks. The overlay rules:

• Stop or a transport failure removes an empty pending assistant but preserves partial text and tool rows the user has already seen.
• Switching away from a real chat aborts the in-flight turn and clears its overlay. The null → id transition during auto-create is not a switch and does not abort.
• Top-level failures render an alert with a Retry button that re-sends the last content. Tool input/output failures stay inline on their tool row and don't tear down the turn.

Add the schemas#

If you're wiring chat by hand, import the two pre-built schemas directly:

// src/schemas.ts
import { AI_CHATS_SCHEMA, AI_MESSAGES_SCHEMA } from 'deepspace/worker'

export const schemas = [
  usersSchema,
  settingsSchema,
  AI_CHATS_SCHEMA,
  AI_MESSAGES_SCHEMA,
  // ...your collections
]

Or, if you ran npx deepspace add ai-chat, spread the array the feature installed:

// src/schemas.ts
import { aiChatSchemas } from './schemas/ai-chat-schema'

export const schemas = [usersSchema, settingsSchema, ...aiChatSchemas]

The four chat endpoints#

The scaffold defines four endpoints in src/ai/chat-routes.ts and registers them in worker.ts with registerAiChatRoutes(app, resolveAuth). The first three manage chat records; the fourth streams a turn.

POST /api/ai/chats
Authorization: Bearer <jwt>
Content-Type: application/json

{ "title": "Untitled" }

{ "chat": { "recordId": "chat_abc", "userId": "...", "title": "Untitled", ... } }

PATCH /api/ai/chats/:id
Authorization: Bearer <jwt>
Content-Type: application/json

{ "title": "Renamed" }

DELETE /api/ai/chats/:id
Authorization: Bearer <jwt>

POST /api/ai/chat
Authorization: Bearer <jwt>
Content-Type: application/json

{ "chatId": "chat_abc", "userMessageId": "umsg_xyz", "content": "Hello", "modelId": "claude-sonnet-5" }

Streaming pipeline#

The POST /api/ai/chat handler runs through these steps:

Switch the model#

The catalog lives in the SDK, not in your app. src/ai/chat-routes.ts resolves whatever the client asked for against it:

import { resolveDeepSpaceAgentModel } from 'deepspace/worker'

const selectedModel = resolveDeepSpaceAgentModel(modelId, 'application')
if (!selectedModel) return c.json({ error: 'unknown_model' }, 400)

Unknown or non-agent modelId values resolve to null and are rejected with 400 - there is no silent fallback. Omitting modelId selects the profile's default.

To render a picker, ask the SDK what the profile supports rather than hardcoding options - this is what the scaffold's ChatPanel does:

import { listDeepSpaceAgentModels } from 'deepspace/worker'

const models = listDeepSpaceAgentModels('application') // default model sorts first

Agent-capable models#

Every model below supports multi-step tool use, which is what the application profile requires. claude-sonnet-5 is the default.

gpt-oss-120b (Cerebras) is available to createDeepSpaceAI but is single-step - it is deliberately absent from the agent profiles, so resolveDeepSpaceAgentModel rejects it for chat.

Provider routing happens via createDeepSpaceAI:

import { createDeepSpaceAI } from 'deepspace/worker'

const provider = createDeepSpaceAI(env, 'anthropic', { authToken: jwt })

The scaffold's chat routes pass the caller's JWT, so each user pays for their own conversation. Omit authToken for autonomous server-side calls (cron, server actions).

Tool use#

The assistant can read and modify your records via a built-in tool catalog. The scaffold ships all of them in src/ai/tools.ts:

The system prompt#

buildSystemPrompt(appName, schemas) in src/ai/tools.ts produces a concise prompt that lists every collection with its columns, marking required columns with !. It ships with mutation guardrails - keep them (or equivalents) when you customize:

• Confirm intent before destructive actions (delete, bulk update).
• If a write is denied by RBAC, tell the user plainly - do not retry blindly.

Customize by editing buildSystemPrompt directly; it's app code, not SDK code.

Adding custom tools#

Extend the ToolSet returned by buildTools in src/ai/tools.ts:

// src/ai/tools.ts
import { tool, type ToolSet } from 'ai'
import { z } from 'zod'
import { BUILT_IN_TOOLS } from 'deepspace/worker'

export function buildTools(executor: ToolExecutor): ToolSet {
  const tools: ToolSet = {}

  // ...existing loop over BUILT_IN_TOOLS...

  tools.lookup_weather = tool({
    description: 'Get weather for a city',
    inputSchema: z.object({ city: z.string() }),
    execute: async ({ city }) => {
      const res = await fetch(`https://api.example.com/weather?city=${encodeURIComponent(city)}`)
      if (!res.ok) return { error: `weather lookup failed: ${res.status}` }
      return await res.json()
    },
  })

  return tools
}

The Zod inputSchema doubles as runtime validation; failing input emits a tool-input-error SSE chunk the client surfaces.

Context compaction#

For long conversations, the scaffold automatically compacts older turns to stay under the model's context budget. The default config exported from deepspace/worker:

import { DEFAULT_CONTEXT_CONFIG } from 'deepspace/worker'

// {
//   contextBudget: 240_000,    // chars - ≈60–80K tokens
//   toolResultCap: 30_000,     // bytes per tool result
//   keepRecentToolResults: 5,
//   minKept: 10,               // sliding-window floor
// }

Tune for shorter-context models by passing your own config to prepareMessagesWithCompaction in chat-routes.ts:

import { DEFAULT_CONTEXT_CONFIG, prepareMessagesWithCompaction } from 'deepspace/worker'

const config = {
  ...DEFAULT_CONTEXT_CONFIG,
  contextBudget: 120_000,    // for 128K-context models
  // contextBudget: 40_000,  // for 32K-class models (some Cerebras open-weights)
}

const { messages: prepared, newSummary } = await prepareMessagesWithCompaction(
  turns,
  config,
  { summarizer, cachedSummary },
)

The pipeline:

1. Truncate old tool-result payloads (preserves the most recent N intact).
2. Apply a cached summary if one exists.
3. If still over budget, summarize the older half of the conversation.
4. As a final fallback, apply a sliding window down to minKept messages.

Custom chat UI#

If you want to build your own chat surface (sidebar, modal, minimal), call POST /api/ai/chat directly and decode the SSE stream with the SDK's wire helpers:

import { parseSseLine, decodeAiStreamChunk, getAuthToken, type AiStreamAction } from 'deepspace'

async function streamTurn(chatId: string, content: string, handleAction: (asstId: string, action: AiStreamAction) => void) {
  const token = await getAuthToken()
  const res = await fetch('/api/ai/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token}` },
    body: JSON.stringify({ chatId, userMessageId: crypto.randomUUID(), content }),
  })

  const asstId = res.headers.get('X-Asst-Id')!
  const reader = res.body!.getReader()
  const decoder = new TextDecoder()
  let buffer = ''

  while (true) {
    const { value, done } = await reader.read()
    if (done) break
    buffer += decoder.decode(value, { stream: true })
    const lines = buffer.split('\n')
    buffer = lines.pop() ?? ''
    for (const line of lines) {
      const chunk = parseSseLine(line)
      if (!chunk) continue
      const action = decodeAiStreamChunk(chunk)
      if (action) handleAction(asstId, action)
    }
  }
}

Action vocabulary#

decodeAiStreamChunk returns one of:

For the canonical message list, query ai-messages from inside a RecordScope:

import { useQuery } from 'deepspace'

type AiMessageData = {
  chatId: string
  userId: string
  role: 'user' | 'assistant'
  content: string
  parts?: unknown[]
}

const { records } = useQuery<AiMessageData>('ai-messages', {
  where: { chatId, userId },
  orderBy: 'createdAt',
  orderDir: 'asc',
})

// Each record is { recordId, data, createdAt, updatedAt } - fields live on `.data`.
records.map((r) => ({ id: r.recordId, role: r.data.role, content: r.data.content }))

The parts field on each data holds UI-shape tool invocations for rendering.

Testing the chat endpoints#

Test the streaming endpoint from api.spec.ts:

test('POST /api/ai/chat streams and persists', async ({ request }) => {
  // Pre-condition: signed-in user token, an existing chat row.
  const chatRes = await request.post('/api/ai/chats', {
    headers: { Authorization: `Bearer ${token}` },
    data: { title: 'test' },
  })
  const { chat } = await chatRes.json()

  const res = await request.post('/api/ai/chat', {
    headers: { Authorization: `Bearer ${token}` },
    data: { chatId: chat.recordId, userMessageId: `umsg-${Date.now()}`, content: 'Hi' },
  })
  expect(res.status()).toBe(200)
  expect(res.headers()['x-asst-id']).toMatch(/^asst-/)
  // Drain the stream, then assert (via the UI or a follow-up query) that two
  // new ai-messages rows exist - one user, one assistant.
})

The rules that keep this suite honest:

• One test per turn-shape - text-only, tool-using, multi-step, abort. Not one test per chunk type.
• Assert behavior, not parser fidelity. The SDK already unit-tests decodeAiStreamChunk against the chunk vocabulary; app-level tests should assert that rows persist and UI state lands, not re-verify the wire format.
• Cover the auth gates: 401 for unauthenticated callers, 404 for a chatId owned by another user. See the testing guide for the standard negative-path pattern.

Limitations#

Next steps#

• Worker AI reference - createDeepSpaceAI, compaction helpers, chat-history wrappers.
• Server actions - privileged routes that bypass user RBAC.
• External APIs - call LLMs and other services through integration.post.