Models

Theazo's model gateway routes to the best LLM for your agent. Use managed models (Theazo handles the API keys) or bring your own model credentials.

Managed models

By default, Theazo manages model credentials via OpenRouter. You select a model and Theazo handles routing, rate limiting, and billing. No API keys to manage.

managed-model.ts

const agent = await session.agents.create({
  compute: 'python',
  model: 'anthropic/claude-sonnet',  // Theazo handles the API key
})

// Available managed models:
// anthropic/claude-sonnet    — Best for complex reasoning + tool use
// anthropic/claude-haiku     — Fast + cheap for simple tasks
// openai/gpt-4o              — Strong general-purpose model
// openai/gpt-4o-mini         — Budget option, good for classification
// google/gemini-2.0-flash    — 1M context, fast, very cheap

Available models

anthropic/claude-sonnet200K context

Input: $3.00/1MOutput: $15.00/1Mtext, vision, tools

anthropic/claude-haiku200K context

Input: $0.25/1MOutput: $1.25/1Mtext, vision, tools

openai/gpt-4o128K context

Input: $2.50/1MOutput: $10.00/1Mtext, vision, tools

openai/gpt-4o-mini128K context

Input: $0.15/1MOutput: $0.60/1Mtext, vision, tools

google/gemini-2.0-flash1M context

Input: $0.10/1MOutput: $0.40/1Mtext, vision, tools

List models programmatically

list-models.ts

const models = await theazo.models.list()

models.forEach(m => {
  console.log(m.id, m.name, m.capabilities)
})

Get model details

get-model.ts

const model = await theazo.models.get('anthropic/claude-sonnet')

console.log(model.name)           // 'Claude Sonnet'
console.log(model.maxTokens)      // 200000
console.log(model.capabilities)   // ['text', 'vision', 'tool_use']
console.log(model.inputCostPer1M) // 300 (cents)

Cost estimation

Estimate the cost of a model call before running it. Useful for setting cost caps and budgeting.

estimate-cost.ts

const estimate = await theazo.models.estimate({
  model: 'anthropic/claude-sonnet',
  inputTokens: 50000,
  outputTokens: 2000,
})

console.log(estimate.estimatedCost.modelCost)   // { amount: 45, currency: 'usd' }
console.log(estimate.estimatedCost.theazoMargin) // { amount: 5, currency: 'usd' }
console.log(estimate.estimatedCost.total)        // { amount: 50, currency: 'usd' }

Theazo applies a 10% margin on managed model usage. This covers routing, rate limit management, and fallback handling. BYOI model usage has no margin.

Bring your own models (BYOI)

If you have your own API keys for model providers, pass them in the Theazo constructor. Theazo will route model calls through your credentials instead of managed ones.

byoi-models.ts

// Configure BYOI models at the platform level (single provider)
const theazo = new Theazo({
  apiKey: 'th_live_...',
  models: {
    provider: 'anthropic',
    credentials: { apiKey: process.env.ANTHROPIC_API_KEY },
  },
})

// Now model calls use YOUR API key
// Theazo charges only for orchestration, not model usage
const session = await theazo.sessions.forUser('user_123')
const result = await session.run('analyst', 'summarize Q2 results')

// For multi-provider, configure per-agent at creation time:
const agent = await session.agents.create({
  compute: 'python',
  model: 'openai/gpt-4o',
  modelCredentials: { apiKey: process.env.OPENAI_API_KEY },
})

With BYOI models, you pay the model provider directly. Theazo still tracks token usage for observability and billing data, but does not charge a model margin.

Gateway BYOI (OpenAI-compatible endpoints)

Use the baseUrl parameter to point Theazo at any OpenAI-compatible endpoint. This unlocks hundreds of models through a single integration.

gateway-byoi.ts

const theazo = new Theazo({
  apiKey: 'th_live_...',
  models: {
    baseUrl: 'https://openrouter.ai/api/v1',
    credentialRef: 'openrouter_key',
  },
})

// Now any model available on the gateway is accessible
const session = await theazo.sessions.forUser('user_123')
const agent = await session.agents.create({
  compute: 'python',
  model: 'meta-llama/llama-3.1-405b-instruct',
})

Compatible gateways include:

OpenRouterhttps://openrouter.ai/api/v1

200+ models, automatic fallbacks

LiteLLMhttp://localhost:4000/v1

Self-hosted proxy, unified API for 100+ providers

Azure OpenAIhttps://{resource}.openai.azure.com/openai/deployments/{model}/v1

Enterprise Azure deployments

vLLMhttp://localhost:8000/v1

Self-hosted inference server

Ollamahttp://localhost:11434/v1

Local models, no API key needed

The baseUrl parameter only works for OpenAI-compatible APIs (any endpoint serving /v1/chat/completions). Direct Anthropic BYOI still uses the anthropic config with createAnthropic() under the hood. If you want to reach Anthropic models through a gateway, use OpenRouter or LiteLLM as the baseUrl instead.

Credential references

Instead of passing raw API keys in code, store them in Theazo's secrets vault and reference them by name. Theazo decrypts credentials at runtime when making model calls.

credential-ref.ts

// Store your key in the secrets vault (dashboard or API)
await theazo.secrets.set('openrouter_key', 'sk-or-...')

// Reference it by name — no raw keys in code
const theazo = new Theazo({
  apiKey: 'th_live_...',
  models: {
    baseUrl: 'https://openrouter.ai/api/v1',
    credentialRef: 'openrouter_key',  // resolved at runtime
  },
})

// Works the same with direct provider BYOI
const theazo2 = new Theazo({
  apiKey: 'th_live_...',
  models: {
    provider: 'anthropic',
    credentials: { credentialRef: 'my_anthropic_key' },
  },
})

Credentials are encrypted with AES-256-GCM and scoped to your platform. They never appear in logs, API responses, or agent output. Use credentialRef over raw apiKey for production deployments.

Per-agent model selection

Model selection supports a 2-level override hierarchy. Set a default at the platform level and override per-agent.

PlatformAll agentsSet in Theazo constructor

AgentThis agent onlySet in agents.create()

model-override.ts

// Level 1: Platform default — all agents use Claude Sonnet
const theazo = new Theazo({
  apiKey: 'th_live_...',
  defaultModel: 'anthropic/claude-sonnet',
})

// Level 2: Agent override — this specific agent uses Haiku (cheap + fast)
const session = await theazo.sessions.forUser('user_123')
const triageAgent = await session.agents.create({
  compute: 'python',
  model: 'anthropic/claude-haiku',  // overrides platform default
})

// Another agent in the same session inherits the platform default (Claude Sonnet)
const analystAgent = await session.agents.create({
  compute: 'python',
  // no model specified → inherits 'anthropic/claude-sonnet' from platform
})

The override chain is: agent > platform > Theazo default (anthropic/claude-sonnet). If no model is specified at any level, Theazo falls back to Claude Sonnet as the default.

API reference

theazo.models.list()Promise<Model[]>List all available models with pricing and capabilities.

theazo.models.get(id)Promise<Model>Get a specific model by ID (e.g., 'anthropic/claude-sonnet').

theazo.models.estimate(opts)Promise<CostEstimate>Estimate cost for a model call given input/output token counts.

REST endpoints

GET/v1/modelsList all available models

GET/v1/models/:idGet model details by ID

POST/v1/models/estimateEstimate cost for token counts