LLM Credentials

Pass LLM provider API keys to agent sessions so keys stay on the server and are injected at session creation, with per-tenant isolation for multi-tenant deployments.

Passing API keys

Pass LLM provider keys via the env option on createSession. The VM does not inherit from the host process.env, so keys must be passed explicitly.

import { createClient } from "@rivet-dev/agentos/client";
import type { registry } from "./server";

const client = createClient<typeof registry>({
  endpoint: "http://localhost:6420",
});

// Pass LLM provider keys via the `env` option on createSession. The VM does
// not inherit from the host process.env, so keys must be passed explicitly.
const session = await client.vm.getOrCreate("my-agent").createSession("pi", {
  env: { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY! },
});

console.log(session.sessionId);

See Full Example

Per-tenant credentials

Give each tenant an isolated VM by keying getOrCreate on the tenant id, look up that tenant’s API key on the server, and inject it via the session env. Credentials stay on the server and never reach the client.

First, declare the agent software on the server:

import { agentOS, setup } from "@rivet-dev/agentos";
import pi from "./software/pi";

// The VM does not inherit the host process.env. LLM provider keys are passed
// explicitly per session, so the server just declares the agent software here.
const vm = agentOS({
  software: [pi],
});

export const registry = setup({ use: { vm } });

registry.start();

See Full Example

Then resolve each tenant’s key and pass it at session creation:

import { createClient } from "@rivet-dev/agentos/client";
import type { registry } from "./server";

const client = createClient<typeof registry>({
  endpoint: "http://localhost:6420",
});

// Stand-in for your own per-tenant credential store.
declare function lookupTenantApiKey(tenantId: string): Promise<string>;

// Give each tenant an isolated VM keyed by their tenant id, then inject that
// tenant's API key from your database at session creation. Keys stay on the
// server and never reach the client.
async function startTenantSession(tenantId: string) {
  const anthropicApiKey = await lookupTenantApiKey(tenantId);

  return client.vm.getOrCreate(tenantId).createSession("pi", {
    env: { ANTHROPIC_API_KEY: anthropicApiKey },
  });
}

const session = await startTenantSession("tenant-123");
console.log(session.sessionId);

See Full Example

Because keys are resolved per tenant from your own credential store (the lookupTenantApiKey stand-in above) and stay on the server, each session uses the tenant’s own key and one tenant’s key never reaches another tenant or the client.

Embedded LLM Gateway

The Embedded LLM Gateway (coming soon) will remove the need to manage API keys manually. It routes all agent LLM requests through a managed proxy built into agentOS, providing per-tenant usage metering, rate limiting, and cost controls without deploying a separate gateway service.