Ruby's most capable AI runtime

Runtime

llm.rb is Ruby's most capable AI runtime.

It runs on Ruby's standard library by default and loads optional pieces only when they are needed. Applications can start with a provider and a single context, then add agents, tools, streaming, persistence, and protocol clients without changing the core shape of the code.

It supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google Gemini, DeepSeek, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp. It also includes ActiveRecord and Sequel support, plus concurrent tool execution through threads, tasks, fibers, ractors, and fork.

Install

Source code and releases are available from github.com/llmrb/llm.rb.

$ gem install llm.rb

Capabilities

Quick start

require "llm"

llm = LLM.openai(key: ENV["KEY"])
agent = LLM::Agent.new(llm, stream: $stdout)
agent.talk "Hello world"

require "llm"

llm = LLM.openai(key: ENV["KEY"])
ctx = LLM::Context.new(llm, stream: $stdout)
ctx.talk "Hello world"

class ReadFile < LLM::Tool
  name "read-file"
  description "Read a file"
  parameter :path, String, "The filename or path"
  required %i[path]

  def call(path:)
    {contents: File.read(path)}
  end
end

Protocols and skills

require "llm"

llm = LLM.openai(key: ENV["KEY"])
mcp = LLM::MCP.stdio(argv: ["ruby", "server.rb"])

mcp.session do
  agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
  agent.talk "Use the available tools to inspect the environment."
end

Streaming and concurrency

Streams can be simple IO objects or subclasses of LLM::Stream with structured callbacks for content, reasoning, tool calls, tool returns, and compaction. Agents can resolve tool work sequentially or concurrently with threads, tasks, fibers, ractors, or forks.

class Agent < LLM::Agent
  model "gpt-5.4-mini"
  tools ReadFile
  concurrency :thread
end

llm = LLM.openai(key: ENV["KEY"])
agent = Agent.new(llm, stream: $stdout)
agent.talk "Read README.md and CHANGELOG.md and compare them."

ORM Integration

llm.rb treats ActiveRecord and Sequel as first-class integration points. The built-in wrappers persist the serialized runtime state in one column while your application keeps control of provider, model, and tool configuration.

With ActiveRecord, acts_as_agent wraps LLM::Agent directly on a model, including agent DSL features such as instructions, tools, and concurrency.

require "llm"
require "active_record"
require "llm/active_record"

class Ticket < ApplicationRecord
  acts_as_agent provider: :set_provider, context: :set_context
  model "gpt-5.4-mini"
  instructions "You are a concise support assistant."
  tools SearchDocs, Escalate
  concurrency :thread

  private

  def set_provider
    LLM.openai(key: ENV["OPENAI_SECRET"])
  end

  def set_context
    {mode: :responses, store: false}
  end
end

ticket = Ticket.create!
puts ticket.talk("How do I rotate my API key?").content

Persistence

Agents and contexts can serialize to JSON and restore later. The same serialized state powers the ActiveRecord and Sequel integrations, so applications can persist conversations in ordinary database columns without designing a separate message store first.

require "llm"

llm = LLM.openai(key: ENV["KEY"])
agent1 = LLM::Agent.new(llm)
agent1.talk "Remember that my favorite language is Ruby"
string = agent1.to_json

agent2 = LLM::Agent.new(llm, stream: $stdout)
agent2.restore(string:)
agent2.talk "What is my favorite language?"

Applications

The llm.rb runtime and its ports power small terminal applications that can be tried over SSH.