Class: LLM::Bedrock

Inherits:
Provider show all
Includes:
RequestAdapter
Defined in:
lib/llm/providers/bedrock.rb,
lib/llm/providers/bedrock/models.rb,
lib/llm/providers/bedrock/signature.rb,
lib/llm/providers/bedrock/error_handler.rb,
lib/llm/providers/bedrock/stream_parser.rb,
lib/llm/providers/bedrock/stream_decoder.rb,
lib/llm/providers/bedrock/request_adapter.rb,
lib/llm/providers/bedrock/response_adapter.rb

Overview

The Bedrock class implements a provider for Amazon Bedrock.

Bedrock provides access to foundation models from Anthropic, Meta, Mistral, AI21 Labs, Cohere, and more through the AWS infrastructure. This provider uses the Bedrock Converse API for chat completions, and the Converse Stream API for streaming.

Unlike other llm.rb providers which use API key authentication, Bedrock uses AWS Signature V4 (SigV4) for request signing. You must provide AWS credentials (access key, secret key, and region) instead of a single API key.

Streaming uses the AWS Event Stream binary protocol instead of standard SSE. The binary framing is decoded inline using only Ruby's stdlib.

Examples:

require "llm"

llm = LLM.bedrock(
  access_key_id: ENV["AWS_ACCESS_KEY_ID"],
  secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
  region: "us-east-1"
)
ctx = LLM::Context.new(llm)
ctx.talk "Hello, how are you?"
ctx.messages.select(&:assistant?).each { puts _1.content }

Defined Under Namespace

Modules: RequestAdapter, ResponseAdapter Classes: ErrorHandler, Models, Signature, StreamDecoder, StreamParser

Constant Summary collapse

HOST_PATTERN = 
"bedrock-runtime.%s.amazonaws.com"

Instance Method Summary collapse

Methods included from RequestAdapter

#adapt

Methods inherited from Provider

#adapt_function, #build_messages, #chat, #developer_role, #host, #inspect, #interrupt!, #ocr, #port, #registry, #request_owner, #respond, #retry_budget, #retry_errors, #schema, #server_tool, #server_tools, #system_role, #tracer, #tracer=, #user_role, #web_search, #with, #with_tracer

Constructor Details

#initialize(access_key_id: nil, secret_access_key: nil, region: nil, session_token: nil, host: nil, port: 443, ssl: true, timeout: 60) ⇒ Bedrock

Returns a new instance of Bedrock.

Parameters:

  • access_key_id (String) (defaults to: nil) —

    AWS access key ID

  • secret_access_key (String) (defaults to: nil) —

    AWS secret access key

  • region (String) (defaults to: nil) —

    AWS region (e.g. "us-east-1")

  • session_token (String, nil) (defaults to: nil) —

    AWS session token for temporary credentials

  • host (String, nil) (defaults to: nil) —

    Override the Bedrock API host

  • port (Integer) (defaults to: 443) —

    Connection port

  • ssl (Boolean) (defaults to: true) —

    Whether to use SSL

  • timeout (Integer) (defaults to: 60) —

    Request timeout in seconds



63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/llm/providers/bedrock.rb', line 63

def initialize(access_key_id: nil, secret_access_key: nil,
               region: nil, session_token: nil,
               host: nil, port: 443, ssl: true, timeout: 60,
               **)
  region ||= "us-east-1"
  @access_key_id = access_key_id
  @secret_access_key = secret_access_key
  @aws_region = region
  @session_token = session_token
  host ||= HOST_PATTERN % region
  @aws_host = host
  super(key: @access_key_id, host:, port:, ssl:, timeout:, persistent: false)
end

Instance Method Details

#key? ⇒ Boolean

Returns true when an API key is configured

Returns:

  • (Boolean) —

    Returns true when an API key is configured



181
182
183
184
185
 
# File 'lib/llm/providers/bedrock.rb', line 181

def key?
  @access_key_id.to_s.strip.size > 0      and
  @secret_access_key.to_s.strip.size > 0  and
  @aws_region.to_s.strip.size > 0
end

#name ⇒ Symbol

Returns the provider's name

Returns:

  • (Symbol) —

    Returns the provider's name



79
80
81
 
# File 'lib/llm/providers/bedrock.rb', line 79

def name
  :bedrock
end

#complete(prompt, params = {}) ⇒ LLM::Response

Provides an interface to the Bedrock Converse API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:

See Also:



91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'lib/llm/providers/bedrock.rb', line 91

def complete(prompt, params = {})
  params, stream, tools, role = normalize_complete_params(params)
  req, messages, body = build_complete_request(prompt, params, role, stream:)
  tracer.set_request_metadata(user_input: extract_user_input(messages, fallback: prompt))
  sign!(req, body)
  model_id = model_id_for(req.path)
  res, span, tracer, request_id = execute(request: req, stream:, operation: "chat", stream_parser:, model: model_id)
  res = ResponseAdapter.adapt(res, type: :completion)
    .extend(Module.new { define_method(:__tools__) { tools } })
  tracer.on_request_finish(operation: "chat", model: model_id, res:, span:, request_id:)
  res
end

#models ⇒ LLM::Bedrock::Models

Note:

Unlike the Converse API (bedrock-runtime), this endpoint lives on the control plane (bedrock..amazonaws.com).

Provides an interface to Bedrock's ListFoundationModels API.

Returns:

See Also:



113
114
115
 
# File 'lib/llm/providers/bedrock.rb', line 113

def models
  LLM::Bedrock::Models.new(self)
end

#files ⇒ Object

Raises:

  • (NotImplementedError) 


119
120
121
 
# File 'lib/llm/providers/bedrock.rb', line 119

def files
  raise NotImplementedError
end

#images ⇒ Object

Raises:

  • (NotImplementedError) 


125
126
127
 
# File 'lib/llm/providers/bedrock.rb', line 125

def images
  raise NotImplementedError
end

#audio ⇒ Object

Raises:

  • (NotImplementedError) 


131
132
133
 
# File 'lib/llm/providers/bedrock.rb', line 131

def audio
  raise NotImplementedError
end

#moderations ⇒ Object

Raises:

  • (NotImplementedError) 


137
138
139
 
# File 'lib/llm/providers/bedrock.rb', line 137

def moderations
  raise NotImplementedError
end

#responses ⇒ Object

Raises:

  • (NotImplementedError) 


143
144
145
 
# File 'lib/llm/providers/bedrock.rb', line 143

def responses
  raise NotImplementedError
end

#vector_stores ⇒ Object

Raises:

  • (NotImplementedError) 


149
150
151
 
# File 'lib/llm/providers/bedrock.rb', line 149

def vector_stores
  raise NotImplementedError
end

#embed(input, model: nil, **params) ⇒ Object

Raises:

  • (NotImplementedError) 


155
156
157
 
# File 'lib/llm/providers/bedrock.rb', line 155

def embed(input, model: nil, **params)
  raise NotImplementedError
end

#assistant_role ⇒ String

Returns:

  • (String) 


161
162
163
 
# File 'lib/llm/providers/bedrock.rb', line 161

def assistant_role
  "assistant"
end

#tool_role ⇒ Symbol

Bedrock expects tool results as user messages containing toolResult content blocks rather than a distinct tool role.

Returns:

  • (Symbol) 


169
170
171
 
# File 'lib/llm/providers/bedrock.rb', line 169

def tool_role
  :user
end

#default_model ⇒ String

Returns:

  • (String) 


175
176
177
 
# File 'lib/llm/providers/bedrock.rb', line 175

def default_model
  "deepseek.v3.2"
end

#digest ⇒ Object

Optional stdlib dependencies



36
 
# File 'lib/llm/providers/bedrock.rb', line 36

LLM.require "digest", "~> 3.3"