Class: LLM::Tracer

Inherits:
Object
  • Object
show all
Defined in:
lib/llm/tracer.rb,
lib/llm/tracer/multi.rb,
lib/llm/tracer/rescue.rb,
lib/llm/tracer/registry.rb

Overview

The LLM::Tracer is the superclass of all LLM tracers. It can be helpful for implementing instrumentation and hooking into the lifecycle of an LLM request. See LLM::Tracer::Telemetry, and LLM::Tracer::Logger for example tracer implementations.

Direct Known Subclasses

Multi

Defined Under Namespace

Modules: Rescue Classes: EnhancedLogger, Logger, Multi, Null, Registry, Telemetry

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(llm, options = {}) ⇒ Tracer

Returns a new instance of Tracer.

Parameters:

  • llm (LLM::Provider) —

    A provider

  • options (Hash) (defaults to: {}) —

    A hash of options



127
128
129
130
 
# File 'lib/llm/tracer.rb', line 127

def initialize(llm, options = {})
  @llm = llm
  @options = {}
end

Instance Attribute Details

#llm ⇒ LLM::Provider (readonly)

Returns:



120
121
122
 
# File 'lib/llm/tracer.rb', line 120

def llm
  @llm
end

Class Method Details

.registry ⇒ LLM::Tracer::Registry

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the registry that counts the open scopes for each tracer.

Returns:



38
39
40
41
42
 
# File 'lib/llm/tracer.rb', line 38

def self.registry
  LLM.lock(:registry) do
    @registry ||= Registry.new
  end
end

.enter(tracer) ⇒ LLM::Tracer::Registry

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Records that a scope was opened for a tracer.

Parameters:

Returns:

See Also:



51
52
53
 
# File 'lib/llm/tracer.rb', line 51

def self.enter(tracer)
  registry.enter(tracer)
end

.exit(tracer)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Records that a scope was closed for a tracer, and tells the tracer it is finished when it was the last one that was open.

Parameters:

See Also:



62
63
64
 
# File 'lib/llm/tracer.rb', line 62

def self.exit(tracer)
  registry.exit(tracer)
end

.enhanced_logger(llm, **options) ⇒ LLM::Tracer::EnhancedLogger

Builds a EnhancedLogger for a provider.

Parameters:

Returns:



73
74
75
 
# File 'lib/llm/tracer.rb', line 73

def self.enhanced_logger(llm, **options)
  EnhancedLogger.new(llm, **options)
end

.logger(llm, **options) ⇒ LLM::Tracer::Logger

Builds a Logger for a provider.

Parameters:

Returns:



85
86
87
 
# File 'lib/llm/tracer.rb', line 85

def self.logger(llm, **options)
  Logger.new(llm, **options)
end

.telemetry(llm, **options) ⇒ LLM::Tracer::Telemetry

Builds a Telemetry for a provider.

Parameters:

Returns:



96
97
98
 
# File 'lib/llm/tracer.rb', line 96

def self.telemetry(llm, **options)
  Telemetry.new(llm, **options)
end

.multi(llm, *tracers) ⇒ LLM::Tracer::Multi

Builds a Multi

Parameters:

Returns:



106
107
108
109
110
111
112
113
114
115
116
 
# File 'lib/llm/tracer.rb', line 106

def self.multi(llm, *tracers)
  Multi.new(
    *tracers.map do |tracer|
      if Class === tracer
        tracer.new(llm)
      else
        tracer
      end
    end
  )
end

.inherited(klass)

This method returns an undefined value.

Parameters:



24
25
26
 
# File 'lib/llm/tracer.rb', line 24

def self.inherited(klass)
  klass.prepend Rescue
end

Instance Method Details

#consume_request_metadata ⇒ Hash

Consume and clear per-request metadata. Called by the telemetry tracer at span start.

Returns:

  • (Hash) 


432
433
434
435
436
437
 
# File 'lib/llm/tracer.rb', line 432

def consume_request_metadata
  key = thread_request_metadata_key
  data = thread[key] || {}
  thread[key] = nil
  data
end

#on_exit

This method returns an undefined value.

This method is called when a tracer exits a scope that has been opened by Provider#with_tracer.

It is unlikely you will ever use the Provider#with_tracer method directly but Agent uses it. An agent can have its own tracer, and it is scoped via Provider#with_tracer. When the tracer is done and Provider#with_tracer returns, this method (on_exit) is called.

A tracer that holds something - a file, a span, a connection pool - can use this method to release it. The Telemetry tracer uses this method to flush its queue of spans for export, and after the tracer has exited is a good time to do that.

Raises:

  • (NotImplementedError)

See Also:



150
151
152
 
# File 'lib/llm/tracer.rb', line 150

def on_exit
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_start(operation:, model: nil, inputs: nil, request_id:)

This method returns an undefined value.

Called before an LLM provider request is executed.

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • inputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.input.messages) from llm.rb or caller.

  • request_id (String) —

    The id of the request. It is a UUIDv7, and it is shared by #on_request_start, #on_request_finish, and #on_request_error for the same request.

Raises:

  • (NotImplementedError) 


164
165
166
 
# File 'lib/llm/tracer.rb', line 164

def on_request_start(operation:, model: nil, inputs: nil, request_id:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil, request_id:)

This method returns an undefined value.

Called after an LLM provider request succeeds.

Parameters:

  • operation (String)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)
  • model (String) (defaults to: nil)
  • outputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.output.messages) from llm.rb or caller.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata from llm.rb or caller.

  • request_id (String) —

    The id of the request that can be matched back to #on_request_start

Raises:

  • (NotImplementedError) 


181
182
183
 
# File 'lib/llm/tracer.rb', line 181

def on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil, request_id:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_error(ex:, span:, request_id:)

This method returns an undefined value.

Called when an LLM provider request fails.

Parameters:

Raises:

  • (NotImplementedError) 


192
193
194
 
# File 'lib/llm/tracer.rb', line 192

def on_request_error(ex:, span:, request_id:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_start(id:, name:, arguments:, model:)

This method returns an undefined value.

Called before a local tool/function executes.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name

Raises:

  • (NotImplementedError) 


207
208
209
 
# File 'lib/llm/tracer.rb', line 207

def on_tool_start(id:, name:, arguments:, model:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_interrupt(ex:, span:, started_at:, finished_at:)

This method returns an undefined value.

Called when a tool is interrupted.

This method is called once per tool - in a batch - and only after all tools have been spawned in parallel and then waited on sequentially. The *_at parameters let you know when a tool has started and finished (as a monotonic clock) since it cannot be inferred from the time the method is called.

Parameters:

Raises:

  • (NotImplementedError) 


230
231
232
 
# File 'lib/llm/tracer.rb', line 230

def on_tool_interrupt(ex:, span:, started_at:, finished_at:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_finish(result:, span:, started_at:, finished_at:)

This method returns an undefined value.

Called after a tool/function succeeds.

This method is called once per tool - in a batch - and only after all tools have been spawned in parallel and then waited on sequentially. The *_at parameters let you know when a tool has started and finished (as a monotonic clock) since it cannot be inferred from the time the method is called.

Parameters:

Raises:

  • (NotImplementedError) 


253
254
255
 
# File 'lib/llm/tracer.rb', line 253

def on_tool_finish(result:, span:, started_at:, finished_at:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_error(ex:, span:, started_at:, finished_at:)

Note:

This method provides the callback with an Function::Return object that encapsulates the exception a tool failed with. It does not provide an Exception object.

This method returns an undefined value.

Called when a local tool/function raises.

This method is called once per tool - in a batch - and only after all tools have been spawned in parallel and then waited on sequentially. The *_at parameters let you know when a tool has started and finished (as a monotonic clock) since it cannot be inferred from the time the method is called.

Parameters:

Raises:

  • (NotImplementedError) 


283
284
285
 
# File 'lib/llm/tracer.rb', line 283

def on_tool_error(ex:, span:, started_at:, finished_at:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_interrupt(scope:, span: nil, request_id: nil)

Note:

This is the last and final callback that is received as part of the interrupt process.

This method returns an undefined value.

This method is called when an agent receives an interrupt. An interrupt can occur in different scopes: :request, :agent, or :tool.

The :request scope provides a span and request ID, and it can be used to close a request that was interrupted. The other scopes (:tool, :agent) occur in a context where there is no span or request ID available.

Parameters:

  • scope (Symbol) —

    :request, :agent, or :tool

  • span (Object, nil) (defaults to: nil) —

    The span that #on_request_start returned

  • request_id (String, nil) (defaults to: nil) —

    The id, as passed to #on_request_start

Raises:

  • (NotImplementedError) 


309
310
311
 
# File 'lib/llm/tracer.rb', line 309

def on_interrupt(scope:, span: nil, request_id: nil)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ self

Opens a trace group so subsequent LLM spans share the same OpenTelemetry trace_id (and appear as one trace in backends like Langfuse). When +trace_group_id+ is a string, it is used to derive the trace_id.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional. Trace-level metadata merged into the trace by tracers that support it.

Returns:

  • (self) 


328
329
330
 
# File 'lib/llm/tracer.rb', line 328

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  self
end

#stop_trace ⇒ self

Finishes the trace group started by #start_trace. Safe to call even if no trace is active.

Returns:

  • (self) 


336
337
338
 
# File 'lib/llm/tracer.rb', line 336

def stop_trace
  self
end

#inspect ⇒ String

Returns:

  • (String) 


342
343
344
 
# File 'lib/llm/tracer.rb', line 342

def inspect
  "#<#{LLM::Utils.object_id(self)} @llm=#{@llm.class} @tracer=#{@tracer.inspect}>"
end

#spans ⇒ Array

Returns:

  • (Array) 


348
349
350
 
# File 'lib/llm/tracer.rb', line 348

def spans
  []
end

#flush! ⇒ nil

Note:

This method is only implemented by the Telemetry tracer. It is a noop for other tracers.

Flush the tracer

Returns:

  • (nil) 


358
359
360
 
# File 'lib/llm/tracer.rb', line 358

def flush!
  nil
end

#merge_extra(metadata: nil, inputs: nil, outputs: nil) ⇒ self

Merges extra attributes for the current trace/span. Used by applications (e.g. chatbot) to add metadata, span inputs, or span outputs to the next span or to the trace. No-op by default.

Parameters:

  • metadata (Hash, nil) (defaults to: nil) —

    Key-value pairs merged into trace/span metadata.

  • inputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the next span at start (e.g. gen_ai.input.messages). Consumed when the span is created.

  • outputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the current span at finish (e.g. gen_ai.output.messages). Must be set before the request finishes (e.g. in a block passed to the provider).

Returns:

  • (self) 


376
377
378
 
# File 'lib/llm/tracer.rb', line 376

def merge_extra(metadata: nil, inputs: nil, outputs: nil)
  self
end

#set_finish_metadata_proc(proc) ⇒ self

Optional: set a proc to supply metadata when the next chat span finishes. The proc is called with the response (res) and should return a Hash of metadata (e.g. { intent: "...", confidence: 1.0 }) to merge onto the span. Cleared after use. Used by apps to attach routing/intent that is only known after the response.

Parameters:

  • proc (Proc, nil) —

    (res) -> Hash or nil

Returns:

  • (self) 


389
390
391
392
 
# File 'lib/llm/tracer.rb', line 389

def set_finish_metadata_proc(proc)
  thread[FINISH_METADATA_PROC_KEY] = proc
  self
end

#consume_extra_inputs ⇒ Hash

Returns and clears extra inputs for the next span. Called by the telemetry tracer when starting a span. Subclasses can override to return stored inputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at start



400
401
402
 
# File 'lib/llm/tracer.rb', line 400

def consume_extra_inputs
  {}
end

#consume_extra_outputs ⇒ Hash

Returns and clears extra outputs for the current span. Called by the telemetry tracer when finishing a span. Subclasses override to return fiber-local outputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at finish



410
411
412
 
# File 'lib/llm/tracer.rb', line 410

def consume_extra_outputs
  {}
end

#set_request_metadata(metadata) ⇒ nil

Store per-request metadata (e.g. user_input) to be consumed by tracers when starting the next span. Used for plain-text input.value / output.value.

Parameters:

  • metadata (Hash) —

    e.g. { user_input: "the user question" }

Returns:

  • (nil) 


420
421
422
423
424
425
426
 
# File 'lib/llm/tracer.rb', line 420

def set_request_metadata(metadata)
  return nil unless metadata && !metadata.empty?
  key = thread_request_metadata_key
  current = thread[key] || {}
  thread[key] = current.merge(metadata.compact)
  nil
end