Class: LLM::Tracer::Langsmith

Inherits:
Tracer::Telemetry
  • Object
show all
Defined in:
lib/llm/tracer/langsmith.rb

Overview

LangSmith-specific tracer built on top of Telemetry. Supports extra inputs/outputs and metadata on traces and spans via #merge_extra and #start_trace(metadata:).

Examples:

Constructor metadata and tags

llm.tracer = LLM::Tracer::Langsmith.new(
  llm,
  session_id: "123e4567-e89b-12d3-a456-426614174000",
  metadata: {env: "dev"},
  tags: ["changelog"]
)

Per-request extra metadata and inputs (e.g. from chatbot)

tracer.merge_extra(
  metadata: { turn_id: turn.id, component: "chatbot_message_stream" },
  inputs: { "gen_ai.input.messages" => messages_json }
)
bot.chat(prompt)

Trace-level metadata via start_trace

tracer.start_trace(trace_group_id: turn.id, name: "chatbot.turn", metadata: { turn_id: turn.id })

Constant Summary collapse

THREAD_EXTRA_KEY = 
:llm_langsmith_extra
UUID = 
/\A
  [0-9a-f]{8}-
  [0-9a-f]{4}-
  [1-5][0-9a-f]{3}-
  [89ab][0-9a-f]{3}-
  [0-9a-f]{12}
\z/ix

Instance Method Summary collapse

Constructor Details

#initialize(provider, options = {}) ⇒ Langsmith

Returns a new instance of Langsmith.



37
38
39
40
 
# File 'lib/llm/tracer/langsmith.rb', line 37

def initialize(provider, options = {})
  super
  setup_langsmith!(options)
end

Instance Method Details

#merge_extra(metadata: nil, inputs: nil, outputs: nil) ⇒ Object 



52
53
54
55
56
57
58
 
# File 'lib/llm/tracer/langsmith.rb', line 52

def merge_extra(metadata: nil, inputs: nil, outputs: nil)
  store = thread_extra
  store[:metadata].merge!(metadata) if metadata && !metadata.empty?
  store[:inputs].merge!(inputs) if inputs && !inputs.empty?
  store[:outputs].merge!(outputs) if outputs && !outputs.empty?
  self
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ Object 



42
43
44
45
 
# File 'lib/llm/tracer/langsmith.rb', line 42

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  merge_extra(metadata: metadata) if metadata && !metadata.empty?
  super
end

#stop_trace ⇒ Object 



47
48
49
50
 
# File 'lib/llm/tracer/langsmith.rb', line 47

def stop_trace
  clear_thread_extra!
  super
end

#current_extra ⇒ Object 



60
61
62
63
64
65
66
67
 
# File 'lib/llm/tracer/langsmith.rb', line 60

def current_extra
  store = thread_extra
  {
    metadata: store[:metadata].dup,
    inputs: store[:inputs].dup,
    outputs: store[:outputs].dup
  }
end

#consume_extra_inputs ⇒ Object 



69
70
71
 
# File 'lib/llm/tracer/langsmith.rb', line 69

def consume_extra_inputs
  thread_extra[:inputs].tap { thread_extra[:inputs] = {} }
end

#consume_extra_outputs ⇒ Object 



73
74
75
 
# File 'lib/llm/tracer/langsmith.rb', line 73

def consume_extra_outputs
  thread_extra[:outputs].tap { thread_extra[:outputs] = {} }
end