Module: LLM::Step

Defined in:
lib/llm/step.rb

Overview

Writes the conversation down when a request completes.

LLM::Step is prepended onto a stream's singleton class by LLM::Stream.try, so a conversation bound to a record is saved as it goes: a turn that is interrupted - by a crash, by a deploy, or by a provider that refuses - can be continued rather than started over.

It saves through the record the context is bound to (Context#record), and it knows both the ORMs llm.rb speaks: an ActiveRecord model is saved with ActiveRecord::Utils.save! and a Sequel model with LLM::Sequel::Plugin::Utils.save!. A context with no record - which is every context that is not backed by a model - has nothing to save, and neither has a record that carries neither plugin, so the step is passed along untouched.

All of this is one method on purpose. The module is put onto an object the caller owns, and a helper would be one more name on that object for as long as it lives.

The write is best effort: a failure is swallowed rather than raised, because this runs inside the request that has just completed, and a stale checkpoint is a smaller failure than a turn that ends because the database was busy. Only the write is covered, so a stream's own on_step is reached whether or not the write lands - a step that a tracer hears about does not depend on a database having been free.

See Also:

Instance Method Summary collapse

Instance Method Details

#on_step(ctx, res) ⇒ nil

Saves the conversation the context holds, then passes the step along.

Parameters:

  • ctx (LLM::Context) —

    The context the request belongs to

  • res (LLM::Response) —

    The response for the request that completed

Returns:

  • (nil) 


42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/llm/step.rb', line 42

def on_step(ctx, res)
  record = ctx.record
  if record and record.class.respond_to?(:llm_plugin_options)
    options = record.class.llm_plugin_options
    begin
      if defined?(::ActiveRecord::Base) and ::ActiveRecord::Base === record
        LLM::ActiveRecord::Utils.save!(record, ctx, options)
      elsif defined?(::Sequel::Model) and ::Sequel::Model === record
        LLM::Sequel::Plugin::Utils.save!(record, ctx, options)
      end
    rescue
      nil
    end
  end
  super
end