Utility class to help implement streaming agents

Constructors

Methods

  • Creates an async iterable from a callback-based stream

    Type Parameters

    • TOutput

    Parameters

    • producer: (
          emit: (chunk: StreamChunk<TOutput>) => void,
          end: () => void,
          error: (err: Error) => void,
      ) => void

      Function that sets up the stream with callbacks

    Returns AsyncIterable<StreamChunk<TOutput>>

    An async iterable of chunks

    const stream = StreamingHelper.fromCallbacks<ChatOutput>((emit, end, error) => {
    llmProvider.stream(input, {
    onChunk: (data) => emit({ data, done: false }),
    onComplete: (data) => { emit({ data, done: true }); end(); },
    onError: error
    });
    });

    for await (const chunk of stream) {
    console.log(chunk.data);
    }