Use embedding distances to score semantic difference between a prediction and reference.

Hierarchy

  • StringEvaluator
    • EmbeddingDistanceEvalChain

Implements

Constructors

Properties

distanceMetric: EmbeddingDistanceType = "cosine"

The distance metric to use for comparing the embeddings.

outputKey: string = "score"
requiresInput: boolean = false
requiresReference: boolean = true
embedding?: any

The embedding objects to vectorize the outputs.

evaluationName?: string = ...

The name of the evaluation.

memory?: any
skipInputWarning?: string = ...
skipReferenceWarning?: string = ...

Accessors

Methods

  • Parameters

    • inputs: ChainValues[]
    • Optionalconfig: any[]

    Returns Promise<ChainValues[]>

    Use .batch() instead. Will be removed in 0.2.0.

    Call the chain on all inputs in the list

  • Parameters

    • values: any
    • Optionalconfig: any
    • Optionaltags: string[]

    Returns Promise<ChainValues>

    Use .invoke() instead. Will be removed in 0.2.0.

    Run the core logic of this chain and add to output if desired.

    Wraps _call and handles memory.

  • Check if the evaluation arguments are valid.

    Parameters

    • Optionalreference: string

      The reference label.

    • Optionalinput: string

      The input string.

    Returns void

    If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

  • Evaluate Chain or LLM output, based on optional input and label.

    Parameters

    • args: StringEvaluatorArgs
    • Optionalconfig: any

    Returns Promise<ChainValues>

    The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:

    • score: the score of the evaluation, if applicable.
    • value: the string value of the evaluation, if applicable.
    • reasoning: the reasoning for the evaluation, if applicable.
  • Invoke the chain with the provided input and returns the output.

    Parameters

    • input: ChainValues

      Input values for the chain run.

    • Optionaloptions: any

    Returns Promise<ChainValues>

    Promise that resolves with the output of the chain run.

  • Parameters

    • inputs: Record<string, unknown>
    • outputs: Record<string, unknown>
    • returnOnlyOutputs: boolean = false

    Returns Promise<Record<string, unknown>>

  • Parameters

    • input: any
    • Optionalconfig: any

    Returns Promise<string>

    Use .invoke() instead. Will be removed in 0.2.0.