Class PairwiseStringEvalChain

A chain for comparing two outputs, such as the outputs of two models, prompts, or outputs of a single model on similar inputs.

Hierarchy (view full)

LLMPairwiseStringEvaluator
- PairwiseStringEvalChain
  - LabeledPairwiseStringEvalChain

Constructors

constructor

new PairwiseStringEvalChain(fields): PairwiseStringEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>
Returns PairwiseStringEvalChain
Inherited from LLMPairwiseStringEvaluator.constructor
- Defined in langchain/src/chains/llm_chain.ts:123

Properties

llm

llm: BaseLanguageModelInterface

LLM Wrapper to use

outputKey

outputKey: string = "text"

Key to use for output, defaults to text

outputParser

outputParser: PairwiseStringResultOutputParser = ...

prompt

prompt: BasePromptTemplate

Prompt object to use

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = false

skipReferenceWarning

skipReferenceWarning: string = ...

`Optional`criterionName

criterionName?: string

`Optional`evaluationName

evaluationName?: string = ...

The name of the evaluation.

`Optional`llmKwargs

llmKwargs?: any

Kwargs to pass to LLM

`Optional`memory

memory?: any

`Optional`skipInputWarning

skipInputWarning?: string = ...

Accessors

inputKeys

get inputKeys(): any
Returns any
Inherited from LLMPairwiseStringEvaluator.inputKeys
- Defined in langchain/src/chains/llm_chain.ts:115

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from LLMPairwiseStringEvaluator.outputKeys
- Defined in langchain/src/chains/llm_chain.ts:119

Methods

apply

apply(inputs, config?): Promise<ChainValues[]>
Parameters
- inputs: ChainValues[]
- Optionalconfig: any[]
Returns Promise<ChainValues[]>
Deprecated
Use .batch() instead. Will be removed in 0.2.0.

Call the chain on all inputs in the list
Inherited from LLMPairwiseStringEvaluator.apply
- Defined in langchain/src/chains/base.ts:258

call

call(values, config?): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: any
- Optionalconfig: any
Returns Promise<ChainValues>
Inherited from LLMPairwiseStringEvaluator.call
- Defined in langchain/src/chains/llm_chain.ts:180

checkEvaluationArgs

checkEvaluationArgs(reference?, input?): void
Check if the evaluation arguments are valid.
Parameters
- Optionalreference: string
  The reference label.
- Optionalinput: string
  The input string.
Returns void
Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from LLMPairwiseStringEvaluator.checkEvaluationArgs
- Defined in langchain/src/evaluation/base.ts:62

evaluateStringPairs

evaluateStringPairs(args, callOptions?, config?): Promise<ChainValues>
Evaluate the output string pairs.
Parameters
- args: LLMPairwiseStringEvaluatorArgs
- OptionalcallOptions: unknown
- Optionalconfig: any
Returns Promise<ChainValues>
A dictionary containing the preference, scores, and/or other information.
Inherited from LLMPairwiseStringEvaluator.evaluateStringPairs
- Defined in langchain/src/evaluation/base.ts:310

invoke

invoke(input, options?): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  Input values for the chain run.
- Optionaloptions: any
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from LLMPairwiseStringEvaluator.invoke
- Defined in langchain/src/chains/base.ts:86

predict

predict(values, callbackManager?): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: any
  keys to pass to prompt template
- OptionalcallbackManager: any
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from LLMPairwiseStringEvaluator.predict
- Defined in langchain/src/chains/llm_chain.ts:246

prepOutputs

prepOutputs(inputs, outputs, returnOnlyOutputs?): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from LLMPairwiseStringEvaluator.prepOutputs
- Defined in langchain/src/chains/base.ts:149

run

run(input, config?): Promise<string>
Parameters
- input: any
- Optionalconfig: any
Returns Promise<string>
Deprecated
Use .invoke() instead. Will be removed in 0.2.0.
Inherited from LLMPairwiseStringEvaluator.run
- Defined in langchain/src/chains/base.ts:190

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain
Deprecated
Inherited from LLMPairwiseStringEvaluator.serialize
- Defined in langchain/src/chains/llm_chain.ts:274

`Static`deserialize

deserialize(data): Promise<LLMChain<string, any>>
Load a chain from a json-like object describing it.
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, any>>
Inherited from LLMPairwiseStringEvaluator.deserialize
- Defined in langchain/src/chains/llm_chain.ts:258

`Static`fromLLM

fromLLM(llm, criteria?, chainOptions?): Promise<PairwiseStringEvalChain>
Create a new instance of the PairwiseStringEvalChain.
Parameters
- llm: BaseLanguageModelInterface
- Optionalcriteria: CriteriaLike
  The criteria to use for evaluation.
- OptionalchainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>, "llm">>
  Options to pass to the chain.
Returns Promise<PairwiseStringEvalChain>
- Defined in langchain/src/evaluation/comparison/pairwise.ts:185

`Static`resolvePairwiseCriteria

resolvePairwiseCriteria(criteria?): Record<string, string>
Parameters
- Optionalcriteria: CriteriaLike
Returns Record<string, string>
- Defined in langchain/src/evaluation/comparison/pairwise.ts:116

`Static`resolvePairwisePrompt

resolvePairwisePrompt(prompt?): any
Parameters
- Optionalprompt: any
Returns any
- Defined in langchain/src/evaluation/comparison/pairwise.ts:158

Class PairwiseStringEvalChain

Hierarchy (view full)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns PairwiseStringEvalChain

Properties

llm

outputKey

outputParser

prompt

requiresInput

requiresReference

skipReferenceWarning

OptionalcriterionName

OptionalevaluationName

OptionalllmKwargs

Optionalmemory

OptionalskipInputWarning

Accessors

inputKeys

Returns any

outputKeys

Returns string[]

Methods

apply

Parameters

Returns Promise<ChainValues[]>

Deprecated

call

Parameters

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Returns void

Throws

evaluateStringPairs

Parameters

Returns Promise<ChainValues>

invoke

Parameters

Returns Promise<ChainValues>

predict

Parameters

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

Returns Promise<Record<string, unknown>>

run

Parameters

Returns Promise<string>

Deprecated

serialize

Returns SerializedLLMChain

Deprecated

Staticdeserialize

Parameters

Returns Promise<LLMChain<string, any>>

StaticfromLLM

Parameters

Returns Promise<PairwiseStringEvalChain>

StaticresolvePairwiseCriteria

Parameters

Returns Record<string, string>

StaticresolvePairwisePrompt

Parameters

Returns any

Settings

On This Page

`Optional`criterionName

`Optional`evaluationName

`Optional`llmKwargs

`Optional`memory

`Optional`skipInputWarning

`Static`deserialize

`Static`fromLLM

`Static`resolvePairwiseCriteria

`Static`resolvePairwisePrompt