Type Alias LabeledCriteria

Configuration to load a "LabeledCriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria and also provides a "ground truth" label for the evaluator to incorporate in its evaluation.

Param: criteria

The criteria to use for the evaluator.

Param: llm

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
  evaluators: [LabeledCriteria("correctness")],
};
@example
```ts
const evalConfig = {
  evaluators: [
    LabeledCriteria({
      "mentionsAllFacts": "Does the include all facts provided in the reference?"
    })
  ],

Copy

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "labeled_criteria",
    criteria: "correctness",
    formatEvaluatorInputs: ...
  }],
};
Copy

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "labeled_criteria",
    criteria: { "mentionsAllFacts": "Does the include all facts provided in the reference?" },
    formatEvaluatorInputs: ...
  }],

Copy