Class GoogleVertexAI

Enables calls to the Google Cloud's Vertex AI API to access Large Language Models.

This entrypoint and class are intended to be used in web environments like Edge functions where you do not have access to the file system. It supports passing service account credentials directly as a "GOOGLE_VERTEX_AI_WEB_CREDENTIALS" environment variable or directly as "authOptions.credentials".

Example

const model = new GoogleVertexAI({
  temperature: 0.7,
});
const stream = await model.stream(
  "What would be a good company name for a company that makes colorful socks?",
);
for await (const chunk of stream) {
  console.log(chunk);
}

Hierarchy

  • BaseGoogleVertexAI<WebGoogleAuthOptions>
    • GoogleVertexAI

Constructors

constructor

Properties

maxOutputTokens model temperature topK topP connection streamedConnection

Methods

extractGenerationFromPrediction extractPredictionFromResponse formatInstance formatInstanceCode formatInstanceText formatParameters

Constructors

constructor

Properties

maxOutputTokens

maxOutputTokens: number = 1024

Maximum number of tokens to generate in the completion.

model

model: string = "text-bison"

Model to use

temperature

temperature: number = 0.7

Sampling temperature to use

topK

topK: number = 40

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

topP

topP: number = 0.8

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Protectedconnection

connection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAILLMInstance, TextPrediction, WebGoogleAuthOptions>

ProtectedstreamedConnection

streamedConnection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAILLMInstance, TextPrediction, WebGoogleAuthOptions>

Methods

extractGenerationFromPrediction

extractPredictionFromResponse

  • extractPredictionFromResponse(result): TextPrediction

  • Extracts the prediction from the API response.

    Parameters

    • result: GoogleVertexAILLMResponse<TextPrediction>

      The API response from which to extract the prediction.

    Returns TextPrediction

    A TextPrediction object representing the extracted prediction.

formatInstance

  • formatInstance(prompt): GoogleVertexAILLMInstance

  • Formats the input instance for the Google Vertex AI model based on the model type (text or code).

    Parameters

    • prompt: string

      Prompt to be formatted as an instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted instance.

formatInstanceCode

  • formatInstanceCode(prompt): GoogleVertexAILLMInstance

  • Formats the input instance as a code instance for the Google Vertex AI model.

    Parameters

    • prompt: string

      Prompt to be formatted as a code instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted code instance.

formatInstanceText

  • formatInstanceText(prompt): GoogleVertexAILLMInstance

  • Formats the input instance as a text instance for the Google Vertex AI model.

    Parameters

    • prompt: string

      Prompt to be formatted as a text instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted text instance.

formatParameters

Settings

Member Visibility

On This Page

Constructors
Properties
Methods