Generate text embeddings by using the ML.GENERATE_EMBEDDING function

Console

1. Go to the BigQuery page.

   Go to BigQuery

2. To create a connection, click add Add, and then click Connections to external data sources.

3. In the Connection type list, select Vertex AI remote models, remote functions and BigLake (Cloud Resource).

4. In the Connection ID field, enter a name for your connection.

5. Click Create connection.

6. Click Go to connection.

7. In the Connection info pane, copy the service account ID for use in a later step.

bq

1. In a command-line environment, create a connection:

   bq mk --connection --location=REGION --project_id=PROJECT_ID \
       --connection_type=CLOUD_RESOURCE CONNECTION_ID

   The --project_id parameter overrides the default project.

   Replace the following:

   • REGION: your connection region
   • PROJECT_ID: your Google Cloud project ID
   • CONNECTION_ID: an ID for your connection

   When you create a connection resource, BigQuery creates a unique system service account and associates it with the connection.

   Troubleshooting: If you get the following connection error, update the Google Cloud SDK:

   Flags parsing error: flag --connection_type=CLOUD_RESOURCE: value should be one of...
   

2. Retrieve and copy the service account ID for use in a later step:

   bq show --connection PROJECT_ID.REGION.CONNECTION_ID

   The output is similar to the following:

   name                          properties
   1234.REGION.CONNECTION_ID     {"serviceAccountId": "connection-1234-9u56h9@gcp-sa-bigquery-condel.iam.gserviceaccount.com"}
   

Terraform

Use the google_bigquery_connection resource.

To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following example creates a Cloud resource connection named my_cloud_resource_connection in the US region:

# This queries the provider for project information.
data "google_project" "default" {}

# This creates a cloud resource connection in the US region named my_cloud_resource_connection.
# Note: The cloud resource nested object has only one output field - serviceAccountId.
resource "google_bigquery_connection" "default" {
  connection_id = "my_cloud_resource_connection"
  project       = data.google_project.default.project_id
  location      = "US"
  cloud_resource {}
}

To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.

Prepare Cloud Shell

1. Launch Cloud Shell.

2. Set the default Google Cloud project where you want to apply your Terraform configurations.

   You only need to run this command once per project, and you can run it in any directory.

   export GOOGLE_CLOUD_PROJECT=PROJECT_ID

   Environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module).

1. In Cloud Shell, create a directory and a new file within that directory. The filename must have the .tf extension—for example main.tf. In this tutorial, the file is referred to as main.tf.

   mkdir DIRECTORY && cd DIRECTORY && touch main.tf

2. If you are following a tutorial, you can copy the sample code in each section or step.

   Copy the sample code into the newly created main.tf.

   Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.

3. Review and modify the sample parameters to apply to your environment.
4. Save your changes.
5. Initialize Terraform. You only need to do this once per directory.

   terraform init

   Optionally, to use the latest Google provider version, include the -upgrade option:

   terraform init -upgrade

Apply the changes

1. Review the configuration and verify that the resources that Terraform is going to create or update match your expectations:

   terraform plan

   Make corrections to the configuration as necessary.

2. Apply the Terraform configuration by running the following command and entering yes at the prompt:

   terraform apply

   Wait until Terraform displays the "Apply complete!" message.

3. Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.

Console

1. Go to the IAM & Admin page.

   Go to IAM & Admin

2. Click person_add Grant access.

   The Add principals dialog opens.

3. In the New principals field, enter the service account ID that you copied earlier.

4. In the Select a role field, select Vertex AI, and then select Vertex AI User.

5. Click Save.

gcloud

Use the gcloud projects add-iam-policy-binding command:

gcloud projects add-iam-policy-binding 'PROJECT_NUMBER' --member='serviceAccount:MEMBER' --role='roles/aiplatform.user' --condition=None

Replace the following:

• PROJECT_NUMBER: your project number
• MEMBER: the service account ID that you copied earlier

text embedding

Generate text embeddings by using a remote model over an embedding model:

SELECT *
FROM ML.GENERATE_EMBEDDING(
  MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,
  TABLE PROJECT_ID.DATASET_ID.TABLE_NAME,
  STRUCT(FLATTEN_JSON AS flatten_json_output,
    TASK_TYPE AS task_type,
    OUTPUT_DIMENSIONALITY AS output_dimensionality)
);

Replace the following:

• PROJECT_ID: your project ID.
• DATASET_ID: the ID of the dataset that contains the model.
• MODEL_NAME: the name of the remote model over an embedding model.
• TABLE_NAME: the name of the table that contains the text to embed. This table must have a column that's named content, or you can use an alias to use a differently named column.
• FLATTEN_JSON: a BOOL value that indicates whether to parse the embedding into a separate column. The default value is TRUE.
• TASK_TYPE: a STRING literal that specifies the intended downstream application to help the model produce better quality embeddings. TASK_TYPE accepts the following values:
  • RETRIEVAL_QUERY: specifies that the given text is a query in a search or retrieval setting.
  • RETRIEVAL_DOCUMENT: specifies that the given text is a document in a search or retrieval setting.

    When using this task type, it is helpful to include the document title in the query statement in order to improve embedding quality. You can use the title option to specify the name of the column that contains the document title, otherwise the document title must be in a column either named title or aliased as title, for example:

          SELECT *
          FROM
            ML.GENERATE_EMBEDDING(
              MODEL mydataset.embedding_model,
              (SELECT abstract as content, header as title, publication_number
              FROM mydataset.publications),
              STRUCT(TRUE AS flatten_json_output, 'RETRIEVAL_DOCUMENT' as task_type)
          );
          

  • SEMANTIC_SIMILARITY: specifies that the given text will be used for Semantic Textual Similarity (STS).
  • CLASSIFICATION: specifies that the embeddings will be used for classification.
  • CLUSTERING: specifies that the embeddings will be used for clustering.
• OUTPUT_DIMENSIONALITY: an INT64 value that specifies the number of dimensions to use when generating embeddings. For example, if you specify 256 AS output_dimensionality, then the ml_generate_embedding_result output column contains 256 embeddings for each input value.

  You can only use this argument if the remote model that you specify in the model argument uses one of the following models as an endpoint:

  • text-embedding-004 or later
  • text-multilingual-embedding-002 or later

multimodal embedding

Generate text embeddings by using a remote model over the multimodalembedding model:

SELECT *
FROM ML.GENERATE_EMBEDDING(
  MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,
  TABLE PROJECT_ID.DATASET_ID.TABLE_NAME,
  STRUCT(FLATTEN_JSON AS flatten_json_output，
  OUTPUT_DIMENSIONALITY AS output_dimensionality)
);

Replace the following:

• PROJECT_ID: your project ID.
• DATASET_ID: the ID of the dataset that contains the model.
• MODEL_NAME: the name of the remote model over a multimodalembedding@001 model.
• TABLE_NAME: the name of the table that contains the text to embed. This table must have a column that's named content, or you can use an alias to use a differently named column.
• FLATTEN_JSON: a BOOL that indicates whether to parse the embedding into a separate column. The default value is TRUE.
• OUTPUT_DIMENSIONALITY: an INT64 value that specifies the number of dimensions to use when generating embeddings. Valid values are 128, 256, 512, and 1408. The default value is 1408. For example, if you specify 256 AS output_dimensionality, then the ml_generate_embedding_result output column contains 256 embeddings for each input value.

text embedding

Generate text embeddings by using a remote model over the embedding model:

SELECT *
FROM ML.GENERATE_EMBEDDING(
  MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,
  (CONTENT_QUERY),
  STRUCT(FLATTEN_JSON AS flatten_json_output,
    TASK_TYPE AS task_type,
    OUTPUT_DIMENSIONALITY AS output_dimensionality)
  );

Replace the following:

• PROJECT_ID: your project ID.
• DATASET_ID: the ID of the dataset that contains the model.
• MODEL_NAME: the name of the remote model over an embedding model.
• CONTENT_QUERY: a query whose result contains a STRING column called content.
• FLATTEN_JSON: a BOOL value that indicates whether to parse the embedding into a separate column. The default value is TRUE.
• TASK_TYPE: a STRING literal that specifies the intended downstream application to help the model produce better quality embeddings. TASK_TYPE accepts the following values:
  • RETRIEVAL_QUERY: specifies that the given text is a query in a search or retrieval setting.
  • RETRIEVAL_DOCUMENT: specifies that the given text is a document in a search or retrieval setting.

    When using this task type, it is helpful to include the document title in the query statement in order to improve embedding quality. You can use the title option to specify the name of the column that contains the document title, otherwise the document title must be in a column either named title or aliased as title, for example:

              SELECT *
              FROM
                ML.GENERATE_EMBEDDING(
                  MODEL mydataset.embedding_model,
                  (SELECT abstract as content, header as title, publication_number
                  FROM mydataset.publications),
                  STRUCT(TRUE AS flatten_json_output, 'RETRIEVAL_DOCUMENT' as task_type)
              );
              

  • SEMANTIC_SIMILARITY: specifies that the given text will be used for Semantic Textual Similarity (STS).
  • CLASSIFICATION: specifies that the embeddings will be used for classification.
  • CLUSTERING: specifies that the embeddings will be used for clustering.
• OUTPUT_DIMENSIONALITY: an INT64 value that specifies the number of dimensions to use when generating embeddings. For example, if you specify 256 AS output_dimensionality, then the ml_generate_embedding_result output column contains 256 embeddings for each input value.

  You can only use this argument if the remote model that you specify in the model argument uses one of the following models as an endpoint:

  • text-embedding-004 or later
  • text-multilingual-embedding-002 or later

multimodal embedding

Generate text embeddings by using a remote model over the multimodalembedding model:

SELECT *
FROM ML.GENERATE_EMBEDDING(
  MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,
  (CONTENT_QUERY),
  STRUCT(FLATTEN_JSON AS flatten_json_output,
  OUTPUT_DIMENSIONALITY AS output_dimensionality)
);

Replace the following:

• PROJECT_ID: your project ID.
• DATASET_ID: the ID of the dataset that contains the model.
• MODEL_NAME: the name of the remote model over a multimodalembedding@001 model.
• CONTENT_QUERY: a query whose result contains a STRING column called content.
• FLATTEN_JSON: a BOOL that indicates whether to parse the embedding into a separate column. The default value is TRUE.
• OUTPUT_DIMENSIONALITY: an INT64 value that specifies the number of dimensions to use when generating embeddings. Valid values are 128, 256, 512, and 1408. The default value is 1408. For example, if you specify 256 AS output_dimensionality, then the ml_generate_embedding_result output column contains 256 embeddings for each input value.