Configuring OpenID Connect with GCP Workload Identity Federation in GitLab CI/CDs

Configuring OpenID Connect with GCP Workload Identity Federation in GitLab CI/CDs

Introduction: GitLab CI/CD is a powerful tool for automating software delivery pipelines, and Google Cloud Platform (GCP) provides a secure and scalable infrastructure for running your applications. In this tutorial, we will explore how to configure OpenID Connect (OIDC) with GCP Workload Identity Federation in GitLab CI/CD. This integration allows you to authenticate to Google Cloud from your CI/CD jobs using JSON Web Token (JWT) tokens, enabling the generation of on-demand, short-lived credentials without the need to store any secrets.

Prerequisites: Before getting started, make sure you have the following prerequisites in place:

• A Google Cloud account

• A Google Cloud project with the Workload Identity Pool Admin, Service Account Token Creator and Cloud Storage Admin permissions assigned to your account.

Repo Structure: To keep our project organized, we will follow the following structure within our GitLab repo1: GitLab-Repo1

You can simply clone my public repository: GitLab-Repo1

Now, let's dive into the details of each component.

Terraform Configuration: The main.tf file contains the Terraform configuration for creating a Google Cloud Workload Identity Pool, Workload Identity Provider, Service Account, and granting IAM permissions. It utilizes the Google Cloud Platform provider.

Let's break down the code to understand each component:

Workload Identity Pool: The google_iam_workload_identity_pool resource creates a Workload Identity Pool. In this case, the pool is named "gitLab" with the ID "dhruv-gitlab-oidc-pool-d". It is enabled and represents an identity pool for GitLab.

Workload Identity Provider: The google_iam_workload_identity_pool_provider resource creates a Workload Identity Provider within the previously defined pool. It is an OpenID Connect (OIDC) provider named "gitlab/gitlab" with the ID "dhruv-oidc-provider-gitlab". The provider's issuer URI is set to "https://gitlab.com/" and the allowed audiences are "https://gitlab.com" and "https://gitlab.example.com". It also includes attribute mappings to extract information from the GitLab claims.

Service Account: The google_service_account resource creates a Service Account named "dhruv-oidc-account" with the display name "dhruv-oidc-account". This service account will be used for authentication purposes.

IAM Permissions: The google_project_iam_member resource grants IAM permissions to the Service Account created earlier. In this case, it assigns the "roles/iam.serviceAccountTokenCreator" role to the Service Account, giving it the necessary permissions to create access tokens.

External Identity Permissions: The google_service_account_iam_binding resource grants external identity permissions to impersonate the Service Account. It assigns the "roles/iam.serviceAccountTokenCreator" role to the external identity, allowing it to act as the specified service account within the Workload Identity Pool.

This configuration sets up a Workload Identity Pool and Provider, creates a Service Account, and grants the necessary permissions for the Service Account to create access tokens and impersonate the Service Account.

# Create the Workload Identity Pool
resource "google_iam_workload_identity_pool" "oidc_pool" {
  workload_identity_pool_id = "your-pool-id" //Replace with your desired pool id
  display_name = "gitLab" //Enter your preferred display name
  description  = "GitLab Workload Identity Pool"
  disabled     = false
}

# Create the Workload Identity Provider
resource "google_iam_workload_identity_pool_provider"  "oidc_provider" {
  workload_identity_pool_id           = google_iam_workload_identity_pool.oidc_pool.workload_identity_pool_id
  workload_identity_pool_provider_id = "oidc-provider-gitlab" //Replace with your preferred provider id
  disabled                           = false
  display_name                       = "gitlab/gitlab" //Enter your preferred display name
  attribute_mapping = {
      "google.subject"       = "assertion.sub"
      "attribute.user_login" = "assertion.user_login"
    }
  oidc {
    issuer_uri        = "https://gitlab.com/"
    allowed_audiences = ["https://gitlab.com", "https://gitlab.example.com" ]
  }
}

# Create the Service Account
resource "google_service_account" "gitlab_oidc_account" {
  account_id   = "my-oidc-account" //Replace with your desired account_id
  display_name = "my-oidc-account" //Enter your preferred display name
}

# Grant IAM permissions to the Service Account
resource "google_project_iam_member" "gitlab_iam_member" {
  project = "your_project_id" //Replace with your desired project_id
  role    = "roles/iam.serviceAccountTokenCreator"
  member  = "serviceAccount:${google_service_account.gitlab_oidc_account.email}"
}

# Grant external identity permissions to impersonate the Service Account
resource "google_service_account_iam_binding" "gitlab_iam_binding" {
  service_account_id = google_service_account.gitlab_oidc_account.name
  role               = "roles/iam.serviceAccountTokenCreator"
  members = [
    "principalSet://iam.googleapis.com/projects/912633640382/locations/global/workloadIdentityPools/dhruv-gitlab-oidc-pool-d/*"
  ]
}

2. Provider Configuration: The provider.tf code sets up the Google Cloud provider for Terraform with the specified project ID, region, and zone. The required_providers block ensures the use of the specified version of the Google provider. The backend "gcs" block configures the backend as Google Cloud Storage (GCS) to store the Terraform state in the specified bucket.

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "4.58.0"
    }
  }

  backend "gcs" {
    bucket  = "your_backend_bucket" // Replace with your backend bucket name
    prefix  = "terraform/state"
  }
}

provider "google" {
  project     = "engineer-cloud-nprod" // Replace with your project ID
  region      = "us-central1" // Enter your preferred region
  zone        = "us-central1-c" // Enter your preferred zone
}

3. GitLab CI/CD Configuration: The .gitlab-ci.yml file sets up the CI/CD pipeline for automating the infrastructure deployment process. It defines stages, jobs, and associated scripts to perform tasks such as linting, testing, and applying Terraform changes.

---
workflow:
  rules:
    - if: $CI_COMMIT_BRANCH != "main" && $CI_PIPELINE_SOURCE != "merge_request_event"
      when: never
    - when: always

variables:
  TF_DIR: ${CI_PROJECT_DIR}/terraform
  STATE_NAME: "terraform-state"

stages:
  - validate
  - plan
  - apply
  - destroy

image:
  name: hashicorp/terraform:light
  entrypoint: [""]
  
before_script:
  - terraform --version
  - cd ${TF_DIR}
  - terraform init -reconfigure

validate:
  stage: validate
  script:
    - terraform validate
  cache:
    key: ${CI_COMMIT_REF_NAME}
    paths:
    - ${TF_DIR}/.terraform
    policy: pull-push

plan:
  stage: plan
  script:
    - terraform plan 
  dependencies:
    - validate
  cache:
    key: ${CI_COMMIT_REF_NAME}
    paths:
    - ${TF_DIR}/.terraform
    policy: pull


apply:
  stage: apply
  script:
    - terraform apply  -auto-approve
  dependencies:
    - plan
  cache:
    key: ${CI_COMMIT_REF_NAME}
    paths:
    - ${TF_DIR}/.terraform
    policy: pull

destroy:
  stage: destroy
  script:
    - terraform destroy  -auto-approve
  dependencies:
    - plan
    - apply
  cache:
    key: ${CI_COMMIT_REF_NAME}
    paths:
    - ${TF_DIR}/.terraform
    policy: pull
  when: manual

Make sure to replace the variable STATE_NAME as per your choice.

Implementation Steps: Now, let's walk through the implementation steps to create the bastion host and establish secure access to private instances using Terraform and GitLab CI/CD.

1. Set up GitLab Repository: Create a new repository on GitLab or use an existing one to host your Terraform code or clone my GitLab repo to yours.

2. Configure GCP Provider: In the provider.tf file within your GitLab repository, configure the Google Cloud Platform (GCP) provider. Provide the necessary information such as your GCS bucket ID, project ID, region, and zone. This will allow Terraform to communicate with your GCP environment.

3. Set Secrets in GitLab: In your GitLab repository, navigate to Settings > CI/CD > Variables. Add a new variable named "GOOGLE_CREDENTIALS" and paste the contents of your Google Cloud service account key file into the value field. This securely provides the necessary credentials for Terraform to authenticate with GCP.

   

   
   NOTE: Remove white spaces in your token content and paste it.

4. Run the Pipeline: Commit and push your Terraform code to the GitLab repository. This will trigger the GitLab CI/CD pipeline. Monitor the pipeline execution in the CI/CD section of your repository to ensure it completes successfully.

5. Check Resource Creation in GCP: After the pipeline completes, verify the creation of resources in the Google Cloud Platform (GCP) Console. Access the GCP Console and navigate to the relevant project. Ensure that the Workload Identity Pool, Workload Identity Provider, Service Account, and associated IAM permissions are created as configured. Validate that the resources align with your desired configuration and meet your requirements for OIDC integration.

Repo Structure: Here is the repo structure of our GitLab repo2: GitLab-Repo2  which we use to generate access token using resources created from repo1.

You can simply clone my public repository: GitLab-Repo2 

Now, let's dive into the details of each component.

get_credentials.sh:

• This script retrieves the access token required for authenticating with Google Cloud Platform (GCP) using OpenID Connect (OIDC) and Workload Identity Federation.

• It constructs a JSON payload specifying the audience, grant type, requested token type, scope, subject token type, and subject token.

• The payload is sent as a request to the STS (Security Token Service) endpoint to obtain a federated token.

• The federated token is then used in another request to the IAM Credentials API to generate an access token.

• The final access token is outputted by the script.

.gitlab-ci.yml:

• This GitLab CI/CD configuration file defines the "generated-credentials" job.

• The job uses an Alpine-based Docker image with curl and jq installed.

• It sets up the execution environment by giving execute permissions to the "get_credentials.sh" script.

• The script is then executed to obtain the access token.

• The job also defines an environment variable "GITLAB_OIDC_TOKEN" with the required audience ("https://gitlab.com") for OIDC authentication.

You can find the code for get_credentials.sh and .gitlab-ci.yml in this repo2: GitLab-Repo2 

Implementation Steps: Now, let's walk through the implementation steps to create the bastion host and establish secure access to private instances using Terraform and GitLab CI/CD.

1. Set up GitLab Repository: Create a new repository on GitLab or use an existing one to host your Terraform code or clone my GitLab repo to yours.

2. Set Secrets in GitLab: In your GitLab repository, navigate to Settings > CI/CD > Variables and add the following variables as secrets:
   

   • PROJECT_NUMBER: Set the value to your Google Cloud project number (not the project name).

   • POOL_ID: Set the value to the ID of the Workload Identity Pool created earlier.

   • PROVIDER_ID: Set the value to the ID of the Workload Identity Provider created earlier.

   • SERVICE_ACCOUNT_EMAIL: Set the value to the email address of the Service Account created earlier in the Terraform configuration. This email address represents the identity of the service account and will be used for granting permissions and generating access tokens.

     

3. Run the Pipeline: Commit and push your code changes to the GitLab repository. This will trigger the GitLab CI/CD pipeline to execute the defined stages and jobs. Monitor the pipeline execution in the CI/CD section of your repository to ensure it completes successfully.

4. Access Token Generation: After running the pipeline, the get_credentials.sh script will be executed, which uses the provided GitLab OIDC token and makes API requests to generate an access token. The output of the access token will be displayed in the pipeline logs.

   Pipeline logs should look like this:

The result is a Google Cloud OAuth 2.0 access token, which you can use to authenticate to most Google Cloud APIs and services when used as a bearer token. You can also pass this value to the gcloud CLI by setting the environment variable CLOUDSDK_AUTH_ACCESS_TOKEN.

Conclusion: By configuring OpenID Connect with GCP Workload Identity Federation in GitLab CI/CD, you can securely authenticate to Google Cloud and obtain short-lived credentials without exposing any secrets. This integration enhances the security and automation capabilities of your CI/CD pipelines, enabling seamless interaction with Google Cloud resources.

To further simplify the process, you can leverage infrastructure-as-code tools like Terraform to automate the creation of the required resources. The provided GitLab repositories offer an example implementation using Terraform and demonstrate how to automate the setup process.

Integrating GitLab CI/CD with GCP Workload Identity Federation empowers your development teams to leverage the full potential of Google Cloud while maintaining a secure and streamlined workflow.

That's it for our tutorial on configuring OpenID Connect with GCP Workload Identity Federation in GitLab CI/CD. We hope this guide helps you enhance the security and efficiency of your CI/CD pipelines on Google Cloud.

References:

• GitLab-Repo1

• GitLab-Repo2 

• https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/