Automating Cloud Run Deployment with Pub/Sub Integration Using Terraform and GitHub Actions

Automating Cloud Run Deployment with Pub/Sub Integration Using Terraform and GitHub Actions

Introduction:

As a cloud engineer experienced in implementing solutions on GCP, you understand the importance of automation in optimizing workflows. In this blog post, we will guide you through the process of automating the deployment of a simple "Hello World" application on Google Cloud Run, coupled with seamless integration with Google Pub/Sub. All of this will be achieved using Terraform for infrastructure as code and GitHub Actions for continuous integration and deployment (CI/CD).

Prerequisites:

Before we dive into the technical details, make sure you have the following prerequisites in place:

• A Google Cloud Platform (GCP) account with the necessary permissions to create resources, including Google Cloud Run services and Pub/Sub topics.

• A GitHub account with a repository set up to manage your Terraform code.

Repository Structure:

To keep our project organized, we'll follow a specific directory structure within our GitHub repository: GitHub-repo
Here's an overview of the structure:

You can clone my public repository for reference: GitHub-repo

Terraform Configuration:

Let's explore the details of our Terraform code, which automates the infrastructure setup:

main.tf:

The main.tf file defines the Terraform configuration for provisioning a Google Cloud Run service and a Pub/Sub topic and subscription. It specifies the container image to deploy and sets up access permissions using IAM policies.

resource "google_pubsub_topic" "cloud_run_topic" {
  name = var.topic_name
}

resource "google_pubsub_subscription" "cloud_run_subscription" {
  name  = var.subscription_name
  topic = google_pubsub_topic.cloud_run_topic.name
}

resource "google_cloud_run_service" "hello_world_service" {
  name     = var.service_name
  location = var.region
  template {
    spec {
      containers {
        image = var.image_name
      }
    }
  }
  traffic {
    percent         = 100
    latest_revision = true
  }
  depends_on = [
    google_pubsub_subscription.cloud_run_subscription,
  ]
}

provider.tf:

The provider.tf file configures the Terraform provider for Google Cloud, specifying the required provider version, project ID, region, and zone.

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "4.58.0"
    }
  }
backend "gcs" {
  bucket  = "demo-backend-bucket"
  prefix  = "terraform-state"
}
}

provider "google" {
  project     = var.project_id
  region      = var.region
  zone        = var.zone
}

variables.tf:

variables.tf defines the Terraform variables used in the configuration:

variable "project_id" {
  description = "The ID of the Google Cloud project."
}

variable "region" {
  description = "The region in which to deploy the Cloud Run service."
}

variable "zone" {
  description = "The zone in which to deploy the Cloud Run service."
}

variable "topic_name" {
  description = "The name of the Pub/Sub topic to create."
}

variable "subscription_name" {
  description = "The name of the Pub/Sub subscription to create."
}

variable "service_name" {
  description = "The name of the Cloud Run service to create."
}

variable "image_name" {
  description = "The name and tag of the Docker image to deploy to Cloud Run."
}

terraform.tfvars:

In terraform.tfvars, you'll define the values for Terraform variables:

project_id = "your-project-id"

region = "us-central1"

zone = "us-central1-c"

topic_name = "cloud-run-topic"

subscription_name = "cloud-run-subscription"

service_name = "my-cloud-run-service"

image_name = "gavininthecloud/helloworld:1.0"

GitHub Actions Workflow:

Our CI/CD pipeline is set up in the .github/workflows/terraform.yml file. It defines stages, jobs, and associated scripts to validate, plan, apply, and potentially destroy Terraform changes.

name: Deploy to Cloud Run

on:
  push:
    branches:
      - main

jobs:
  terraform:
    name: "Terraform"
    runs-on: ubuntu-latest
    env:
      GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
    defaults:
      run:
        working-directory: src
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: 1.0.1
          terraform_wrapper: false

      - name: Build and Push Docker Image
        run: |
          gcloud auth configure-docker
          docker build -t gavininthecloud/helloworld:1.0 .
          docker push gavininthecloud/helloworld:1.0

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Format
        id: fmt
        run: terraform fmt 
        
      - name: Terraform Plan
        id: plan
        run: terraform plan 

      - name: Terraform Apply
        id: apply
        run: terraform apply -auto-approve

        #- name: Terraform Destroy
        #id: destroy
        #run: terraform destroy -auto-approve

Dockerfile:

This section defines a Docker image based on Alpine Linux and specifies the command to run when a container is launched, which in this case, is a simple "Hello World!" message.

FROM alpine
CMD echo "Hello World!"

Implementation Steps:

With the configurations in place, here are the steps to implement this automated deployment:

1. Set Up GitHub Repository: Create a new repository on GitHub or use an existing one to host your Terraform code. If needed, clone this repository: GitHub-repo

2. Configure GCP Provider: In the provider.tf file, configure the GCP provider with your backend bucket, project ID, region, and zone.

3. Set Secrets in GitHub: In your GitHub repository, navigate to Settings > Secrets > New repository secret. Add a new secret named "GOOGLE_CREDENTIALS" and paste the contents of your Google Cloud service account key file into the value field. This securely provides the necessary credentials for Terraform to authenticate with GCP.

Note: Ensure that there are no whitespace characters in your token content before pasting it.

4. Run the GitHub Actions Workflow: Commit and push your Terraform code to the GitHub repository. The GitHub Actions workflow will automatically trigger, executing the defined stages and jobs.

5. Verify Resource Creation in GCP: Check the Google Cloud Run console to confirm the successful deployment of your "Hello World" application and the creation of the Pub/Sub resources.

Conclusion:

In this blog post, we've demonstrated how to automate the deployment of a "Hello World" application on Google Cloud Run while seamlessly integrating it with Google Pub/Sub. By following the provided steps and configurations, you can efficiently manage and automate your GCP resources. Remember to regularly update your Terraform code and pipeline to reflect any changes in your cloud infrastructure requirements. Automation with Terraform and GitHub Actions streamlines cloud workflows, enhances consistency, and reduces manual intervention. Happy automating!

References: GitHub-repo