GCP Installation Guide

Instructions to install in GCP infra

Infrastructure Requirements

  1. Minimal Installation:

    • You need a system with a minimum of 16 CPUs. We recommend using 2 nodes with 8 cores each, totalling 32GB.

    • All machines must reside in the same zone to avoid data transfer charges across zones. Our Obsrv installer will automatically create the GKE cluster for you.

  2. Networking Environment:

    • The networking requirements change w.r.t the type of cluster we are creating. Follow this link to find out different types of clusters supported by Google Cloud in the region you are installation.

    • By default our setup script creates a Single Zonal Public Cluster and the required VPC’s for it.

Software Prerequisites

Installation of Obsrv requires the following CLI tools as prerequisites. Please note that the following instructions for installing the prerequisites are provided only for Linux based operating systems. Please follow the instructions for the specific tools depending upon your operating system.


  • Terraform CLI version 1.5.x or older. Versions above 1.5.x are not MPL licensed.

    curl "<https://releases.hashicorp.com/terraform/1.5.2/terraform_1.5.2_linux_amd64.zip>" -o "terraform.zip" && unzip terraform.zip && sudo mv terraform /usr/local/bin/ && rm terraform.zip


  • Terragrunt CLI version 0.48 or later.

    curl -OL https://github.com/gruntwork-io/terragrunt/releases/download/v0.49.0/terragrunt_linux_amd64 && sudo mv terragrunt_linux_amd64 /usr/local/bin/terragrunt && sudo chmod +x /usr/local/bin/terragrunt


  • Terrahelp version 0.7.5 or later

    curl -OL https://github.com/opencredo/terrahelp/releases/download/v0.7.5/terrahelp_0.7.5_linux_386.tar.gz && tar -xzf terrahelp_0.7.5_linux_386.tar.gz && sudo mv terrahelp /usr/local/bin/terrahelp && sudo chmod +x /usr/local/bin/terrahelp


  • Helm version 3.10.2 or later

    curl https://get.helm.sh/helm-v3.10.2-linux-amd64.tar.gz -o helm.tar.gz && tar -zxvf helm.tar.gz && sudo mv linux-amd64/helm /usr/local/bin/

Setup Google Cloud SDK

  • Install Google Cloud SDK by following the instructions here

    gcloud init
    gcloud auth application-default login
  • Install additional dependencies to authenticate with GKE. Please see Installing the gke-gcloud-auth-plugin for reference.

    gcloud components install gke-gcloud-auth-plugin

Create a GCP Project

Installation Steps:

  1. Clone the obsrv-automation repository:

    git clone https://github.com/Sanketika-Obsrv/obsrv-automation.git
  2. Navigate to the setup directory:

    cd ./obsrv-automation/terraform/gcp
  3. Update Configuration Files:

    Update the vars/cluster_overrides.tfvars file to match your environment and requirement settings. Example configuration in cluster_overides.tf

    project                       = "<myproject>"
    building_block                = "obsrv"
    env                           = "dev"
    region                        = "us-central-1"
    gke_cluster_location          = "us-central-1"
    zone                          = "us-central-1-a"
    timezone                      = "UTC"
    service_type                  = "LoadBalancer"
    # cluster sizing
    gke_node_pool_instance_type   = "c2d-standard-8"
    gke_node_pool_scaling_config = {
      desired_size = 2
      max_size = 2
      min_size = 1
    # Image Tags
    command_service_image_tag    = "1.0.0-GA"
    web_console_image_tag        = "1.0.0-GA"
    dataset_api_image_tag        = "1.0.2-GA"
    flink_image_tag              = "1.0.1-GA"
    secor_image_tag              = "1.0.0-GA"
    superset_image_tag           = "3.0.2"

    Note: if you are changing the region, also check if the instance type is supported in that region. If not please change the instance type from the supported list which can be found in this link

  4. GCS Bucket:

    Obsrv installation requires a GCS bucket to be created which will be subsequently used for storing the cluster state. The bucket name will then be referenced in the installation script mentioned below. For e.g, if the bucket created is obsrv-cluster-state, please export it as a environment variable as show below, which will be referenced in the terragrunt.hcl under the environment variable GOOGLE_TERRAFORM_BACKEND_BUCKET. The bucket will be created automatically if it doesn’t exist. The region can be updated by passing it in an environment variable GOOGLE_TERRAFORM_BACKEND_BUCKET_REGION

    export GOOGLE_TERRAFORM_BACKEND_BUCKET=obsrv-cluster-tfstate
  5. Run Installation Script: Execute the following command to start the installation process:

    terragrunt init
    terragrunt plan --var-file=vars/cluster_overrides.tfvars # to verify if everything is in order
    terragrunt apply --var-file=vars/cluster_overrides.tfvars
  6. Monitor Installation Progress: The script will begin installing Obsrv within the GKE cluster. Monitor the progress and follow any on-screen prompts or instructions.

  7. Completion: Once the installation process is complete, verify that Obsrv has been successfully installed and configured within your GKE cluster.

Upgrade Steps:

  1. Take latest code from obsrv-automation repository

    cd ./obsrv-automation
    git pull
    cd ./terraform/gcp
  2. Ensure all the configuration configured during the installation is properly updated in all places.

  3. Run the terraform to upgrade the cluster to the latest versions.

    terragrunt apply --var-file=vars/cluster_overrides.tfvars

Last updated