Google Cloud Storage Hosting w CI/CD and BigQuery Hit History Done In 5 Minutes!

Google Cloud Storage, HTTPS Load Balancing, Cloud CDN, Source Repositories, Cloud Build, Cloud Logging, and BigQuery.

5 min readJan 10, 2023

This article walks through the steps to host a website on Google Cloud Storage, setup a Git based CI/CD system on Google Cloud Source Repositories w Cloud Build Triggers, and sink website hit records to BigQuery. In five minutes.

Prerequisites

— A domain name and access to add DNS records to it
— A Google Cloud Project with Billing enabled
— OpenSSH and Git installed on your computer

Five minutes. We will use the command line when possible to save time. Let’s get started.

Preliminary Tasks

Using your web browser, sign out of all google accounts then sign back in to the Google Cloud Console using the single account you wish to use.

Browse to the Cloud Billing page and add a billing account.
Browse to the Google Cloud Resource Manager. Create and name your project.
Now browse to the Google Domain Verification page and verify ownership of your domain with Google Cloud.

Create Cloud Storage Bucket

Browse to the Google Cloud Console and launch a Cloud Shell (top right you will see an icon >). The bucket name should be your domain name, e.g., mydomain.com. Enter the following command to create your Cloud Storage bucket that will store your website files.

gsutil mb -c standard -l us-central1 gs://yourdomain.com

Grant Storage Object Viewer access to allUsers to allow public access to the files in your bucket.

gsutil iam ch allUsers:objectViewer gs://yourdomain.com

Enter the command below to specify the main page and the error page to serve.

gsutil web set -m index.html -e 404.html gs://yourdomain.com

Create Cloud Source Repository

Enter the command below to create a git repository on Google Cloud. Name the repository whatever you want. The example below uses the name myrepository.

gcloud source repos create myrepository

Create and Setup Your SSH keys

Generate Keys

Windows users launch another PowerShell window and navigate to their OpenSSH directory, e.g. cd C:\Users\myusername\.ssh. Generate the keys by entering the command below. Leave the passphrase empty, and name the key google.

ssh-keygen

Create a file namedconfig in the ssh directory. Replace User me@mydomain.com with your actual google cloud identity sign in email address. NOTE: the Add-Content lines should have 4 spaces after the first quote. The below set of PowerShell commands produces the file.

Set-Content config 'Host source.developers.google.com'
Add-Content config '    HostName source.developers.google.com'
Add-Content config '    User me@mydomain.com'
Add-Content config '    IdentityFile ~/.ssh/google'

Register Public Key with Google Cloud Repositories

Browse to Google Source Repository Key Management, and register your public key. Name it laptop, for example.

Setup Local Development Environment w/ Git

Create a development folder or directory on your computer called website. Open a terminal window such as PowerShell and navigate there, e.g., cd C:\Users\myusername\Documents\mydomain\website. Enter the following commands to initialize Git. Note that you are using an SSH address in setting up the git remote ssh://<your-email> @ source.developers.google.com/p/<project-name>/r/<repo-name>.

git init
git config user.name "John Doe"
git config user.email me@mydomain.com
git remote add google ssh://me@mydomain.com@source.developers.google.com:2022/p/my-project-name/r/myrepository
git add .
git commit -m "initial push"
git push --all google

Setup HTTPS Load Balancer

Enable the Compute Engine API

Using Cloud Shell in the Google Cloud Console, enable the Cloud Build API by entering the following command.

gcloud services enable compute.googleapis.com

Now browse to the load balancer page and Click Create Load Balancer. Click Start Configuration under HTTP(S) Load Balancing. Choose From Internet to my VMs or serverless services, and Choose Global HTTP(S) Load Balancer. Click Continue.

Front End Configuration

Name it your new frontendfrontend. Change protocol to https. Change IP Address to Create New IP Address and name it public. Click Reserve.

Click the Certificate field. Click Create New Certificate. Name it whatever you want. Choose Create Google-managed certificate
and add your domain name to the Domains field. Click Create.

Enable HTTP to HTTPS redirect.

Backend Configuration

In the Backend Services and Backend Buckets dropdown choose Create Backend Bucket. Name it backend. Click browse for bucket and highlight your bucket. Click Select. Enable Cloud CDN and Enable Cache Static Content.

Routing Rules

Leave to default settings.

Click Create. If prompted to, name the load balancer glb.

Direct Domain To Load Balancer

Get Load Balancer IP Address

Browse to the Load Balancer page and click on your `glb` resource and make note of its IP address, e.g., 34.160.39.131. You will need this address to paste in to your domain name registrar’s system in the next step. Note: you do not want to copy the :443 or :80 at the end of the address.

Update DNS Records with Domain Name Registrar

Sign in to your domain name registrar’s admin page and update your DNS records. Add an A record pointing to your glb IP address and a CNAME record pointing www to yourdomain.com.

Setup CI/CD with Cloud Build

Setup Biuld Trigger

Using Cloud Shell in the Google Cloud Console, enable the Cloud Build API by entering the following command.

gcloud services enable cloudbuild.googleapis.com

Browse to the Build Triggers page. Click Create Trigger. Name it push. Choose the Region that your bucket is in. Under Source, choose the repository that you previously created. In the Cloud Build configuration file location, enter cloudbuild.yaml. Leave the Service Account field blank. Click Create.

Place Cloud Build File in Website Folder

Create cloudbuild.yaml on your local development machine in the website folder. Change myrepository and gs://mydomain.com/ to your specific values.

steps:
  - name: gcr.io/google.com/cloudsdktool/cloud-sdk
    args:
      - rm
      - '-rf'
      - myrepository
  - name: gcr.io/google.com/cloudsdktool/cloud-sdk
    args:
      - gcloud
      - source
      - repos
      - clone
      - myrepository
  - name: gcr.io/google.com/cloudsdktool/cloud-sdk
    args:
      - gsutil
      - cp
      - myrepository/*
      - 'gs://mydomain.com/'
options:
  logging: CLOUD_LOGGING_ONLY

Create BigQuery Data Set

Using Cloud Shell in the Google Cloud Console, enter the following command to create a new BigQuery Dataset named hits.

bq mk - location=us-central1 hits

Configure Cloud Logging Sink

— Browse to the Logs Router page and click Create Sink.
— Name the sink “bigquery” click Next.
— Select Sink Service BigQuery Dataset.
— Select the hits BigQuery Dataset previously created.
— Paste the following into Logs To Send To Sink:

resource.type="http_load_balancer"
resource.labels.forwarding_rule_name="frontend"
resource.labels.url_map_name="glb"
httpRequest.requestUrl : ("mydomain.com")

Click Create.

Conclusion

You now have a website hosted on Cloud Storage, a Git based CI/CD system, and website hit logs being sent to BigQuery for historical analysis.