Google and GitLab Strengthen Partnership with CI/CD and Cloud Deploy Integration: VivaOps Driving Client Value

Continuous Delivery (CD) has emerged as a cornerstone of modern development practices, enabling teams to automate the entire release process from code commit to production. By leveraging CD, businesses can accelerate their time-to-market, reduce the risk of errors, and maintain a consistent, reliable deployment process. 

Automation plays a pivotal role in this transformation, ensuring that every release is not only faster but also more reliable, allowing development teams to focus on innovation rather than manual tasks. 

In this blog, we'll explore how integrating GitLab CI/CD with Google Cloud's Cloud Deploy can elevate your software delivery pipeline to new heights, driving efficiency and success in every deployment.

Elements of the Solution

GitLab CI/CD

• Integrated Automation Platform: GitLab CI/CD is a comprehensive platform that automates the entire software development lifecycle—from building to testing and deploying code.

• Benefits of Automation:

  • Reduces manual errors.

  • Accelerates release cycles.

  • Ensures consistent quality across all stages of development.

• AI Integration in GitLab:

  • Automates repetitive tasks.

  • Identifies potential issues early.

  • Provides intelligent insights to guide developers toward optimal solutions.

• Efficiency and Focus: This combination of automation and AI allows development teams to concentrate on innovation, driving overall productivity.

  
  

Cloud Deploy

• Google-Managed Service: Cloud Deploy automates the deployment of applications across various runtime environments, including Google Kubernetes Engine (GKE) and Cloud Run.

• Key Features:

  • Delivery Pipelines: Define and manage the deployment sequence across different stages.

  • Advanced Deployment Strategies:

    • Progressive Releases: Gradual rollout of updates to reduce risks.

    • Approvals: Ensure that only thoroughly vetted code reaches production.

• Reliable Deployments: Cloud Deploy ensures every release is executed with precision, enhancing the speed and reliability of deployments.

Google Cloud GitLab Integration

• Streamlined Authentication:

  • Uses workload identity federation to authenticate GitLab CI/CD with Google Cloud services.

  • Eliminates the need for service account keys, reducing security risks.

• Artifact Management:

  • Seamless management of artifacts in Google Artifact Registry directly from GitLab.

• Automated Deployment:

  • Facilitates deployment processes using Cloud Deploy, enhancing efficiency.

• Enhanced Security and Workflow: These integrations streamline workflows and bolster security, allowing teams to focus on delivering value rather than managing infrastructure complexities.

  
  

Pipeline Flow

Setting up and executing a Continuous Delivery pipeline using GitLab CI/CD and Cloud Deploy involves a series of well-orchestrated steps, each playing a crucial role in ensuring that code moves seamlessly from development to production. Here’s a breakdown of the process:

1. Setting Up the Pipeline

• Pipeline Configuration:

  • The pipeline is defined in the .gitlab-ci.yml file, specifying the stages and jobs to be executed.

  • This includes stages like build, push, deploy-to-qa, and promote-to-prod, each associated with specific tasks in the pipeline.

• Defining Roles:

  • Developers: Start the process by creating a feature branch, making code changes, and initiating a merge request.

  • QA Teams: Review and test the changes in a QA environment, ensuring they meet the required standards.

  • Product Managers: Approve the final deployment to production, overseeing the release process to ensure it aligns with business objectives.

    
    

2. Key Pipeline Components

• Image-Building:

  • The pipeline begins with building a container image using the updated code. This is done automatically when a merge request is opened.

  • The built image is then tagged and prepared for deployment.

• Artifact Uploading:

  • Once the image is built, it is uploaded to Google Artifact Registry. This step is critical as it ensures that the artifact (the built image) is stored securely and is accessible for deployment.

• Release Creation and Promotion:

  • Deploy-to-QA Stage: A release is created in Cloud Deploy, and the image is deployed to a QA environment. The QA team conducts user acceptance testing in this controlled setting.

  • Promote-to-Prod Stage: After successful QA testing and approval from the QA team, the release is promoted to production. This stage may include canary releases, where the new version is rolled out gradually to ensure stability.

    
    

3. Executing the Pipeline

• Merge Request Process:

  • The developer opens a merge request, which triggers the pipeline to start executing the defined jobs.

  • The image is built and pushed to the Artifact Registry, and the release is deployed to the QA environment.

• Testing and Approval:

  • The QA team tests the deployed application in the QA environment.

  • Upon successful testing, the code is merged into the main branch, triggering the promotion to the production stage.

• Final Deployment:

  • The product manager reviews and approves the final rollout to production.

  • Cloud Deploy handles the release to production, potentially using a canary strategy to ensure a smooth and stable transition.

  
  

Suggested Read: Key Strategies for Secure GitLab CI/CD Pipelines

This structured approach ensures that every stage of the pipeline is handled efficiently, with clear roles and responsibilities assigned to each team member. By leveraging GitLab CI/CD and Cloud Deploy, organizations can achieve a streamlined, reliable, and automated Continuous Delivery process, ensuring that every deployment is executed precisely and confidently.

Source: Google Cloud Blogs

Prepare Your Environment

To successfully set up a Continuous Delivery pipeline using GitLab CI/CD and Google Cloud’s Cloud Deploy, you’ll need to prepare your environment by ensuring all necessary components are in place. Below are the prerequisites and detailed steps to configure your environment.

1. Prerequisites

• GitLab Account:

  • Ensure you have a GitLab account (Free, Premium, or Ultimate).

  • Fork the required GitLab repository to your account for cloning locally.

• Google Cloud Project:

  • You need a Google Cloud project with project owner access to manage resources.

  • Set up billing and enable necessary APIs (Cloud Run, Cloud Deploy, Artifact Registry).

• Repository Setup:

  • Clone the GitLab repository to your local workstation.

  • Ensure the repository contains the .gitlab-ci.yml file, which defines your pipeline configuration.

    
    

2. Configuring Necessary Components

• Workload Identity Federation:

  • Purpose: Enables secure, keyless authentication between GitLab CI/CD jobs and Google Cloud services.

  • Setup:

    1. In Google Cloud, create a Workload Identity Pool.

    2. Set up the workload identity provider that GitLab will use for authentication.

    3. Map the GitLab service account to a Google Cloud service account with necessary permissions.

• Artifact Registry:

  • Purpose: Stores container images and other build artifacts securely.

  • Setup:

    1. Create a standard Docker repository in Google Artifact Registry.

    2. Ensure the repository is in the desired Google Cloud region.

    3. Configure your GitLab CI/CD pipeline to push built images to this repository.

• Cloud Deploy Pipeline Creation:

  • Purpose: Automates the deployment of your application across different environments (e.g., QA, production).

  • Setup:

    1. Create a delivery pipeline manifest (.yaml file) in your local repository, defining the stages (e.g., QA, prod) and targets (e.g., Cloud Run services).

    2. Use the gcloud deploy command to apply this manifest, creating the pipeline and targets in your Google Cloud project.

    3. Configure your GitLab CI/CD pipeline to interact with Cloud Deploy, automating the release and promotion processes.

       
       

3. Finalizing Configuration

• Permissions:

  • Assign appropriate roles to the Google Cloud service account used by GitLab, including:

    • roles/artifactregistry.reader

    • roles/artifactregistry.writer

    • roles/clouddeploy.approver

    • roles/clouddeploy.releaser

    • roles/iam.serviceAccountUser

    • roles/run.admin

    • roles/storage.admin

• Running the Initial Pipeline: Manual Execution:

  • From the GitLab UI, manually trigger the pipeline to create the initial release and deploy to the QA and production Cloud Run services.

  • Validate that the setup is correct by accessing the deployed services through their respective URLs.

    
    

By carefully preparing your environment with these prerequisites and configurations, you ensure a smooth setup process, enabling the automation of your software delivery pipeline using GitLab CI/CD and Google Cloud services. This preparation lays the foundation for a robust Continuous Delivery system that supports rapid, reliable deployments.

Running the Pipeline

Once your environment is set up, running the pipeline becomes a straightforward process that ensures your code changes are seamlessly integrated, tested, and deployed across different environments. Here’s a step-by-step guide on how to update code, create branches, execute the pipeline, and manage approvals and rollouts.

1. Updating Code and Creating Branches

• Creating a Feature Branch:

  • Start by navigating to the root of your local repository.

  • Create a new feature branch where you will make your code changes:

    git checkout -b new-feature

  • This isolated branch allows you to work on specific features or fixes without affecting the main branch.

• Making Code Changes:

  • Open the relevant files in your repository and make the necessary updates. For example, you might update a message in your application’s code.

  • Once your changes are complete, stage and commit them:

    git add .

    git commit -m "Implemented new feature"

• Pushing Changes to GitLab:

  • Push your changes to the remote repository on GitLab:

    git push origin new-feature

  • After pushing, you’ll receive a URL to open a merge request (MR) in GitLab.

2. Executing the Pipeline

• Opening a Merge Request:

  • In GitLab, open the URL provided in the terminal or navigate to your project’s merge request section.

  • Click “Create merge request” to initiate the process. This action automatically triggers the pipeline, which starts executing the predefined jobs in the .gitlab-ci.yml file.

• Pipeline Stages:

  • Build Stage: The pipeline builds a container image using your updated code.

  • Push Stage: The built image is pushed to Google Artifact Registry.

  • Deploy-to-QA Stage: The image is deployed to the QA environment, where the QA team can perform tests.

• Monitoring Pipeline Execution:

  • You can monitor the progress of each job in the GitLab pipeline dashboard. Clicking on individual jobs provides detailed logs of the execution process.

  • Once the QA tests are successful, the pipeline pauses, waiting for further actions.

    
    

3. Managing Approvals and Rollouts

• Merging the Code:

  • After the QA team completes their tests and approves the changes, the code can be merged into the main branch. This merge triggers the next stage of the pipeline, which promotes the release to production.

• Canary Release Strategy:

  • Initial Rollout: The pipeline deploys the new version of the application to production using a canary release strategy. This approach gradually shifts a small percentage of traffic (e.g., 10%) to the new version, while the majority of users continue to use the old version.

  • Monitoring: During this phase, the application’s performance and stability are monitored to ensure that the new release does not introduce any issues.

  • Approval for Full Rollout: Once the canary phase is deemed successful, a product manager or a designated approver reviews the rollout and grants approval for full deployment.

• Final Rollout:

  • After approval, the pipeline advances the rollout to 100%, directing all user traffic to the new version of the application.

  • This gradual, controlled approach minimizes risks, ensuring a smooth transition to the updated application.

    
    

4. Observing the Rollout

• Traffic Management:

  • You can observe how the traffic is split between the old and new versions by generating requests to the production URL. Over time, as the rollout progresses, all traffic will be routed to the new version.

• Finalizing the Deployment:

  • Once the rollout is complete and stable, the pipeline marks the deployment as successful, and the updated application is fully live in production.

    
    

By following these steps, you can efficiently manage the entire process of updating, testing, and deploying your application using GitLab CI/CD and Cloud Deploy. The canary release strategy ensures that new features are introduced in a controlled manner, minimizing potential disruptions and allowing for quick rollback if necessary. This methodical approach ensures high reliability and confidence in every deployment.

Final Thought:

VivaOps is committed to helping enterprises maximize their GitLab investments through expert DevSecOps solutions. Our team, with experience from leading tech companies, specializes in integrating AI into DevSecOps, enhancing automation, and improving code quality. By aligning with the practices discussed in this blog, we ensure that your CI/CD pipelines are not only efficient but also secure and scalable. Whether you’re setting up a new pipeline or optimizing an existing one, VivaOps provides the expertise to streamline your software delivery process.