Deploying Pinecone with Infrastructure as Code (IaC)

Infrastructure as Code (IaC) is a programming paradigm that describes your desired cloud infrastructure declaratively. IaC tools read this declaration to reconcile your desired state with cloud provider APIs to create reproducible deployments across teams.

Pinecone supports two of the most popular IaC tools, Terraform and Pulumi. In this section, we provide tips for using Infrastructure as Code tools with Pinecone in your CI/CD workflows.

Environment Variables

Integrating IaC with CI/CD tools is often facilitated by using environment variables and custom workflows, providing a dynamic and flexible approach to managing complex environments.

These are essential in customizing the behavior of IaC scripts across different stages of a CI/CD pipeline. By leveraging environment variables, teams can dynamically adjust infrastructure configurations such as server sizes, region settings, and feature flags without altering the core IaC scripts.

Consider the following Terraform variables file:

variable "instance_size" {
  type    = string
  default = "t2.micro"
}

variable "region" {
  type    = string
  default = "us-west-1"
}

provider "aws" {
  region = var.region
}

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = var.instance_size
}

In a CI/CD environment, you can specify values by setting the following environment variables:

export TF_VAR_instance_size="t2.large"
export TF_VAR_region="us-east-1"
terraform apply

This approach decouples the configuration from the code, allowing for safer and more predictable changes that can be managed and reviewed through version control systems.

Custom Workflows with GitHub Actions

CI/CD tools allow the creation of custom workflows that can include steps to initialize, plan, apply, and destroy infrastructure managed by IaC tools.

name: Terraform CI/CD

on:
  push:
    branches:
      - main
      - develop
  pull_request:
    branches:
      - main
      - develop

jobs:
  terraform:
    name: Terraform
    runs-on: ubuntu-latest

    # Environment variables can be set here
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      TF_VAR_instance_size: "t2.micro"
      TF_VAR_region: "us-west-1"

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: "1.0.0"

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Plan
        id: plan
        run: terraform plan -out=tfplan

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        run: terraform apply -auto-approve tfplan

      - name: Terraform Destroy
        if: github.event_name == 'delete'
        run: terraform destroy -auto-approve

      # Manual approval step before applying changes on production
      - name: Wait for manual approval
        if: github.ref == 'refs/heads/main'
        uses: softprops/turnstyle@v1
        with:
          poll-interval-seconds: 10
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

You can configure these workflows to trigger based on specific conditions such as branch pushes, pull requests, or manual approvals, ensuring that infrastructure changes are integrated into the software development lifecycle in a controlled and traceable manner.

GitHub Actions provide a powerful platform to implement CI/CD pipelines directly within your GitHub repositories. When paired with IaC tools like Terraform and Pulumi, actions automate the provisioning and management of infrastructure alongside application code, making it an ideal reference setup for CI/CD workflows.

GitLab has similar functionality, and there are other popular CI/CD workflow technologies, including Jenkins, Spinnaker, ArgoCD, and CircleCI. The concepts in this blog apply to any CI/CD tool.

Terraform

As we showed above, CI/CD tools can be paired with Terraform to manage infrastructure.

A typical workflow might include steps to initialize the Terraform configuration, create an execution plan, apply the plan to provision resources and provide feedback on the deployment process. You can find more about Pinecone's Terraform provider at https://docs.pinecone.io/integrations/terraform.

Pulumi

Pulumi takes a different approach by using general-purpose programming languages like JavaScript, Python, or Go to define infrastructure:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Load environment variables
const instanceSize = process.env.INSTANCE_SIZE || "t2.micro";
const region = process.env.AWS_REGION || "us-west-1";

// Set the AWS region
const provider = new aws.Provider("aws-provider", {
    region: region,
});

// Create an AWS instance
const instance = new aws.ec2.Instance("web-server", {
    instanceType: instanceSize,
    ami: "ami-0c55b159cbfafe1f0",
}, { provider: provider });

export const instanceId = instance.id;
export const instanceRegion = provider.region;

CI/CD tools can run Pulumi scripts to deploy infrastructure and then handle application code deployment. Pulumi supports:

• Steps for previewing changes.
• Updating resources.
• Running integration tests to ensure deployments meet the desired state as defined in the code.

You can find out more about Pinecone's Pulumi provider at https://docs.pinecone.io/integrations/pulumi.