My Kubernetes cluster (GKE) with Crossplane


The target of this documentation is to be able to create and manage a GKE cluster using Crossplane.

Crossplane leverages Kubernetes base principles in order to provision cloud resources and much more: a declarative approach with drift detections and reconciliations using control loops 🤯. In other words, we declare what cloud resources we want and Crossplane ensures that the target state matches the one applied through the Kubernetes API.

Here are the steps we'll follow in order to get a Kubernetes cluster for development and experimentations use cases.

🐳 Create the local k3d cluster for Crossplane's control plane

k3d is a lightweight kubernetes cluster that leverages k3s that runs in our local laptop. There are several deployment models for Crossplane, we could for instance deploy the control plane on a management cluster on Kubernetes or a control plane per Kubernetes cluster.
Here I chose a simple method which is fine for a personal use case: A local Kubernetes instance in which I'll deploy Crossplane.

Let's install k3d using asdf.

1asdf plugin-add k3d
3asdf install k3d $(asdf latest k3d)
4* Downloading k3d release 5.4.1...
5k3d 5.4.1 installation was successful!

Create a single node Kubernetes cluster.

1k3d cluster create crossplane
3INFO[0043] You can now use it like this:
4kubectl cluster-info
6k3d cluster list
7crossplane   1/1       0/0      true

Check that the cluster is reachable using the kubectl CLI.

1kubectl cluster-info
2Kubernetes control plane is running at
3CoreDNS is running at
4Metrics-server is running at

We only need a single node for our Crossplane use case.

1kubectl get nodes
2NAME                      STATUS   ROLES                  AGE   VERSION
3k3d-crossplane-server-0   Ready    control-plane,master   26h   v1.22.7+k3s1

☁️ Generate the Google Cloud service account


Store the downloaded crossplane.json credentials file in a safe place.

Create a service account

2gcloud iam service-accounts create crossplane --display-name "Crossplane" --project=${GCP_PROJECT}
3Created service account [crossplane].

Assign the proper permissions to the service account.

  • Compute Network Admin
  • Kubernetes Engine Admin
  • Service Account User
 1SA_EMAIL=$(gcloud iam service-accounts list --filter="email ~ ^crossplane" --format='value(email)')
 3gcloud projects add-iam-policy-binding "${GCP_PROJECT}" --member=serviceAccount:"${SA_EMAIL}" \
 4--role=roles/container.admin --role=roles/compute.networkAdmin --role=roles/iam.serviceAccountUser
 5Updated IAM policy for project [<project>].
 7- members:
 8  - serviceAccount:crossplane@<project>
 9  role: roles/compute.networkAdmin
10- members:
11  - serviceAccount:crossplane@<project>
13version: 1

Download the service account key (json format)

1gcloud iam service-accounts keys create crossplane.json --iam-account ${SA_EMAIL}
2created key [ea2eb9ce2939127xxxxxxxxxx] of type [json] as [crossplane.json] for [crossplane@<project>]

🚧 Deploy and configure Crossplane

Now that we have a credentials file for Google Cloud, we can deploy the Crossplane operator and configure the provider-gcp provider.


Most of the following steps are issued from the official documentation

We'll first use Helm in order to install the operator

 1helm repo add crossplane-master
 2"crossplane-master" has been added to your repositories
 4helm repo update
 5...Successfully got an update from the "crossplane-master" chart repository
 7helm install crossplane --namespace crossplane-system --create-namespace \
 8--version 1.18.1 crossplane-stable/crossplane
10NAME: crossplane
11LAST DEPLOYED: Mon Jun  6 22:00:02 2022
12NAMESPACE: crossplane-system
13STATUS: deployed
17Release: crossplane

Check that the operator is running properly.

1kubectl get po -n crossplane-system
2NAME                                       READY   STATUS    RESTARTS   AGE
3crossplane-rbac-manager-54d96cd559-222hc   1/1     Running   0          3m37s
4crossplane-688c575476-lgklq                1/1     Running   0          3m37s

All the files used for the upcoming steps are stored within this blog repository. So you should clone and change the current directory:

1git clone

Now we'll configure Crossplane so that it will be able to create and manage GCP resources. This is done by configuring the provider provider-gcp as follows.


2kind: Provider
4  name: crossplane-provider-gcp
6  package: crossplane/provider-gcp:v0.21.0
1kubectl apply -f provider.yaml created
4kubectl get providers
5NAME                      INSTALLED   HEALTHY   PACKAGE                           AGE
6crossplane-provider-gcp   True        True      crossplane/provider-gcp:v0.21.0   10s

Create the Kubernetes secret that holds the GCP credentials file created above

1kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./crossplane.json
2secret/gcp-creds created

Then we need to create a resource named ProviderConfig and reference the newly created secret.


 2kind: ProviderConfig
 4  name: default
 6  projectID: ${GCP_PROJECT}
 7  credentials:
 8    source: Secret
 9    secretRef:
10      namespace: crossplane-system
11      name: gcp-creds
12      key: creds
1kubectl apply -f provider-config.yaml created

If the serviceaccount has the proper permissions we can create resources in GCP. In order to learn about all the available resources and parameters we can have a look to the provider's API reference.

The first resource we'll create is the network that will host our Kubernetes cluster.


 2kind: Network
 4  name: dev-network
 5  labels:
 6    service: vpc
 7    creation: crossplane
 9  forProvider:
10    autoCreateSubnetworks: false
11    description: "Network used for experimentations and POCs"
12    routingConfig:
13      routingMode: REGIONAL
1kubectl get network
3dev-network   True    True

You can even get more details by describing this resource. For instance if something fails you would see the message returned by the Cloud provider in the events.

 1kubectl describe network dev-network | grep -A 20 '^Status:'
 3  At Provider:
 4    Creation Timestamp:  2022-06-28T09:45:30.703-07:00
 5    Id:                  3005424280727359173
 6    Self Link: ${GCP_PROJECT}/global/networks/dev-network
 7  Conditions:
 8    Last Transition Time:  2022-06-28T16:45:31Z
 9    Reason:                Available
10    Status:                True
11    Type:                  Ready
12    Last Transition Time:  2022-06-30T16:36:59Z
13    Reason:                ReconcileSuccess
14    Status:                True
15    Type:                  Synced

🚀 Create a GKE cluster

Everything is ready so that we can create our GKE cluster. Applying the file cluster.yaml will create a cluster and attach a node group to it.


 3kind: Cluster
 5  name: dev-cluster
 7  forProvider:
 8    description: "Kubernetes cluster for experimentations and POCs"
 9    initialClusterVersion: "1.24"
10    releaseChannel:
11      channel: "RAPID"
12    location: europe-west9-a
13    addonsConfig:
14      gcePersistentDiskCsiDriverConfig:
15        enabled: true
16      networkPolicyConfig:
17        disabled: false
18    networkRef:
19      name: dev-network
20    ipAllocationPolicy:
21      createSubnetwork: true
22      useIpAliases: true
23    defaultMaxPodsConstraint:
24      maxPodsPerNode: 110
25    networkPolicy:
26      enabled: false
27  writeConnectionSecretToRef:
28    namespace: default
29    name: gke-conn
32kind: NodePool
34  name: main-np
36  forProvider:
37    initialNodeCount: 1
38    autoscaling:
39      autoprovisioned: false
40      enabled: true
41      maxNodeCount: 4
42      minNodeCount: 1
43    clusterRef:
44      name: dev-cluster
45    config:
46      machineType: n2-standard-2
47      diskSizeGb: 120
48      diskType: pd-standard
49      imageType: cos_containerd
50      preemptible: true
51      labels:
52        environment: dev
53        managed-by: crossplane
54      oauthScopes:
55        - ""
56        - ""
57        - ""
58        - ""
59        - ""
60        - ""
61      metadata:
62        disable-legacy-endpoints: "true"
63      shieldedInstanceConfig:
64        enableIntegrityMonitoring: true
65        enableSecureBoot: true
66    management:
67      autoRepair: true
68      autoUpgrade: true
69    maxPodsConstraint:
70      maxPodsPerNode: 60
71    locations:
72      - "europe-west9-a"
1kubectl apply -f cluster.yaml created created

Note that it takes around 10 minutes for the Kubernetes API and the nodes to be available. The STATE will transition from PROVISIONING to RUNNING and when a change is being applied the cluster status is RECONCILING

1watch 'kubectl get cluster,nodepool'
2NAME                                              READY   SYNCED   STATE          ENDPOINT       LOCATION         AGE   False   True     PROVISIONING   europe-west9-a   3m15s
5NAME                                           READY   SYNCED   STATE   CLUSTER-REF   AGE   False   False            dev-cluster   3m15s


When the column READY switches to True you can download the cluster's credentials.

1kubectl get cluster
2NAME          READY   SYNCED   STATE         ENDPOINT         LOCATION         AGE
3dev-cluster   True    True     RECONCILING      europe-west9-a   6m23s
5gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project ${GCP_PROJECT}
6Fetching cluster endpoint and auth data.
7kubeconfig entry generated for dev-cluster.

For better readability you may want to rename the context id for the newly created cluster

1kubectl config rename-context gke_${GCP_PROJECT}_europe-west9-a_dev-cluster dev-cluster
2Context "gke_${GCP_PROJECT}_europe-west9-a_dev-cluster" renamed to "dev-cluster".
4kubectl config get-contexts
5CURRENT   NAME             CLUSTER                                                       AUTHINFO                                                      NAMESPACE
6*         dev-cluster      gke_cloud-native-computing-paris_europe-west9-a_dev-cluster   gke_cloud-native-computing-paris_europe-west9-a_dev-cluster
7          k3d-crossplane   k3d-crossplane                                                admin@k3d-crossplane

Check that you can call our brand new GKE API

1kubectl get nodes
2NAME                                    STATUS   ROLES    AGE   VERSION
3gke-dev-cluster-main-np-d0d978f9-5fc0   Ready    <none>   10m   v1.24.1-gke.1400

That's great 🎉 we know have a GKE cluster up and running.

💭 final thoughts

I've been using Crossplane for a few months now in a production environment.

Even if I'm conviced about the declarative approach using the Kubernetes API, we decided to move with caution with it. It clearly doesn't have Terraform's community and maturity. We're still declaring our resources using the deletionPolicy: Orphan so that even if something goes wrong on the controller side the resource won't be deleted.

Furthermore we limited to a specific list of usual AWS resources requested by our developers. Nevertheless our target has always been to empower developers and we had really positive feedback from them. That's the best indicator for us. As the project matures, we'll move more and more resources from Terraform to Crossplane.

IMHO the key success of Crossplane depends on the providers maintenance and evolution. The Cloud providers interest and involvement is really important.

In our next article we'll see how to use a GitOps engine to run all the above steps.