Google Kubernetes Engine (GKE)

Prerequisites

You must have a Google Kubernetes Engine (GKE) cluster that has Helm configured. If you have not installed the Helm client (helm), see Installing Helm.

The YugabyteDB Helm chart has been tested with the following software versions:

GKE running Kubernetes 1.20 or later. The Helm chart you use to install YugabyteDB creates three YB-Master and three YB-TServers, each with 2 CPU cores, for a total of 12 CPU cores. This means you need a Kubernetes cluster with more than 12 CPU cores. If the cluster contains three nodes, then each node should have more than 4 cores.
Helm 3.4 or later.
For optimal performance, ensure you set the appropriate system limits using ulimit on each node in your Kubernetes cluster.

The following steps show how to meet these prerequisites:

Download and install the Google Cloud SDK.
Configure defaults for Google Cloud.

Execute the following command to set the project ID to yugabyte. You can change this as needed.
```
gcloud config set project yugabyte
```
Execute the following command to set the default compute zone to us-west1-b. You can change this as needed.
```
gcloud config set compute/zone us-west1-b
```
Install kubectl. Refer to kubectl installation instructions for your operating system.

Note that GKE is usually two or three major releases behind the upstream or OSS Kubernetes release. This means you have to make sure that you have the latest kubectl version that is compatible across different Kubernetes distributions.
Ensure that helm is installed.

First, check the Helm version, as follows:
```
helm version
```
Expect to see the output similar to the following. Note that the tiller server-side component has been removed in Helm 3.
```
version.BuildInfo{Version:"v3.0.3", GitCommit:"ac925eb7279f4a6955df663a0128044a8a6b7593", GitTreeState:"clean", GoVersion:"go1.13.6"}
```

Create a GKE cluster

Create a private Kubernetes cluster by running the following command.

gcloud container clusters create cluster_name --enable-private-nodes --machine-type=n1-standard-8

Note that you must set up Cloud NAT for a private Kubernetes cluster in Google Cloud to ensure that your cluster can access the internet while its nodes do not have public IP addresses. Refer to Configuring Private Google Access and Cloud NAT in Google Cloud Platform (GCP).

As stated in Prerequisites, the default configuration in the YugabyteDB Helm chart requires Kubernetes nodes to have a total of 12 CPU cores and 45 GB RAM allocated to YugabyteDB. This can be three nodes with 4 CPU cores and 15 GB RAM allocated to YugabyteDB. The smallest Google Cloud machine type that meets this requirement is n1-standard-8 which has 8 CPU cores and 30GB RAM.

Create a YugabyteDB cluster

Creating a YugabyteDB cluster involves a number of steps.

Add charts repository

To add the YugabyteDB charts repository, run the following command:

helm repo add yugabytedb https://charts.yugabyte.com

Fetch updates from the repository

Make sure that you have the latest updates to the repository by running the following command:

helm repo update

Validate the Chart version

Execute the following command:

helm search repo yugabytedb/yugabyte --version 2.23.1

Expect the following output:

NAME                 CHART VERSION  APP VERSION   DESCRIPTION
yugabytedb/yugabyte  2.23.1          2.23.1.0-b220  YugabyteDB is the high-performance distributed ...

Install YugabyteDB

Run the following commands to create a namespace and then install YugabyteDB:

kubectl create namespace yb-demo
helm install yb-demo yugabytedb/yugabyte --version 2.23.1 --namespace yb-demo --wait

Check the cluster status

You can check the status of the cluster using the following command:

helm status yb-demo -n yb-demo

NAME: yb-demo
LAST DEPLOYED: Thu Feb 13 13:29:13 2020
NAMESPACE: yb-demo
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get YugabyteDB Pods by running this command:
  kubectl --namespace yb-demo get pods

2. Get list of YugabyteDB services that are running:
  kubectl --namespace yb-demo get services

3. Get information about the load balancer services:
  kubectl get svc --namespace yb-demo

4. Connect to one of the tablet server:
  kubectl exec --namespace yb-demo -it yb-tserver-0 -- bash

5. Run YSQL shell from inside of a tablet server:
  kubectl exec --namespace yb-demo -it yb-tserver-0 -- ysqlsh -h yb-tserver-0.yb-tservers.yb-demo

6. Cleanup YugabyteDB Pods
  helm delete yb-demo --purge
  NOTE: You need to manually delete the persistent volume
  kubectl delete pvc --namespace yb-demo -l app=yb-master
  kubectl delete pvc --namespace yb-demo -l app=yb-tserver

Check the pods, as follows:

kubectl get pods --namespace yb-demo

NAME           READY     STATUS    RESTARTS   AGE
yb-master-0    1/1       Running   0          4m
yb-master-1    1/1       Running   0          4m
yb-master-2    1/1       Running   0          4m
yb-tserver-0   1/1       Running   0          4m
yb-tserver-1   1/1       Running   0          4m
yb-tserver-2   1/1       Running   0          4m

Check the services, as follows:

kubectl get services --namespace yb-demo

NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                                        AGE
yb-master-ui         LoadBalancer   10.109.39.242   35.225.153.213 7000:31920/TCP                                 10s
yb-masters           ClusterIP      None            <none>         7100/TCP,7000/TCP                              10s
yb-tserver-service   LoadBalancer   10.98.36.163    35.225.153.214 6379:30929/TCP,9042:30975/TCP,5433:30048/TCP   10s
yb-tservers          ClusterIP      None            <none>         7100/TCP,9000/TCP,6379/TCP,9042/TCP,5433/TCP   10s

You can even check the history of the yb-demo deployment, as follows:

helm history yb-demo -n yb-demo

REVISION  UPDATED                   STATUS    CHART           APP VERSION   DESCRIPTION
1         Tue Apr 21 17:29:01 2020  deployed  yugabyte-2.23.1  2.23.1.0-b220  Install complete

Connect using YugabyteDB shells

To connect and use the YSQL Shell ysqlsh, run the following command:

kubectl exec -n yb-demo -it yb-tserver-0 -- ysqlsh -h yb-tserver-0.yb-tservers.yb-demo

To connect and use the YCQL Shell ycqlsh, run the following command:

kubectl exec -n yb-demo -it yb-tserver-0 -- ycqlsh yb-tserver-0.yb-tservers.yb-demo

Connect using external clients

To connect an external program, get the load balancer EXTERNAL-IP address of the yb-tserver-service service and connect using port 5433 for YSQL or port 9042 for YCQL, as follows:

kubectl get services --namespace yb-demo

NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP        PORT(S)                                        AGE
...
yb-tserver-service   LoadBalancer   10.98.36.163    35.225.153.214     6379:30929/TCP,9042:30975/TCP,5433:30048/TCP   10s
...

Configure cluster

You can configure the cluster using the same commands and options that are described in Open Source Kubernetes.

Independent LoadBalancers

By default, the YugabyteDB Helm chart exposes the client API endpoints, as well as YB-Master UI endpoint using two LoadBalancers. To expose the client APIs using independent LoadBalancers, you can execute the following command:

helm install yb-demo yugabytedb/yugabyte -f https://raw.githubusercontent.com/yugabyte/charts/master/stable/yugabyte/expose-all.yaml --version 2.23.1 --namespace yb-demo --wait

You can also bring up an internal LoadBalancer (for either YB-Master or YB-TServer services), if required. You would need to specify the annotation required for your cloud provider. The following command brings up an internal LoadBalancer for the YB-TServer service in Google Cloud Platform:

helm install yugabyte -f https://raw.githubusercontent.com/yugabyte/charts/master/stable/yugabyte/expose-all.yaml --version 2.23.1 --namespace yb-demo --name yb-demo \
  --set annotations.tserver.loadbalancer."cloud\.google\.com/load-balancer-type"=Internal --wait