Sandbox Environment (kind)

Deploy kind with Submariner Locally

kind is a tool for running local Kubernetes clusters using Docker container nodes. This guide uses kind to demonstrate deployment and operation of Submariner in three Kubernetes clusters running locally on your computer.

Submariner provides automation to deploy clusters using kind and connect them using Submariner.

Prerequisites

  1. Install Docker and ensure it is running properly on your computer.
  2. Install and set up kubectl.

You may need to increase your inotify resource limits.

Deploy Automatically

To create kind clusters and deploy Submariner with service discovery enabled, run:

git clone https://github.com/submariner-io/submariner-operator
cd submariner-operator
make deploy using=lighthouse

To deploy IPv4/IPv6 dual-stack Kubernetes clusters, set using=dual-stack.

By default, the automation configuration in the submariner-io/submariner-operator repository deploys two clusters, with cluster1 configured as the Broker. See the settings file for details.

Once you become familiar with Submariner’s basics, you may want to visit the Building and Testing page to learn more about customizing your Submariner development deployment. To understand how Submariner’s development deployment infrastructure works under the hood, see Deployment Customization in the Shipyard documentation.

Deploy Manually

If you wish to try out Submariner deployment manually, an easy option is to create kind clusters using our scripts and deploy Submariner with subctl.

Create kind Clusters

To create kind clusters, run:

git clone https://github.com/submariner-io/submariner-operator
cd submariner-operator
make clusters

Once the clusters are deployed, make clusters will indicate how to access them:

Your virtual cluster(s) are deployed and working properly and can be accessed with:

export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)

$ kubectl config use-context cluster1 # or cluster2, cluster3..

To clean everthing up, just run: make clean-clusters

The export KUBECONFIG command has to be run before kubectl can be used.

make clusters creates two Kubernetes clusters: cluster1 and cluster2. To see the list of kind clusters, use the following command:

$ kind get clusters
cluster1
cluster2

To list the local Kubernetes contexts, use the following command:

$ kubectl config get-contexts
CURRENT   NAME       CLUSTER    AUTHINFO   NAMESPACE
          cluster1   cluster1   cluster1      
*         cluster2   cluster2   cluster2

Since multiple clusters are running, you need to choose which cluster kubectl talks to. You can set a default cluster for kubectl by setting the current context in the Kubernetes kubeconfig file. Additionally, you can run the following command to set the current context for kubectl:

kubectl config use-context <cluster name>

For more information on interacting with kind, please refer to the kind documentation.

Install subctl

Download the subctl binary and make it available on your PATH.

curl -Ls https://get.submariner.io | bash
export PATH=$PATH:~/.local/bin
echo export PATH=\$PATH:~/.local/bin >> ~/.profile

If you have Go and the source code, you can build and install subctl instead:

cd go/src/submariner-io/subctl
go build -o $GOBIN/subctl github.com/submariner-io/subctl/cmd

(and ensure your go/bin directory is on your PATH).

Use cluster1 as Broker

subctl deploy-broker --kubeconfig output/kubeconfigs/kind-config-cluster1

Join cluster1 and cluster2 to the Broker

subctl join --kubeconfig output/kubeconfigs/kind-config-cluster1 broker-info.subm --clusterid cluster1 --natt=false

subctl join --kubeconfig output/kubeconfigs/kind-config-cluster2 broker-info.subm --clusterid cluster2 --natt=false

You now have a Submariner environment that you can experiment with.

Verify Deployment

Verify Automatically with subctl

This will perform automated verifications between the clusters.

export KUBECONFIG=output/kubeconfigs/kind-config-cluster1:output/kubeconfigs/kind-config-cluster2
subctl verify --context cluster1 --tocontext cluster2 --only service-discovery,connectivity --verbose

Verify Manually

To manually verify the deployment, follow the steps below using either a headless or ClusterIP nginx service deployed in cluster2.

Deploy ClusterIP Service
kubectl --kubeconfig output/kubeconfigs/kind-config-cluster2 create deployment nginx --image=nginx
kubectl --kubeconfig output/kubeconfigs/kind-config-cluster2 expose deployment nginx --port=80
subctl export service --kubeconfig output/kubeconfigs/kind-config-cluster2 --namespace default nginx
Deploy Headless Service
kubectl --kubeconfig output/kubeconfigs/kind-config-cluster2 create deployment nginx --image=nginx
kubectl --kubeconfig output/kubeconfigs/kind-config-cluster2 expose deployment nginx --port=80 --cluster-ip=None
subctl export service --kubeconfig output/kubeconfigs/kind-config-cluster2 --namespace default nginx
Verify

Run nettest from cluster1 to access the nginx service:

kubectl --kubeconfig output/kubeconfigs/kind-config-cluster1 -n default run tmp-shell --rm -i --tty --image quay.io/submariner/nettest \
-- /bin/bash
curl nginx.default.svc.clusterset.local

Cleanup

When you are done experimenting and you want to delete the clusters deployed in any of the previous steps, use the following command:

make clean-clusters