With Service Discovery


openshift-install and pull-secret

Download the openshift-install and oc tools, and copy your pull secret from:


Find more detailed instructions here:


Make sure the aws cli is properly installed and configured

Installation instructions


$ aws configure
AWS Access Key ID [None]: ....
AWS Secret Access Key [None]: ....
Default region name [None]: ....
Default output format [None]: text

See also for more details:


Create and deploy cluster A

In this step you will deploy cluster A, with the default IP CIDRs

Pod CIDR Service CIDR
openshift-install create install-config --dir cluster-a
openshift-install create cluster --dir cluster-a

The create cluster step will take some time, you can create Cluster B in parallel if you wish.

Create and deploy cluster B

In this step you will deploy cluster B, modifying the default IP CIDRs

Pod CIDR Service CIDR
openshift-install create install-config --dir cluster-b

Change the POD IP network, please note it’s a /14 range by default so you need to use +4 increments for “128”, for example:,,, …

sed -i 's/' cluster-b/install-config.yaml

Change the service IP network, this is a /16 range by default, so just use +1 increments for “30”: for example:,,, …

sed -i 's/' cluster-b/install-config.yaml

And finally deploy

openshift-install create cluster --dir cluster-b

Make your clusters ready for submariner

Submariner gateway nodes need to be able to accept traffic over ports 4500/UDP and 500/UDP when using IPSEC. In addition we use port 4800/UDP to encapsulate traffic from the worker nodes to the gateway nodes and ensuring that Pod IP addresses are preserved.

Additionally, the default Openshift deployments don’t allow assigning an elastic public IP to existing worker nodes, something that it’s necessary at least on one end of the IPSEC connections.

To handle all those details we provide a script that will prepare your AWS OpenShift deployment for submariner, and will create an additional gateway node with an external IP.

curl https://raw.githubusercontent.com/submariner-io/submariner/master/tools/openshift/ocp-ipi-aws/prep_for_subm.sh -L -O
chmod a+x ./prep_for_subm.sh

./prep_for_subm.sh cluster-a     # respond yes when terraform asks
./prep_for_subm.sh cluster-b      # respond yes when terraform asks

INFO Please note that oc, aws-cli, terraform, and unzip need to be installed before running the prep_for_subm.sh script.

Install subctl

Download the subctl binary and make it available on your PATH.

curl -Ls https://raw.githubusercontent.com/submariner-io/submariner-operator/master/scripts/subctl/getsubctl.sh | bash
export PATH=$PATH:~/.local/bin
echo export PATH=\$PATH:~/.local/bin >> ~/.profile

Install Submariner with Service Discovery

To install Submariner with multi-cluster service discovery follow the steps below.

Use cluster-a as broker with service discovery enabled

subctl deploy-broker --kubeconfig cluster-a/auth/kubeconfig --service-discovery

Join cluster-a and cluster-b to the broker

subctl join --kubeconfig cluster-a/auth/kubeconfig broker-info.subm --clusterid cluster-a
subctl join --kubeconfig cluster-b/auth/kubeconfig broker-info.subm --clusterid cluster-b

Verify Deployment

To verify the deployment follow the steps below which creates an nginx service and ServiceExport for it.

export KUBECONFIG=cluster-b/auth/kubeconfig
kubectl -n default create deployment nginx --image=nginxinc/nginx-unprivileged:stable-alpine
kubectl -n default expose deployment nginx --port=8080
subctl export service --namespace default nginx
export KUBECONFIG=cluster-a/auth/kubeconfig
kubectl -n default  run --generator=run-pod/v1 tmp-shell --rm -i --tty --image quay.io/submariner/nettest -- /bin/bash
curl nginx.default.svc.supercluster.local:8080

Perform automated verification

This will perform all automated verification between your clusters

subctl verify cluster-a/auth/kubeconfig cluster-b/auth/kubeconfig --only service-discovery,connectivity --verbose