With Service Discovery and Globalnet

AWS

openshift-install and pull-secret

Download the openshift-install and oc tools, and copy your pull secret from:

https://cloud.redhat.com/openshift/install/aws/installer-provisioned

Find more detailed instructions here:

https://docs.openshift.com/container-platform/4.3/installing/installing_aws/installing-aws-default.html

Make sure the aws cli is properly installed and configured

Installation instructions

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

$ aws configure
AWS Access Key ID [None]: ....
AWS Secret Access Key [None]: ....
Default region name [None]: ....
Default output format [None]: text

See also for more details:

https://docs.openshift.com/container-platform/4.3/installing/installing_aws/installing-aws-account.html

Create cluster A

This step will create a cluster named “cluster-a” with the default IP CIDRs.

Pod CIDR Service CIDR
10.128.0.0/14 172.30.0.0/16
openshift-install create install-config --dir cluster-a
openshift-install create cluster --dir cluster-a

This may take some time to complete so you can move on to the next section in parallel if you wish.

Create cluster B

This step will create a cluster named “cluster-b” with the default IP CIDRs.

Pod CIDR Service CIDR
10.128.0.0/14 172.30.0.0/16
openshift-install create install-config --dir cluster-b

And finally deploy

openshift-install create cluster --dir cluster-b

Make your clusters ready for submariner

Submariner gateway nodes need to be able to accept traffic over ports 4500/UDP and 500/UDP when using IPSEC. In addition we use port 4800/UDP to encapsulate traffic from the worker nodes to the gateway nodes and ensuring that Pod IP addresses are preserved.

Additionally, the default Openshift deployments don’t allow assigning an elastic public IP to existing worker nodes, something that it’s necessary at least on one end of the IPSEC connections.

To handle all those details we provide a script that will prepare your AWS OpenShift deployment for submariner, and will create an additional gateway node with an external IP.


curl https://raw.githubusercontent.com/submariner-io/submariner/master/tools/openshift/ocp-ipi-aws/prep_for_subm.sh -L -O
chmod a+x ./prep_for_subm.sh

./prep_for_subm.sh cluster-a     # respond yes when terraform asks
./prep_for_subm.sh cluster-b      # respond yes when terraform asks

INFO Please note that oc, aws-cli, terraform, and unzip need to be installed before running the prep_for_subm.sh script.

Install subctl

Download the subctl binary and make it available on your PATH.

curl -Ls https://raw.githubusercontent.com/submariner-io/submariner-operator/master/scripts/subctl/getsubctl.sh | bash
export PATH=$PATH:~/.local/bin
echo export PATH=\$PATH:~/.local/bin >> ~/.profile

Install Submariner with Service Discovery and Globalnet

To install Submariner with multi-cluster service discovery and support for overlapping CIDRs follow the steps below.

Use cluster-a as broker with service discovery and globalnet enabled

subctl deploy-broker  --kubeconfig cluster-a/auth/kubeconfig --service-discovery --globalnet

Join cluster-a and cluster-b to the broker

subctl join --kubeconfig cluster-a/auth/kubeconfig broker-info.subm --clusterid west
subctl join --kubeconfig cluster-b/auth/kubeconfig broker-info.subm --clusterid east

Verify Deployment

To verify the deployment follow the steps below which creates an nginx service and ServiceExport for it.

export KUBECONFIG=cluster-b/auth/kubeconfig
kubectl -n default create deployment nginx --image=nginxinc/nginx-unprivileged:stable-alpine
kubectl -n default expose deployment nginx --port=8080
subctl export service --namespace default nginx
export KUBECONFIG=cluster-a/auth/kubeconfig
kubectl -n default  run --generator=run-pod/v1 tmp-shell --rm -i --tty --image quay.io/submariner/nettest -- /bin/bash
curl nginx.default.svc.supercluster.local:8080

Perform automated verification

This will perform all automated verification between your clusters

subctl verify cluster-a/auth/kubeconfig cluster-b/auth/kubeconfig --only service-discovery,connectivity --verbose