Uninstalling Submariner

Starting with Submariner 0.12, the recommended way to uninstall Submariner is via the subctl uninstall command. This will automatically remove Submariner and its components from a given cluster. For previous versions, Submariner would need to be uninstalled manually.

Automated Uninstall

Issue the subctl uninstall command against the cluster you want to uninstall Submariner from. Example output:

$ subctl uninstall --kubeconfig output/kubeconfigs/cluster1

? This will completely uninstall Submariner from the cluster cluster1. Are you sure you want to continue? Yes
 ✓ Checking if the connectivity component is installed
 ✓ The connectivity component is installed
 ✓ Deleting the Submariner resource - this may take some time
 ✓ Deleting the Submariner cluster roles and bindings
 ✓ Deleted the "submariner-diagnose" cluster role and binding
 ✓ Deleted the "submariner-gateway" cluster role and binding
 ✓ Deleted the "submariner-globalnet" cluster role and binding
 ✓ Deleted the "submariner-lighthouse-agent" cluster role and binding
 ✓ Deleted the "submariner-lighthouse-coredns" cluster role and binding
 ✓ Deleted the "submariner-networkplugin-syncer" cluster role and binding
 ✓ Deleted the "submariner-operator" cluster role and binding
 ✓ Deleted the "submariner-routeagent" cluster role and binding
 ✓ Deleting the Submariner namespace "submariner-operator"
 ✓ Deleting the broker namespace "submariner-k8s-broker"
 ✓ Deleting the Submariner custom resource definitions
 ✓ Deleted the "brokers.submariner.io" custom resource definition
 ✓ Deleted the "clusterglobalegressips.submariner.io" custom resource definition
 ✓ Deleted the "clusters.submariner.io" custom resource definition
 ✓ Deleted the "endpoints.submariner.io" custom resource definition
 ✓ Deleted the "gateways.submariner.io" custom resource definition
 ✓ Deleted the "globalegressips.submariner.io" custom resource definition
 ✓ Deleted the "globalingressips.submariner.io" custom resource definition
 ✓ Deleted the "servicediscoveries.submariner.io" custom resource definition
 ✓ Deleted the "submariners.submariner.io" custom resource definition
 ✓ Unlabeling gateway nodes

Manual Uninstall

To manually uninstall Submariner from a cluster, follow the steps below:

Note

Make sure KUBECONFIG for all participating clusters is exported and all participating clusters are accessible via kubectl.

Delete Submariner-related namespaces

For each participating cluster, issue the following command:
```
kubectl delete namespace submariner-operator
```
For the Broker cluster, issue the following command:
```
kubectl delete namespace submariner-k8s-broker
```
For submariner version 0.9 and above, also delete submariner-operator namespace from the Broker cluster by issuing the following command:
```
kubectl delete namespace submariner-operator
```

Delete the Submariner CRDs

For each participating cluster, issue the following command:

for CRD in `kubectl get crds | grep -iE 'submariner|multicluster.x-k8s.io'| awk '{print $1}'`; do kubectl delete crd $CRD; done

Delete Submariner’s ClusterRoles and ClusterRoleBindings

For each participating cluster, issue the following command:

roles="submariner-operator submariner-operator-globalnet submariner-lighthouse submariner-networkplugin-syncer"
kubectl delete clusterrole,clusterrolebinding $roles --ignore-not-found

Remove the Submariner gateway labels

For each participating cluster, issue the following command:
```
kubectl label --all node submariner.io/gateway-
```
For OpenShift deployments, delete Lighthouse entry from default DNS.

For each participating cluster, issue the following command:
```
kubectl apply -f -  <<EOF
apiVersion: operator.openshift.io/v1
kind: DNS
metadata:
  finalizers:
  - dns.operator.openshift.io/dns-controller
  name: default
spec:
  servers: []
EOF
```
This deletes the Lighthouse entry from the Data section in Corefile of the configmap.
```
#lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT
clusterset.local:53 {
    forward . 100.3.185.93
}
#lighthouse-end
```
Verify that the Lighthouse entry is deleted from Corefile of dns-default configmap by running following command on an OpenShift cluster
```
kubectl describe configmap dns-default -n openshift-dns
```
For Kubernetes deployments, manually edit the Corefile of coredns configmap and delete the Lighthouse entry by issuing below commands
```
kubectl edit cm coredns -n kube-system
```
This will also restart the coredns. Below command can also be issued to manually restart coredns.
```
kubectl rollout restart -n kube-system deployment/coredns
```
Verify that the Lighthouse entry is deleted from Data section in Corefile of dns-default config map by running following command on a Kubernetes cluster
```
kubectl describe configmap coredns -n kube-system
```
Note
Following commands need to be executed from inside the cluster nodes.

Remove Submariner’s iptables chains

On all nodes in each participating cluster, issue the following commands:

iptables --flush SUBMARINER-INPUT
iptables -D INPUT $(iptables -L INPUT --line-numbers | grep SUBMARINER-INPUT | awk '{print $1}')
iptables --delete-chain SUBMARINER-INPUT

iptables -t nat --flush SUBMARINER-POSTROUTING
iptables -t nat -D POSTROUTING $(iptables -t nat -L POSTROUTING --line-numbers | grep SUBMARINER-POSTROUTING | awk '{print $1}')
iptables -t nat --delete-chain SUBMARINER-POSTROUTING

iptables -t mangle --flush SUBMARINER-POSTROUTING
iptables -t mangle -D POSTROUTING $(iptables -t mangle -L POSTROUTING --line-numbers | grep SUBMARINER-POSTROUTING | awk '{print $1}')
iptables -t mangle --delete-chain SUBMARINER-POSTROUTING

ipset destroy SUBMARINER-LOCALCIDRS
ipset destroy SUBMARINER-REMOTECIDRS

If Globalnet is enabled in the setup, additionally issue the following commands on gateway nodes:

iptables -t nat --flush SUBMARINER-GN-INGRESS
iptables -t nat -D PREROUTING $(iptables -t nat -L PREROUTING --line-numbers | grep SUBMARINER-GN-INGRESS | awk '{print $1}')
iptables -t nat --delete-chain SUBMARINER-GN-INGRESS

iptables -t nat --flush SUBMARINER-GN-EGRESS
iptables -t nat --delete-chain SUBMARINER-GN-EGRESS

iptables -t nat -t nat --flush SUBMARINER-GN-MARK
iptables -t nat --delete-chain SUBMARINER-GN-MARK

Delete the vx-submariner interface

On all nodes in each participating cluster, issue the following command:
```
ip link delete vx-submariner
```

If Globalnet release 0.9 (or earlier) is enabled in the setup, issue the following commands to remove the annotations from all the Pods and Services.

For each participating cluster, issue the following command:

for ns in `kubectl get ns -o jsonpath="{.items[*].metadata.name}"`; do
    kubectl annotate pods -n $ns --all submariner.io/globalIp-
    kubectl annotate services -n $ns --all submariner.io/globalIp-
done