Update Istio configuration to use CNI node agents instead of istio-init containers

[MaciejKaras]

Summary

We are using Istio as a service mesh provider for our Multi Cluster tests. The way it works by default is Istio adds privileged init-istio container to every Pod that configures network accordingly.

By default Istio injects an init container, istio-init, in pods deployed in the mesh. The istio-init container sets up the pod network traffic redirection to/from the Istio sidecar proxy. This requires the user or service-account deploying pods to the mesh to have sufficient Kubernetes RBAC permissions to deploy containers with the NET_ADMIN and NET_RAW capabilities.

While this works fine it is not meeting the PSS restricted level, thus making it less secure. Related HELP-81729 and #473 that enables restricted level in warn mode. Additionally we provide Istio sidecar configuration as an example in our code snippets thus not following the best practice.

There is another way to configure Istio mesh that does not require istio-init init-container - using Istio CNI node agent. This PR configures our e2e tests and code snippets that way. Great blog entry about difference between istio-init and Istio CNI node agent architecture -> https://www.solo.io/blog/traffic-ambient-mesh-istio-cni-node-configuration.

With istio-init:

With Istio CNI node agent:

⚠️ Init containers execute before the sidecar proxy starts, which can result in traffic loss during their execution. This can be avoided by setting runAsUser: 1337. More info -> https://istio.io/latest/docs/setup/additional-setup/cni/#compatibility-with-application-init-containers

Proof of Work

Passing CI is enough. Since private_gke_code_snippets are not run automatically in CI I've triggered manual patch to test this -> https://spruce.mongodb.com/version/68d50e694baed3000742566d/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC

Checklist

• Have you linked a jira ticket and/or is the ticket in the title?
• Have you checked whether your jira ticket required DOCSP changes?
• Have you added changelog file?
  • use skip-changelog label if not needed
  • refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

[github-actions]

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.6.0 Release Notes

New Features

• MongoDBCommunity: Added support to configure custom cluster domain via newly introduced spec.clusterDomain resource field. If spec.clusterDomain is not set, environment variable CLUSTER_DOMAIN is used as cluster domain. If the environment variable CLUSTER_DOMAIN is also not set, operator falls back to cluster.local as default cluster domain.
• Helm Chart: Introduced two new helm fields operator.podSecurityContext and operator.securityContext that can be used to configure securityContext for Operator deployment through Helm Chart.

Bug Fixes

• Fixed parsing of the customEnvVars Helm value when values contain = characters.

[MaciejKaras]

This file was unused

[mircea-cosbuc]

q: Was this accidentally commited into the scripts/release/kubectl-mongodb folder?

[MaciejKaras]

yes, exactly. This was added mistakenly added during kubectl-mongodb plugin refactoring

[mircea-cosbuc]

This is great if it passes tests. It's currently failing a couple.

[lsierant]

LGTM!