Microgateway configuration in sidecar mode

Configuration architecture in sidecar mode

Within the Kubernetes cluster, the Microgateway Operator container injects and operates the Airlock Microgateway containers in the Web application Pod. This section describes how these pieces are connected together and what exactly is modified and done by the Microgateway Operator. It also covers the network configuration task of the Microgateway CNI plugin, which configures the network routing through Microgateway Engine sidecars within Web application Pods.

The following example illustrates the setup when no service mesh is used. The setup shows two namespaces, airlock-microgateway-system for the Microgateway Operator Pod and a second one for the Web application Pod.

Airlock Microgateway configuration concept
  1. Initial Microgateway Operator configuration
  2. The Microgateway Operator reads its configuration from the Microgateway Operator Configuration at startup. The configuration contains settings like the templates used to inject the Microgateway containers.
  3. The Microgateway Operator watches for events such as Pod create, update, or delete.
  1. Steps to secure a Web application Pod in the namespace X:
  2. Create a Custom Resource ContentSecurity in the namespace of the Web application with the required references to additional CRs such as DenyRules.
  3. Create a second Custom Resource of the kind SidecarGateway in the Web application's namespace and refer to the Custom Resource ContentSecurity created previously.
  4. Configure the container port in the Custom Resource SidecarGateway, which should be protected. Ensure that it is the same port as in the Pod specification.
  5. Configure the podSelector to match the Web application Pods, which should be protected in the Custom Resource SidecarGateway. Ensure that it corresponds to the labels in the Pod specification.
  6. Label the Web application Pod with sidecar.microgateway.airlock.com/inject: "true".
  7. The Microgateway Operator and the Microgateway CNI plugin are triggered by the creation of the Pod with the sidecar label previously.
    • The Operator:
    • Injects the sidecar container Microgateway Engine with the bootstrap configuration.
    • The Microgateway CNI plugin:
    • Configures the network inside the Web application Pod, so that all traffic is routed through the Microgateway Engine sidecar.
  8. The Microgateway Engine subscribes to the configuration from the Microgateway Operator. The Microgateway Operator generates the configuration based on the Custom Resources such as SidecarGateway and the referenced resources.
  • The example describes the most simple setup. Although not outlined explicitly, the following things are possible:
  • Create a Kubernetes Deployment, StatefulSet, or DaemonSet and annotate the Pod with Microgateway annotations in spec.template.metadata.annotations and set the inject label.
  • Configure other Microgateway annotations described in this manual.

Microgateway Operator

The ​Microgateway Operator​ container injects the ​Microgateway Engine​ Container into web application Pods labeled with sidecar.microgateway.airlock.com/inject: "true". In addition, the Microgateway Operator monitors these pods and reconfigures them whenever a Custom Resource changes.

Helm chart configuration

Helm Microgateway Operator chart configuration must take place before or with installation or upgrade. The configuration can be persistently adjusted in a local values.yaml file or passed with every Helm installation using the --set option.

Configuration change

Description

Installation mode types

Airlock Microgateway can be installed in different cluster- and namespace-scoped modes as described in the architecture article Sidecar-based mode installation types.

The installation modes can be configured via the Microgateway Operator Helm chart by setting the operator.watchNamespaces and operator.watchNamespaceSelector values.

The following example adds the namespace airlock-mgw-system to the watchNamespaces list for an OwnNamespace installation:

copy
helm install airlock-microgateway -n airlock-microgateway-system oci://quay.io/airlockcharts/microgateway \​ 
    --set operator.watchNamespaces[0]=airlock-microgateway-system --wait --version 4.5.0

Grafana dashboards

Airlock Microgateway offers several preconfigured Helm-based Grafana dashboard templates that can be enabled and individually disabled in the Operator Helm chart.

The following example makes the dashboard templates available for Grafana:

copy
helm install airlock-microgateway -n airlock-microgateway-system oci://quay.io/airlockcharts/microgateway \​ 
   --set dashboards.create=true --version 4.5.0 --wait​ 
  • Only one Microgateway Operator per cluster may be configured to provide the Grafana dashboard templates.
  • Grafana must be deployed with sidecar.dashboards.enabled=true to enable the cluster-wide search for the dashboard templates.

TLS certificate generation and renewal

Any communication between containers in the airlock-microgateway-system namespace and, i.e., Microgateway Engine containers in web application namespaces, is secured using TLS/mTLS.

During the Microgateway Operator startup, the following self-signed certificates are generated and stored as secrets in the airlock-microgateway-system namespace:

  • airlock-microgateway-ca-cert – the CA certificate to generate self-signed TLS certificates.
  • webhook-server-cert – the server certificate of the Microgateway Operator.

Each time an Airlock Microgateway Engine sidecar is injected, an airlock-microgateway-bootstrap-secret is generated and saved to the web application namespace. This secret holds the required certificates and keys for mTLS-based communication between the Microgateway Engine and the Operator. The TLS certificates of the bootstrap secret are renewed automatically every 48h.

Labels and annotations in sidecar mode

The Microgateway Operator injects the Microgateway Engine containers to the upstream application Pods triggered by the label following label:

copy
sidecar.microgateway.airlock.com/inject: "true"

See also example manifests w/o Istio mesh configuration at the end of this article.

Adding the label to an existing, running Pod has no effect. Use kubectl apply -f <deployment.yaml> to apply the configuration changes and to have the Microgateway Engine injected with a Deployment.

Annotations for the Microgateway Operator

Annotations can be added to configure some general behavior of the container. The table below lists and explains the annotations the Microgateway Operator is aware of.

Annotation

Default

Possible values

Description

sidecar.microgateway.airlock.com/excludeInboundPorts

""

"<int>, ..."

Comma-separated list of inbound ports to be excluded from redirection to the Microgateway Engine by the Microgateway CNI plugin.

The Istio default ports for the Istio sidecar proxy are automatically excluded when setting the annotation sidecar.microgateway.airlock.com/serviceMesh: istio.

  • This means ports:
  • 15000, 15001
  • 15004
  • 15006
  • 15008, 15009
  • 15020, 15021
  • 15053
  • 15090

sidecar.microgateway.airlock.com/serviceMesh

none

none, istio

Annotation to set which service mesh should be used. This annotation is relevant info for the Microgateway CNI plugin to configure the redirection of the traffic to the Microgateway Engine correctly.

sidecar.microgateway.airlock.com/engineLogLevel

info

trace, debug, info, warn, error, critical, off

Annotation to set the Microgateway Engine log level.

sidecar.microgateway.airlock.com/engineCpuRequests

By default, no engine resource requirements are set.

Engine resources can be set via annotation or Helm values. If both are set, annotations take precedence.

1, 100m, 0.5, ...

Overrides resources.requests.cpu for the Microgateway engine sidecar container.

sidecar.microgateway.airlock.com/engineCpuLimits

By default, no engine resource requirements are set.

Engine resources can be set via annotation or Helm values. If both are set, annotations take precedence.

1, 100m, 0.5, ...

Any CPU resource unit can be used.

Overrides resources.limits.cpu for the Microgateway engine sidecar container.

sidecar.microgateway.airlock.com/engineMemoryRequests

By default, no engine resource requirements are set.

Engine resources can be set via annotation or Helm values. If both are set, annotations take precedence.

64Mi, 1G, ...

Any Kubernetes memory resource unit can be used.

Overrides resources.requests.memory for the Microgateway engine sidecar container.

sidecar.microgateway.airlock.com/engineMemoryLimits

By default, no engine resource requirements are set.

Engine resources can be set via annotation or Helm values. If both are set, annotations take precedence.

128Mi, 1G, ...

Any Kubernetes memory resource unit can be used.

Overrides resources.limits.memory for the Microgateway engine sidecar container.

Example configuration for a web application without Istio

The following example Kubernetes manifest file contains annotations to protect the web application with Airlock Microgateway.

copy
apiVersion: apps/v1  
kind: Deployment  
metadata:  
  labels:  
    app: echo-server  
  name: echo-server  
spec:  
  selector:  
    matchLabels:  
      app: echo-server  
  template:  
    metadata:  
      annotations:  
        sidecar.microgateway.airlock.com/excludeInboundPorts: "9998,9999" 
        sidecar.microgateway.airlock.com/serviceMesh: "none" 
       labels:  
        sidecar.microgateway.airlock.com/inject: "true"  
        app: echo-server  
        version: 1.0.0
    spec: 
      containers: 
        - image: ealen/echo-server:latest 
          name: echo-server 
          ports: 
            - containerPort: 5051

Ensure that in such a setup the annotation sidecar.microgateway.airlock.com/serviceMesh is set to none or not set at all.

  • SidecarGateway configuration to protect a web application:
  • Our example uses a simple web application Pod with the label app: example-app.
  • The spec.applications.containerPort is the container port of the web application that has to be secured by Airlock Microgateway. The default 8080 can be overridden to connect to the application's HTTP port. In our example, the example-app Pod listens for requests on port 5051.
  • The security settings to be applied are referenced in the CR ContentSecurity named content-security-example.

This setup is still possible if Istio is deployed in your Kubernetes cluster. The important thing is, that the web application Pod does not have Istio injected. Neither through labeling the namespace nor with the corresponding annotation. For more information consult (Istio) Sidecar injection.

Example for a web application with Istio

The following example Kubernetes manifest file contains annotations to protect the web application with Airlock Microgateway alongside Istio service mesh:

copy
apiVersion: apps/v1  
kind: Deployment  
metadata:  
  labels:  
    app: echo-server  
  name: echo-server  
spec:  
  selector:  
    matchLabels:  
      app: echo-server  
  template:  
    metadata:  
      annotations:  
        sidecar.microgateway.airlock.com/excludeInboundPorts: "9998,9999" 
        sidecar.microgateway.airlock.com/serviceMesh: "istio" 
        sidecar.istio.io/inject: "true"  
      labels:  
        sidecar.microgateway.airlock.com/inject: "true"  
        app: echo-server  
        version: 1.0.0
    spec: 
      containers: 
        - image: ealen/echo-server:latest 
          name: echo-server 
          ports: 
            - containerPort: 5051

Ensure that in such a setup the annotation sidecar.microgateway.airlock.com/serviceMesh is set to istio.

  • SidecarGateway configuration to protect a web application:
  • Our example uses a simple web application Pod with the label app: example-app.
  • The spec.applications.containerPort is the container port of the web application that has to be secured by Airlock Microgateway. The default 8080 can be overridden to connect to the application's HTTP port. In our example, the example-app Pod listens for requests on port 5051.
  • The security settings to be applied are referenced in the CR ContentSecurity named content-security-example.

The web application Pod has Istio injected. The example illustrates the way by annotating the Pod. For more information consult (Istio) Sidecar injection.

Sidecar injection and configuration

After labeling a Pod with sidecar.microgateway.airlock.com/inject: "true", the Microgateway Operator injects the Microgateway Engine and Session Agent container into the labeled Pod. The Microgateway Engine is injected as a sidecar in the protected Pod to secure them. It is built based on the Envoy proxy and enriched with Airlock-specific features.

  1. The Microgateway Engine configuration is divided into several parts:
  2. The Engine container configuration. It may be configured in the Microgateway Operator Helm chart, e.g. when using a custom registry or for setting resource limits.
  3. The Envoy bootstrap configuration. It should normally not be modified.
  4. The web application-specific configuration.

To configure the Microgateway Engine, we created the required Custom Resource Definitions. See also the Airlock Microgateway API reference documentation detailed information, such as example configurations and default settings of the CRs. When integrating or hardening a web application, only the settings within the CRs are required.

Microgateway CNI

The Microgateway CNI DaemonSet should be installed in the kube-system Namespace. This should be done using our CNI Helm chart. See installation-related articles for environment-depending installation.

The CNI Helm chart also installs a ServiceAccount with the ClusterRole, ClusterRoleBinding and a ConfigMap.
OpenShift requires to install Role and RoleBinding used for SCC and a NetworkAttachmentDefinition and adapting the k8s.v1.cni.cncf.io/networks annotation to allow the CNI plugin to work together with OpenShift Multus as described in the article Installation-related instructions.

After the installation, the airlock-microgateway-cni DaemonSet is installed in the kube-system Namespace and will deploy one Microgateway CNI plugin to every Node. Once a Microgateway CNI plugin is installed on a Node, it handles the network configuration inside containers with Microgateway Engine Pods every time an Engine is created.

Customizing the installation

The default installation values like the image registry and image name can be looked up and modified to your needs in the values.yaml file of the CNI Helm chart. For detailed information about the default configuration values and their meanings, see the explanations in the values.yaml file.

  • Ensure that the replaced Airlock Microgateway images are always specified in the Kubernetes manifest files with a tag and a digest.

Microgateway Session Agent

The Microgateway Session Agent is automatically injected as a sidecar with the Microgateway Engine sidecar. The Session Agent handles the session store connection to the Redis database, which caches the session information. Access control requires session handling, including a Redis deployment ( as a session store. Configuration is done in the CRDs CR RedisProvider and CR SessionHandling.