Troubleshooting Microgateway Operator Helm test

As of Airlock Microgateway 4.2, Airlock Microgateway Operator is installed via Helm. To verify whether the installation was successful, a helm test is integrated in the Airlock Microgateway Operator installation procedure (see also Install the Airlock Microgateway Operator). This article explains what to do when this helm test failed.

You can find the Airlock Microgateway Helm charts here:

The helm test of checks whether the previously installed Microgateway Operator is up and running as expected. The tested setup resembles a basic Airlock Microgateway environment, including the Microgateway Operator and a Web application Pod (it is assumed that the Microgateway CNI is already installed).

  1. The helm test includes the following steps:
  2. Cleaning up Web application Pods that were possibly leftover from previous (test) runs.
  3. Creating a Web application Pod for testing purposes.
  4. Checking that the Microgateway Operator injected the Microgateway Engine sidecar container into the Web application Pod. The injection should have been triggered by the previous step.
  5. Verifying the Airlock Microgateway license.
  6. Generating and applying a basic CustomResource SidecarGateway to the Web application Pod for testing purposes.
  7. Verifying that the Web application Pod and the Microgateway Engine are up and working.
  8. Sending a valid request and verifying that it correctly returns HTTP status code 200 OK.
  9. Sending a request with an injection attack and verifying that it is blocked returning HTTP status code 400 Bad Request.

On successful helm test and with Airlock Microgateway running as expected, the message Installation of airlock-microgateway.<full name> succeeded is displayed at the bottom of the test log. On failure, the log will contain a failure message, such as Timeout occurred, Pod not ready or Sidecar container not injected.

The failure message Microgateway license is missing or invalid may occur when upgrading to Airlock Microgateway 4.2 due to a change of the license format. If your current license is incompatible with the new format, update the license. For more information, go to Configuration and monitoring of licenses.
The failure message will also occur when more than 3 Microgateway Engine sidecars have been deployed and an Airlock Microgateway Community edition is used. Check the number of deployed sidecars and consider either upgrading to a Premium edition license or removing the excess sidecars. For more information, go to Airlock Microgateway license editions and support.

To tackle all other failure messages, proceed with the next section.

Troubleshooting

General measures

To gain insight into the cause of the failing helm test, perform the following checks:

  1. To find out whether the Microgateway Operator deployments and Pod are running as expected, check their status.
    • Check the Microgateway Operator deployments.
      • copy
      kubectl describe deployments.apps -n airlock-microgateway-system airlock-microgateway-operator
    • Check the Microgateway Operator Pod.
      • copy
      kubectl describe pod -n airlock-microgateway-system airlock-microgateway-operator-<pod identifier>
  2. Check the status of the Web application Pod that was created for the helm test.
    3. copy
    kubectl describe pod -n <web application namespace> airlock-microgateway-<identifier web application pod>
  3. Additionally, check the log messages of both the Microgateway Operator Pod and the Web application Pod:
    4. copy
    kubectl logs -n airlock-microgateway-system airlock-microgateway-operator-<pod identifier>
    copy
    kubectl logs -n <web application namespace> airlock-microgateway-<identifier web application pod>

Further steps, based on the information gathered from the checks

  1. If the logs include the message statement ErrImagePullBackoff, there may be something wrong with the image of the Microgateway Engine. Check whether the value of engine.image.repository in the Microgateway Operator Helm chart resembles your setup.
  2. If the logs state that the network sanity check performed by the Microgateway Network Validator failed, there may be a problem with the Microgateway CNI plugin. The CNI plugin configures the routing within the Web application Pod and must be installed before installing the Microgateway Operator. Note that both must have the same release version.
    To find the problem, check the status and logs of the Microgateway CNI plugin.
    3. copy
    kubectl describe pod -n kube-system airlock-microgateway-cni-<suffix>
    copy
    kubectl logs -n kube-system airlock-microgateway-cni-<suffix>

    For more information, go to Troubleshooting Microgateway CNI Helm test and Network Validator sanity check.

  3. General network problems may cause the helm test to fail. Check your network values and correct them, if necessary. Additionally, have a look at the article Troubleshooting for network routing issues in the application pod.