Reducing false-positives

False positives are created whenever Airlock Anomaly Shield flags legitimate sessions as anomalous. The causes of false positives can be very different – thus we have to choose the correct measures to successfully reduce the number of false positives.

Keep training data clean:

  • Training data should be based solely on legitimate traffic.

  • Create traffic exclusions to exclude non-legitimate traffic before gathering training data and ML model training.
    See also Optional configuration of Traffic Matchers.

Configure response rule exceptions and anomaly detection exclusions for Anomaly Shield applications:

  • Adding new response rule exceptions to Anomaly Shield applications can effectively reduce the number of blocks. However, creating exceptions will not influence the anomaly detection itself – logs will still be written whenever an anomaly has been detected.
    Anomaly detection exclusions allow bypassing certain traffic from being analyzed, which can be an appropriate measure to reduce false positives. Detection exclusions can also reduce the system load.

  • Configure response rule exceptions for Anomaly Shield applications if applicable.
    See also Section – Anomaly Detection.
  • Configure anomaly detection exclusions for Anomaly Shield applications if applicable.
    See also Traffic Matchers detail page.

Fine-tune the thresholds for anomaly detection to reduce the false positive rate:

  • Tuning anomaly detection thresholds greatly impact what behavior may result in what action. Keep in mind that the default threshold set is the result based on extensive research and is suitable for a wide variety of use cases.

  • Ensure the ML models have been trained with clean traffic data and that exceptions have been configured before testing/attempting any fine-tuning measures.
  • Airlock Anomaly Shield evaluates sessions after the first 15 requests by default. This threshold can be increased so that the session evaluation happens at a later point of a session. See the table below.
 
Notice

Starting with release 8.6, indicator thresholds are configured per application in the GUI (see Section – Indicator Thresholds). Threshold values previously configured in Expert Settings are no longer taken into account.

  • If threshold values from Expert Settings are still required, transfer them to the corresponding application.

Inspect the log messages

With clean training data in the ColdDB and the required enforcement exception rules configured, we have to inspect the WR-SG-NMLY-200 log messages in order to decide what to fine-tune:

Log-observation

Tuning measure

One ML model appears too aggressive and is active for most or all sessions.

This is a strong indicator for increasing the default model thresholds.

Airlock Anomaly Shield evaluates a session as anomalous within the first few requests, but the bit count decreases to non-anomalous values while the session is ongoing.

This is a strong indicator for increasing the number of requests within a session prior to the evaluation.

Increasing the number of requests within a session

The minimal session length value can be increased if sessions are tagged anomalous after too few session requests and when the same session-bit count drops with additional requests. Note that the minimal required (request) aggregation value should be equally increased.
By doing so, session evaluation requires more requests within a session before the session is tagged.

  1. Go to:
    /opt/airlock/ml-service/devel/conf/ml.ini
  2. Increase the min_sess_length (default is 15) gradually, e.g., in +5 steps.
  3. Go to:
    /opt/airlock/ml-service/devel/conf/default_hyper_parameters.ini
  4. Increase the min_req_aggregation (default is 15) equally to min_sess_length.
  5. Re-train the models as described in the Training and model enforcement article and activate the configuration.
  6. Check if the changes show the desired effect on reducing false positives during operation. If not, further increase the values until the outcome is acceptable.