Log messages and actions

Log IDs

Airlock Anomaly Shield can be configured to trigger actions, whenever an anomaly has been detected. Log messages can be very helpful to identify sessions that triggered actions and/or to refine the trigger settings like Minimal bit count or Pattern based on log messages. This section describes the different log messages and how they can be identified.

 
Info

The ML-related log messages can be identified within the log file by their content. The syntax is “log_id”: “WR-SG-NMLY-<log_id>”.

The available actions and log-IDs are:

  • Tag session as anomalous for sessions – WR-SG-NMLY-401 and WR-SG-SUMMARY
  • Tag session as anomalous for IP aggregates – WR-SG-NMLY-411 and WR-SG-SUMMARY
  • Terminate session – WR-SG-NMLY-420
  • Block IP – WR-SG-NMLY-421

Log level settings

It is possible to report the information returned by the Airlock Anomaly Shield ML service even without creating any rules or actions. This requires enabling Anomaly Shield and to set the Log sessions anomaly details under Application detail page.

The following settings can be selected:

  • Never – to never write the ML information for the ML application.
  • When session anomaly pattern changes – to only write the ML information on a change in the resulting pattern.
  • When raw session anomaly values change – to only write the ML information on a change in the raw values.
  • For every request – to always write the ML information for the ML application.

Log message bodies

The Anomaly Shield ML service information is reported with the following log message:

 
Example
{ 
    "log_id": "WR-SG-NMLY-<log_id>", 
    "req_id": "<current_request_id>", 
    "sess_id": "<current_session_id>", 
    ..., 
    "mapping": "<current_mapping>", 
    "client_ip": "<current_client_ip>", 
    ..., 
    "ml_app": "<current_ml_application_id>", 
    "message": "ML values: <raw_ml_values>; thresholds: <configured_thresholds>; pattern: <resulting_pattern>; bitcount: <number_of_active_indicators>" 
}

The format of the raw ML values and the thresholds is:

  • cba:<float>
  • grm:<float>
  • ifo:<float>
  • qpm:<float>
  • scm:<float>
  • tcs:<float>

  • Where <float> is between '0.00' and '1.0'.

The format of the resulting anomaly pattern is:

  • cba:<bool>
  • grm:<bool>
  • ifo:<bool>
  • qpm:<bool>
  • scm:<bool>
  • tcs:<bool>

  • Where <bool> may be either '1' for true or '0' for false.

Abbreviations of ML model names

Log messages contain abbreviations for the different types of machine-learning models behind the pattern names.

Example:

 
Example
{ 
    "log_id": "WR-SG-NMLY-200", 
    "req_id": ""YGxArETHlhOM62aHU58LAQAAADg, 
    "sess_id": "ab6be40964329b226f4b0c97c663da36", 
    ..., 
    "mapping": "my_mapping", 
    "client_ip": "80.253.98.123", 
    ..., 
    "ml_app": "my_ml_application", 
    "message": "ML values: cba:0.01,grm:1.0,ifo:0,qpm:0.01,scm:0.01,tcs:0.98; thresholds: cba:0.99,grm:0.99,ifo:0.99,qpm:0.99,scm:0.99,tcs:0.97; pattern: cba:0,grm:1,ifo:0,qpm:0,scm:0,tcs:1; bitcount:2" 
}

The following abbreviations are made:

Abbreviation in logs

Labels in the configuration*

UI, figures, and descriptive texts

cba

Client Behavior

Client Behavior

grm

GraphMetricsCluster

Graph Metrics Cluster

ifo

IsolationForest

Isolation Forest

qpm

Query Parameters

Query Parameters

scm

StatusCodeMeta

Status Code Meta

tcs

TimingCluster

Timing Cluster

*

Used in: /opt/airlock/ml-service/conf/analytics_indicator_thresholds.json.

Log messages for Tag session as anomalous

When a MARK_ANOMALOUS action is issued, the following log message will be written:

 
Example
{ 
  "log_id": "WR-SG-NMLY-401", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Marking session as anomalous. Matched rule \"<matching_rule_name>\", with triggers (<list_of_matching_triggers>)" 
}

The anomalous property of the session will be reported in each WR-SG-SUMMARY message with the field ml_anomaly.

It can have the following values:

normal

The session was never marked as anomalous.

anomalous

The session is currently marked as anomalous.

exception

The session would be marked as anomalous but isn't because of a matched exception rule.

redeemed

The session is currently not marked as anomalous but was in the past.

 
Info

This is a non-blocking action. Rules with only non-blocking actions are best configured to run after the rules with blocking actions.

  • Since the first matching rule aborts the rule processing, this action could mistakenly act as a allow list of blocking actions if configured incorrectly.

Log messages for Terminate session

When a TERMINATE_SESSION action is issued, the current session is being terminated and the following log message will be written:

 
Example
# Block message 
{ 
  "log_id": "WR-SG-BLOCK-155" 
  "req_id": "<current_request_id>", 
  "sess_id": "<current_session_id>", 
  ..., 
  "attack_type": "Behaviour anomaly", 
  "block_type": "Machine Learning", 
  "constraint": "Behaviour", 
  "th_mode": "<current_threat_handling_mode>", 
  "ml_app": "<current_ml_application_id>", 
  "message": "Session anomaly detected by ML rule \"<matching_rule_name>\". Executing block actions: <list_of_blocking_actions>" 
  
} 
# Additional log message confirming the action in integration mode 
{ 
  "log_id": "WR-SG-NMLY-420", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Terminating current session" 
}

Log messages for Block IP

When a Block IP action is issued, the current IP is being blocked for a certain amount of time.

The following log message will be written when an IP is blocked:

 
Example
# Block message 
{ 
  "log_id": "WR-SG-BLOCK-155" 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "attack_type": "Behaviour anomaly", 
  "block_type": "Machine Learning", 
  "constraint": "Behaviour"), 
  "th_mode": "<current_threat_handling_mode>", 
  "ml_app": "<current_ml_application_id>", 
  "message": "Session anomaly detected by ML rule \"<matching_rule_name>\". Executing block actions: <list_of_blocking_actions>" 
  
} 
# Additional log message confirming the action in integration mode 
{ 
  "log_id": "WR-SG-NMLY-421", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Blocking client IP <current_client_ip> for <configured_time> seconds" 
}
 
Info

Note that the blocking time can be configured as described here: Miscellaneous tuning