Log messages and actions
Log messages and actions of Airlock Anomaly Shield

Log IDs

Airlock Anomaly Shield can be configured to trigger actions, whenever an anomaly has been detected. Log messages can be very helpful to identify sessions that triggered actions and/or to refine the trigger settings like Minimal bit count or Pattern based on log messages. This section describes the different log messages and how they can be identified.

The ML-related log messages can be identified within the log file by their content. The syntax is "log_id": "WR-SG-NMLY-<log_id>".

  • The available actions and log-IDs are:
  • Log incident – (WR-SG-NMLY-400)
  • Tag session as anomalous – (WR-SG-NMLY-401)
  • Terminate session – (WR-SG-NMLY-420)
  • Block IP – (WR-SG-NMLY-421)

Log level settings

It is possible to report the information returned by the Airlock Anomaly Shield ML service even without creating any rules or actions. This requires enabling Anomaly Shield and to set the Log sessions anomaly details under Instruction part 3 – Configure the anomaly detection and response.

  • The following settings can be selected:
  • Never – to never write the ML information for the ML application.
  • When session anomaly pattern changes – to only write the ML information on a change in the resulting pattern.
  • When raw session anomaly values change – to only write the ML information on a change in the raw values.
  • For every request – to always write the ML information for the ML application.

Log message bodies

The Anomaly Shield ML service information is reported with the following log message:

{ 
    "log_id": "WR-SG-NMLY-<log_id>", 
    "req_id": "<current_request_id>", 
    "sess_id": "<current_session_id>", 
    ..., 
    "mapping": "<current_mapping>", 
    "client_ip": "<current_client_ip>", 
    ..., 
    "ml_app": "<current_ml_application_id>", 
    "message": "ML values: <raw_ml_values>; thresholds: <configured_thresholds>; pattern: <resulting_pattern>; bitcount: <number_of_active_indicators>" 
}
  • The format of the raw ML values and the thresholds is:
  • con:<float>
  • grm:<float>
  • ifo:<float>
  • mco:<float>
  • scm:<float>
  • tcs:<float>
  • Where <float> is between '0.00' and '1.0'.

  • The format of the resulting anomaly pattern is:
  • con:<bool>
  • grm:<bool>
  • ifo:<bool>
  • mco:<bool>
  • scm:<bool>
  • tcs:<bool>
  • Where <bool> may be either '1' for true or '0' for false.

Abbreviations of ML model names

Log messages contain abbreviations for the different types of machine learning models behind the pattern names.

Example:

{ 
    "log_id": "WR-SG-NMLY-200", 
    "req_id": ""YGxArETHlhOM62aHU58LAQAAADg, 
    "sess_id": "ab6be40964329b226f4b0c97c663da36", 
    ..., 
    "mapping": "my_mapping", 
    "client_ip": "80.253.98.123", 
    ..., 
    "ml_app": "my_ml_application", 
    "message": "ML values: con:0.01,grm:1.0,ifo:0,mco:0.01,scm:0.01,tcs:0.01; thresholds: con:0.99,grm:0.99,ifo:0.99,mco:0.99,scm:0.99,tcs:0.97; pattern: con:0,grm:1,ifo:0,mco:0,scm:0,tcs:1; bitcount:2" 
}

The following abbreviations are made:

Abbreviation in logs
Labels in the configuration*
UI, figures, and descriptive texts
con
ConnectionMetrics
Connection Metrics
grm
GraphMetricsCluster
Graph Metrics Cluster
ifo
IsolationForest
Isolation Forest
mco
MultipleCountries
Multiple Countries
scm
StatusCodeMeta
Status Code Meta
tcs
TimingCluster
Timing Cluster
*
Used in: /opt/airlock/ml-service/conf/analytics_indicator_thresholds.json.

For a brief description of the machine learning models/pattern names, see Instruction – Section Patterns.

Log message for Log incident

When a Log incident action is issued, the following log message will be written:

{ 
  "log_id": "WR-SG-NMLY-400", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Session anomaly notification. Matched rule \"<matching_rule_name>\"" 
}

This is a non-blocking action. Rules with only non-blocking actions are best configured to run after the rules with blocking actions.

  • Since the first matching rule aborts the rule processing, this action could mistakenly act as an allow list of blocking actions if configured incorrectly.

Log messages for Tag session as anomalous

When a MARK_ANOMALOUS action is issued, the following log message will be written:

{ 
  "log_id": "WR-SG-NMLY-401", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Marking session as anomalous. Matched rule \"<matching_rule_name>\", with triggers (<list_of_matching_triggers>)" 
}

The anomalous property of the session will be reported in each WR-SG-SUMMARY message with the field ml_anomaly.

It can have the following values:

normal
The session was never marked as anomalous.
anomalous
The session is currently marked as anomalous.
redeemed
The session is currently not marked as anomalous but was in the past.
exception
The session would be marked as anomalous but isn't because of a matched exception rule.

This is a non-blocking action. Rules with only non-blocking actions are best configured to run after the rules with blocking actions.

  • Since the first matching rule aborts the rule processing, this action could mistakenly act as a allow list of blocking actions if configured incorrectly.

Log messages for Terminate session

When a TERMINATE_SESSION action is issued, the current session is being terminated and the following log message will be written:

# Block message 
{ 
  "log_id": "WR-SG-BLOCK-155" 
  "req_id": "<current_request_id>", 
  "sess_id": "<current_session_id>", 
  ..., 
  "attack_type": "Behaviour anomaly", 
  "block_type": "Machine Learning", 
  "constraint": "Behaviour", 
  "th_mode": "<current_threat_handling_mode>", 
  "ml_app": "<current_ml_application_id>", 
  "message": "Session anomaly detected by ML rule \"<matching_rule_name>\". Executing block actions: <list_of_blocking_actions>" 
  
} 
# Additional log message confirming the action in integration mode 
{ 
  "log_id": "WR-SG-NMLY-420", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Terminating current session" 
}
…Show more…Show less

Log messages for Block IP

When a Block IP action is issued, the current IP is being blocked for a certain amount of time.

The following log message will be written when an IP is blocked:

# Block message 
{ 
  "log_id": "WR-SG-BLOCK-155" 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "attack_type": "Behaviour anomaly", 
  "block_type": "Machine Learning", 
  "constraint": "Behaviour"), 
  "th_mode": "<current_threat_handling_mode>", 
  "ml_app": "<current_ml_application_id>", 
  "message": "Session anomaly detected by ML rule \"<matching_rule_name>\". Executing block actions: <list_of_blocking_actions>" 
  
} 
# Additional log message confirming the action in integration mode 
{ 
  "log_id": "WR-SG-NMLY-421", 
  "req_id": "<current_request_id>",  
  "sess_id": "<current_session_id>",  
  ..., 
  "message": "ML: Blocking client IP <current_client_ip> for <configured_time> seconds" 
}

Note that the blocking time can be configured as described here: Miscellaneous tuning

Further information and links