CLI ColdDB tool

Note that with Airlock Gateway 8.0 the CLI ColdDB tool has become widely obsolete due to the improved, UI-based Anomaly Shield Model Management. Basic training and management tasks (i.e. export/import of data) are covered by the UI functionalities.

The airlock-ml-colddb-tool is a CLI application that allows administrators to perform a set of actions on ColdDB machine learning data collections. This includes data check, merge, move, copy and delete actions. It can also be used to shrink the ColdDB itself.

Path to the tool:

copy
/opt/airlock/ml-service/bin/airlock-ml-colddb-tool

Use ./airlock-ml-colddb-tool --help for detailed tool description.

Copy command

The copy command can be used to merge ColdDBs, e.g. to consolidate data collections between Airlock Gateway instances. Adjust the code of the following example before use:

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool --cold-db /var/tmp/failovernode-cold.db --other-cold-db /var/airlock/ml-service/cold.db copy
  • Example explained:
  • In our example, all session metrics from failovernode-cold.db are copied and merged into the cold.db database.

The copy command can also be used to restore archived session metrics data. Adjust the code of the following example before use:

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool --application ebanking --start 2020-10-17 --end 2020-11-01 --cold-db /var/tmp/archive-cold.db --other-cold-db /var/airlock/ml-service/cold.db copy
  • Example explained:
  • In our example, all session metrics from the application named ebanking from 2020-10-17 to 2020-11-01 are restored from the archive-cold.db into the cold.db database.

Summary command

Before machine learning models can be trained for an application, sufficient session metrics data must be collected. To check the content of the ColdDB, use the summary command. Adjust the code of the following example before use:

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool summary 
    Application                first                 last  SessionCount 
0      BookShop  2021-03-09T10:28:33  2021-04-07T10:28:44          1337 
1      OtherApp  2021-03-11T08:12:01  2021-04-07T10:28:40          2503
  • Example explained:
  • Application – the name of the application for which session metrics have been collected.
  • first – the timestamp of the first sample.
  • last – the timestamp of the last sample.
  • SessionCount – the number of collected session metrics.

Note that only sessions with a minimum number of requests are collected. Very short sessions, e.g. with only a single request, are not collected because they have little to no value as training data.

Move command

The move command can be used to archive session metrics. Adjust the code of the following example before use:

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool --application ebanking --start 2020-10-17 --end 2020-11-01 --other-cold-db /var/tmp/archive-cold.db move
  • Example explained:
  • In our example, all session metrics from the application named ebanking from 2020-10-17 to 2020-11-01 are moved into a ColdDB named archive-cold.db
  • To restore the data from the archive-cold.db, the copy command can be used.

Delete command

The delete command can be used to remove session metrics from ColdDB. Note that deleting session metrics is permanent. Adjust the code of the following example before use:

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool --application ebanking --start 2020-10-17 --end 2020-11-01 delete
  • Example explained:
  • In our example, all session metrics from the application named ebanking from 2020-10-17 to 2020-11-01 are removed from the ColdDB by the delete command.
  • After deletion, the vacuum command can be used to reduce the size of the ColdDB.

Vacuum command

The vacuum command can be used to shrink and repackage the ColdDB. This command is usually used to free up space after the deletion of session metrics from the ColdDB.

copy
cd /opt/airlock/ml-service/bin 
./airlock-ml-colddb-tool vacuum