Configure the data storage

Overview

QALIPSIS stores the master data and the time-series data generated by the scenario (events, meters) in separate databases.

The master data requires transactional consistency and the time-series data requires high throughput and strong calculation capabilities. Separate databases allow for the different requirements.

They can be activated separately, allowing you to use QALIPSIS:

with master data storage only,
with time-series data storage only,
with both master and time-series data storage,
or without any storage.

Master data storage

The master data is stored in a relational database. It contains the configuration of the scenarios, the users, the campaigns, and the summaries of the test iterations.

This storage is connected to the QALIPSIS head node or QALIPSIS standalone, not to the factories which are stateless.

While this installation requires the installation of a PostgreSQL database, we will not go into those details here as they are provided in the official PostgreSQL documentation.

The master data storage is not enabled by default even when properly configured and can be activated by passing the command line argument --persistent when executing QALIPSIS.

Prerequisites

Before configuring the data storage, you need to have a PostgreSQL database installed and running. You also need to have a user and a database dedicated to QALIPSIS. We discourage using the default postgres user and database for QALIPSIS.

They both can have the name of your choice.

Prepare the database

Connect to the database you created for QALIPSIS with your admin user (generally postgres).

Execute the following script initializing the database, replacing [username] with the name of the user you created for QALIPSIS:

CREATE SCHEMA qalipsis_liquibase IF NOT EXISTS;
ALTER SCHEMA qalipsis_liquibase OWNER TO <<username>>;
CREATE SCHEMA qalipsis IF NOT EXISTS;
ALTER SCHEMA qalipsis OWNER TO <<username>>;

Configure the access for QALIPSIS

Navigate to your installation folder where the qalipsis.sh and qalipsis.bat files are located.
Provide the following configuration to allow QALIPSIS to access the database:
```
datasource:
  host: <host>
  port: <port>
  database: <database-name>
  username: <username>
  password: <password>
```
Replace the <host>, <port> (generally 5432), <database-name>, <username>, and <password> values with the appropriate values for you configuration.

The access to the database is performed using a pool of connections to optimize performance. This means that multiple requests to the database can be handled concurrently, improving the overall throughput and responsiveness of the system.

If the default values for the connection pool are not sufficient for your use case, you can adjust them (see the example below that provides the default values).

datasource:
  pool:
    min: 4 (1)
    max: 20 (2)
    maxLifeTime: PT10M (3)
    maxIdleTime: PT4M (4)
    maxAcquireTime: PT5S (5)

1	The minimum number of connections in the pool. This is the number of connections that will be kept open even if they are not used.
2	The maximum number of connections in the pool. This is the maximum number of connections that can be opened to the database at the same time. When this limit is reached, new requests will wait until a connection is available.
3	The maximum lifetime of a connection in the pool. After this time, the connection will be closed and a new one will be opened.
4	The maximum idle time of a connection in the pool. After this time, the connection will be closed if it is not used.
5	The maximum time to acquire a connection from the pool. If this time is exceeded, an exception will be thrown. Note that the times use the ISO-8601 format, where `PT` stands for "Period of Time" and `M` for "Minutes", `S` for "Seconds", etc.

Time-series data storage

Generating time-series data

The time-series data are stored in a time-series database. They contains the events and meters generated by the scenarios in order to allow for the analysis of the test results.

The generation of data is performed by the factories or QALIPSIS standalone. It is supported by plugins that have to be activated in the factories or QALIPSIS standalone and properly configured.

You can even enable several plugins at the same time to store the data in different databases. However, configuring a storage for the time-series data is not mandatory, as QALIPSIS can operate without it.

QALIPSIS supports several time-series databases, including elasticsearch, influxDB, timescaleDB, and graphite. Refer to the applicable plugin section for details.

Most of those databases come with their own GUI, which will allow you to perform advanced data analysis and visualization.

Rendering time-series data

The rendering of data using the GUI or the REST API is performed by the head or QALIPSIS standalone and is for the moment only supported by the timescaleDB plugin which has to be activated in the QALIPSIS installation and properly configured.