InfluxDB plugin

Overview

The InfluxDB plugin connects QALIPSIS to InfluxDB.

Technologies addressed
Dependency

io.qalipsis.plugin:qalipsis-plugin-influxdb

Namespace in scenario

influxdb()

Client library

influxdb-client-kotlin: refer to the influxdb-client-kotlin.

Supported steps

Poll step

The poll step within the InfluxDB plugin polls the newest records from the database, with a delay between each execution of a select query.

Ancestor

Scenario

Functionality

The poll step is created only once in a scenario; it is then used throughout the scenario as called. The poll step within the InfluxDB plugin uses a strategy of “delivered at least once”.

Example
influxdb().poll {
    connect { (1)
        server(
            url = "http://localhost:18086",
            bucket = "iot",
            org = "qalipsis"
        )
        basic(user = "DJones", password = "Oxford1987")
    }
    query( (2)
        """
        from(bucket:"${iot}")
            |> range(start: -15m)
            |> filter(
                fn: (r) => r._measurement == "battery_state"
            )
    """.trimIndent()
    )
    pollDelay(Duration.ofSeconds(1))
}
1 Define the connection parameters to InfluxDB.
2 Specify a query respecting the requirements of InfluxDB.
Tip

Use the poll step with a join operator to verify the values that the tested system saves to the database.

Reference: Documentation

Refer to the InfluxDB documentation for further parameter and configuration information.

Save step

The save step within the InfluxDB plugin persists a batch of records into the InfluxDB bucket using a specific points class.

Ancestor

Step

Functionality

The save step’s input and step context are used to generate the bucket, the list of points with their measurement, and tags and fields, which are then combined and forwarded to the database.

Example
.save {
    connect { (1)
        server(
            url =  "http://localhost:18086",
            bucket = "iot",
            org = "qalipsis"
        )
        basic(user = "DJones", password = "Oxford1987")
    }
    query {  (2)
        bucket = { StepContext, input -> "iot" }
        organization = { _, _ -> "qalipsis" }
        points = { _, input ->
            listOf(
                Point.measurement("battery_state")
                    .addField("battery_level", input.batteryLevel)
                    .addTag("device_id", input.deviceId)
                    .addTag("timestamp", input.timestamp.epochSecond.toString())
            )
        }
    }
}
1 Define the connection parameters to InfluxDB.
2 Specify a query respecting the requirements of InfluxDB.
Tips

  • Use the save step after a collect step to reduce the number of queries executed onto the database and to reduce the load your scenario generates on the test system.

  • The save step is extremely flexible: bucket, tags, measurement and fields might depend on the context and input value received from the previous step.

Reference documentation

Refer to the InfluxDB documentation for further parameter and configuration information.

Search step

The search step within the InfluxDB plugin searches records in the database using a search query.

Ancestor

Step

Functionality

The search step’s input and step context are used to generate a query; the batch of results is then forwarded to the next step(s).

Example
.influxdb()
.search {
    connect { (1)
        server(
            url = "http://localhost:18086",
            bucket = "iot",
            org = "qalipsis"
        )
        basic(user = "DJones", password = "Oxford1987")
    }
    query { StepContext, input -> (2)
        """
            from(bucket: "${iot}")
                |> range(start: -15m)
                |> filter(
                    fn: (r) => r._measurement == "${"battery_state"}" and
                    r.${"device_id"} == "${input.deviceId}" and
                    r.${"timestamp"} == "${input.timestamp.epochSecond}"
                )
        """.trimIndent()
    }
}
1 Define the connection parameters to InfluxDB.
2 Specify a query respecting the requirements of InfluxDB.
Tip

Use the search step after a collect step to reduce the number of queries executed onto the database and to reduce the load the scenario generates on the test system.

Reference documentation

Refer to the InfluxDB documentation for further parameter and configuration information.

Configuration

DSL parameters

Available parameters are described in the table below.

Parameter Description

connect

Defines the connection parameters to InfluxDB.
Applicable Steps: Poll, Search, Save
Optional/Required: Required
Data Type: Lambda within which connection parameters are defined.
Default Value: N/A

Example
connect {
    server(
        url = "http://localhost:18086",
        bucket = "iot",
        org = "qalipsis"
    )
}

connect.server

Identifies the server parameters for the connection to InfluxDB.
Applicable Steps: Poll, Search, Save
Optional/Required: Required
Data Type: Named function argument
Default Value: N/A

Example
connect {
    server(
        url = "http://localhost:18086",
        bucket = "iot",
        org = "qalipsis"
    )
}

connect.server.url

Identifies the url for the connection to InfluxDB.
Applicable Steps: Poll, Search, Save
Optional/Required: Required
Data Type: url
Default Value: http://localhost:8086

Example
connect {
    server(
        url = "http://localhost:8086"
    )
}

connect.server.bucket

Name of the bucket to use; a bucket defines the the highest level of data organization in InfluxDB.
Applicable Steps: Poll, Search, Save
Optional/Required: Required
Data Type: String
Default Value: ""

Example
connect {
    server(
        bucket = "iot"
    )
}

connect.server.org

The name of the organization (i.e., workspace) associated with the bucket.
Applicable Steps: Poll, Search, Save
Optional/Required: Required
Data Type: String
Default Value: N/A

Example
connect {
    server(
    org = "qalipsis"
    )
}

query

Specifies a query respecting the requirements of InfluxDB.
Applicable Steps: Poll, Save, Search
Optional/Required: Required
Data Type: Lambda that returns a String.
Default Value: N/A

Example (Poll)
query(
    """
    from(bucket:"${iot}")
        |> range(start: -15m)
        |> filter(
            fn: (r) => r._measurement == "battery_state"
        )
""".trimIndent()
)
Example (Save)
query {
    bucket = { _, _ -> "iot" }
    organization = { _, _ -> "qalipsis" }
    points = { _, input ->
        listOf(
            Point.measurement("battery_state")
                .addField("battery_level", input.batteryLevel)
                .addTag("device_id", input.deviceId)
                .addTag("timestamp", input.timestamp.epochSecond.toString())
        )
    }
}
Example (Search)
query { _, input -> <
    """
        from(bucket: "${iot}")
            |> range(start: -15m)
            |> filter(
                fn: (r) => r._measurement == "${"battery_state"}" and
                r.${"device_id"} == "${input.deviceId}" and
                r.${"timestamp"} == "${input.timestamp.epochSecond}"
        )
    """.trimIndent()
}

parameters

Binds values to the clauses of the prepared query.
Applicable Step: Poll
Optional/Required: Optional
Data Type: + Lambda that returns a string.
Default Value: List<Any> / vararg Any

Example
parameters(listOf(1, "abc")) or parameters(1, "abc")

pollDelay

Specifies the delay between query executions.
Applicable Step: Poll
Optional/Required: Optional
Data Type: java.time.Duration
Default Value: N/A

Example
pollDelay(Duration.ofSeconds(1))

singletonConfiguration

Configures how polled results are delivered to scenario minions.
Refer to SingletonConfiguration parameters for details on strategies and examples of usage.

Shared defaults for InfluxDB steps

You can define defaults once in the scenario section or just after, and let all following InfluxDB steps inherit them.

scenario {
  influxdb().defaults { (1)
    connect {
      server(
        url = "http://localhost:8086",
        bucket = "iot",
        org = "qalipsis"
      )
    }
    monitoring {
      events = true
      meters = true
    }
  }
}
1 Defaults are applied to subsequent InfluxDB steps in the same scenario. Individual steps can still override values.

Analytics

QALIPSIS can publish the meters and events collected during a campaign to InfluxDB through dedicated publishers.

Configure the publishers in the factory configuration, separate from the scenario DSL. Refer to Provide the configuration to QALIPSIS for information about specifying the configuration to QALIPSIS.

Meters

The meters publisher writes the meter snapshots collected during a campaign as records to an InfluxDB bucket.

Example configuration file (YAML)
meters:
  export:
    influxdb:
      enabled: true
      url: http://localhost:8086
      user-name:
      password:
      org: qalipsis
      bucket: qalipsis-meter
      prefix: qalipsis.
      publishers: 1

The parameters used to configure the publication of meters to InfluxDB are described in the table below.

Parameter Description

meters.export.influxdb.enabled

Activates the publication of meters to InfluxDB. Must be set to true to enable publishing.
Data Type: Boolean
Default Value: false

meters.export.influxdb.url

URL of the InfluxDB instance receiving the meters.
Data Type: String
Default Value: http://localhost:8086

meters.export.influxdb.user-name

Name of the user used for basic authentication when connecting to InfluxDB.
Data Type: String
Default Value: Empty string

meters.export.influxdb.password

Password of the user used for basic authentication when connecting to InfluxDB.
Data Type: String
Default Value: Empty string

meters.export.influxdb.org

Name of the InfluxDB organization to which the meters are written.
Data Type: String
Default Value: qalipsis

meters.export.influxdb.bucket

Name of the InfluxDB bucket where the meters are written.
Data Type: String
Default Value: qalipsis-meter

meters.export.influxdb.prefix

Prefix prepended to every metric name when publishing to InfluxDB.
Data Type: String
Default Value: qalipsis.

meters.export.influxdb.publishers

Number of concurrent publishers that can send meter batches to InfluxDB.
Data Type: Positive integer
Default Value: 1 (no concurrency)

Events

The events publisher buffers the events emitted during a campaign and writes them in bulk as records to an InfluxDB bucket.

Example configuration file (YAML)
events:
  export:
    influxdb:
      enabled: true
      url: http://localhost:8086
      min-level: INFO
      linger-period: 10s
      batch-size: 40000
      publishers: 1
      username:
      password:
      org: qalipsis
      bucket: qalipsis-event

The parameters used to configure the publication of events to InfluxDB are described in the table below.

Parameter Description

events.export.influxdb.enabled

Activates the publication of events to InfluxDB. Must be set to true to enable publishing.
Data Type: Boolean
Default Value: false

events.export.influxdb.url

URL of the InfluxDB instance receiving the events.
Data Type: String
Default Value: http://localhost:8086

events.export.influxdb.min-level

Minimum event level published. Acceptable values are TRACE, DEBUG, INFO, WARN, ERROR, OFF.
Data Type: ENUM
Default Value: INFO

events.export.influxdb.linger-period

Maximum time the events are buffered before forcing a flush to InfluxDB, even when the batch size is not reached.
Data Type: Positive duration
Default Value: 10s

events.export.influxdb.batch-size

Maximum number of events buffered between two publications to InfluxDB.
Data Type: Positive integer
Default Value: 40000

events.export.influxdb.publishers

Number of concurrent publishers that can send event batches to InfluxDB.
Data Type: Positive integer
Default Value: 1 (no concurrency)

events.export.influxdb.username

Name of the user used for basic authentication when connecting to InfluxDB.
Data Type: String
Default Value: Empty string

events.export.influxdb.password

Password of the user used for basic authentication when connecting to InfluxDB.
Data Type: String
Default Value: Empty string

events.export.influxdb.org

Name of the InfluxDB organization to which the events are written.
Data Type: String
Default Value: Empty string

events.export.influxdb.bucket

Name of the InfluxDB bucket where the events are written.
Data Type: String
Default Value: Empty string
Reference Documentation

Refer to Monitoring test campaigns for a further explanation of the meter and event parameter values.