Netty plugin

Overview

The Netty plugin supports TCP, UDP, HTTP and MQTT clients.

Technologies addressed
Dependency

io.qalipsis.plugin:qalipsis-plugin-netty

Namespace in scenario

netty()

Client library

Netty: refer to the Netty project.

Supported steps

  • HTTP: executes an HTTP request using HTTP 1.1 or HTTP 2.0.

  • HTTP WITH: executes an HTTP request reusing the HTTP connection created by a precedent step.

  • CLOSE HTTP: closes an HTTP connection created by a precedent step.

  • MQTT Publish: sends data to an MQTT server.

  • MQTT Subscribe: consumes data from an MQTT server.

  • TCP: exchanges data with a TCP server.

  • TCP WITH: exchanges data reusing the TCP connection created by a precedent step.

  • CLOSE TCP: closes a TCP connection created by a precedent step.

  • UDP: exchanges data with a UDP endpoint.

HTTP step

The HTTP step within the Netty plugin executes an HTTP request using HTTP 1.1 or HTTP 2.0.

Ancestor

Scenario or Step

Functionality

  • The HTTP step establishes a connection with the server using either a pool of connections or an individual connection for each minion.

  • If an individual connection is used, the connection is maintained until the latest execution for the minion is performed, including the ones from the related HTTP WITH step, as in the example.

  • When the step is kept open without limit, a CLOSE TCP step must be executed to properly close the connections and avoid saturation of the system.

    Be aware that the number of open connections that can be concurrently maintained depends on the configuration of your operating system. If there are issues with maintaining open connections, try adapting the limits of the file descriptors.

Example
netty().http {
    connect {
        url("https://my-website.com:8080/test")
        connectTimeout = Duration.ofSeconds(5)
        inflate()
        charset(Charsets.ISO_8859_1)
        maxContentLength(123)
        followRedirections(15)
        noDelay = false
        keepConnectionAlive = false
        tls {
            disableCertificateVerification = true
        }
        proxy {
            type = HttpProxyType.SOCKS5
            address("my-proxy", 9876)
        }
        followRedirections(5)
    }
    pool {
        size = 143
        checkHealthBeforeUse = true
    }
    request { context, input -> simple(HttpMethod.HEAD, "/head") }
}.deserialize(String::class)
Notable parameters

  • connect: configures the connection to the remote endpoint.

    • version: version of the HTTP protocol to use: HTTP_1_1 or HTTP_2_0,; defaults to HTTP_1_1.

    • url: the root URL, the path of the request is appended to it.

    • connectTimeout: limit duration to fully establish the connection; defaults to 10 seconds.

    • readTimeout: limit duration to receive a response and read it totally after a request; defaults to 10 seconds.

    • shutdownTimeout: limit duration to close a connection; defaults to 10 seconds.

    • sendBufferSize: the size of the buffer used to prepare the data to send; defaults to 1024 bytes.

    • receiveBufferSize: the size of the buffer to receive the data of the response; defaults to 1024 bytes.

    • nettyChannelOptions: overwrites the options of the Netty channel; use with care.

    • noDelay: disables the Nagle Algorithm; defaults to true.

    • keepConnectionAlive: when set to true, the HTTP header connection is set to Keep-Alive and the client connection is kept open until CLOSEHTTP() is called.

    • tls: configuration for SSL/TLS

      • disableCertificationVerification: set it to true when the certificate on server should not be verified (for example for a self-signed certificate)

      • protocols: supported protocols; defaults to the Netty default for the related HTTP version.

      • ciphers: supported cipher algorithms; defaults to the Netty default for the related HTTP version.

    • proxy: configuration to exchange HTTP data via a proxy; defaults to no configuration.

      • address: the host and port of the proxy.

      • type: type of the proxy to use: HTTP, SOCKS5; defaults to HTTP.

      • authenticate: username and optional password to authenticate to the proxy, if it supports and requires it.

    • charset: character encoding of the HTTP requests,

    • maxContentLength: maximal expected size of a request, in bytes, defaults to 1048576, which is the minimum allowed.

    • inflate: enables the acceptance of compressed response. Compression of the request is not supported.

    • followRedirections: when the server returns an HTTP status 3xx (redirection) in the response, the client will resend the request to the provided location. Otherwise, the response is kept as is. The argument maxRedirections provides the maximal number of redirections to follow; defaults to 10. By default, the redirections are not followed at all. Note that following a redirections makes the request to be sent there, exactly as it is (same method and content).

  • pool: configuration of the pool of connections; the pool is disabled by default.

    • size: number of constant connections maintained in the pool.

    • checkHealthBeforeUse: verifies the health of the connection when polled from the pool and before it is used.

  • request: a factory to create the request to exchange, using the step context and the input as arguments. You can create simple, form or multipart requests.

  • deserialize: specifies the class to use to deserialize the body of the response.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

HTTP WITH step

The HTTP WITH step within the Netty plugin executes an HTTP request reusing the HTTP connection created by a precedent step.

Ancestor

Step

Functionality

  • The HTTP WITH step is a direct or indirect follower of an HTTP step. HTTP WITH specifies the name of the step that owns the connection, and reuses a connection from that step as it was created (either from a pool or the connection attached to the minion).

  • HTTP WITH cannot configure the connection differently than the HTTP step it relates to.

Example
netty().httpWith("name-of-a-previous-HTTP-step") {
    request { context, input -> simple(HttpMethod.HEAD, "/head") }
}.deserialize(String::class)
Notable parameters

  • netty().httpWith("name-of-a-previous-HTTP-step"): the name of the HTTP step that owns the connection.

    • request: a factory to create the request to exchange, using the step context and the input as arguments.

  • deserialize: specifies the class to use to deserialize the body of the response.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

CLOSE HTTP step

The CLOSE HTTP step within the Netty plugin closes the HTTP connection attached to the minion, and created by a direct or indirect ancestor step, when the tail of the minion workflow is received.

It is always a best practice to use a CLOSE HTTP step when reusing an HTTP connection across several steps without a pool of connections.

Ancestor

Step

Functionality

The CLOSE HTTP step is a direct or indirect follower of an HTTP step which keeps the connection open. CLOSE HTTP simply forwards the input it receives to the next step(s).

Example
netty().closeHttp("name-of-a-previous-HTTP-step")
Notable parameter

  • closeHttp (required): specifies the name of an HTTP step owning the HTTP connection to close.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

MQTT Publish step

The MQTT Publish step within the Netty plugin sends data to an MQTT server.

Ancestor

Step

Functionality

The MQTT Publish step provides a variety of options for controlling the quality and reliability of messages that are published to the server.

Example
.netty().mqttPublish {
    connect {
        host =  "localhost"
        port = 11883
    }
    clientName("mqtt-client-1")
    records { _, batteryState ->
        listOf(
            MqttPublishRecord(
                value = objectMapper.writeValueAsString(batteryState),
                topicName = "battery-state"
            )
        )
    }
}
Notable parameters

  • connect (required): configures the host and port for the connection.

  • clientName (required): uniquely identifies the client onto the MQTT server.

  • topicName (required): identifies the MQTT topic to write to.

Reference Documentation

Refer to the Netty documentation for further configuration and parameter information.

MQTT Subscribe step

The MQTT Subscribe step within the Netty plugin consumes data from an MQTT server.

Ancestor

Step

Functionality

The MQTT Subscribe step provides a variety of options for controlling the quality and reliability of the messages that are subscribed to from the server.

Example
it.netty().mqttSubscribe {
    connect {
        host = localhost
        port = 11883
    }
    clientName("mqtt-client-1")
    topicFilter(ServerConfiguration.TOPIC_NAME)
}
Notable Parameters

  • connect (required): configures the host and port for the connection.

  • clientName (required): uniquely identifies the client onto the MQTT server.

  • topicFilter (required): MQTT topics or patterns to consume.

TCP step

The TCP step within the Netty plugin is a client for TCP.

Ancestor

Scenario or Step

Functionality

The TCP step establishes a connection with the server using either a pool of connections or an individual connection for each minion.

If an individual connection is used, the connection is maintained until the latest execution for the minion is performed, including the ones from the related TCP WITH step, as in the example.

When the step is kept open without limit, a CLOSE TCP step should be executed to properly close the connections and avoid a saturation of the system.

Be aware that the number of open connections that can be concurrently maintained depends on the configuration of your operating system. If there are issues with maintaining open connections, adapting the limits for file descriptors may help.

Example
netty().tcp {
    request { context, input -> input.encodeToByteArray() }
    connect {
        address("remote-server.acme.com", 9000)
        connectTimeout = Duration.ofMinutes(1)
        noDelay = false
        keepConnectionAlive = false
        tls {
            disableCertificateVerification = true
        }
        proxy {
            type = HttpProxyType.SOCKS5
            address("my-proxy", 9876)
        }
    }
    pool {
        size = 1000
        checkHealthBeforeUse = true
    }
}
Notable parameters

  • request: a factory to create the request to exchange, using the step context and the input as arguments; the request is an array of bytes.

  • connect:configures the connection to the remote endpoint.

    • address:hostname or IP address and port of the remote server.

      • connectTimeout: limit duration to fully establish the connection; defaults to 10 seconds.

      • readTimeout:limit duration to receive and read a response after a request; defaults to 10 seconds.

      • shutdownTimeout:limit duration to close a connection; defaults to 10 seconds.

      • sendBufferSize: the size of the buffer used to prepare the data to send; defaults to 1024 bytes.

      • receiveBufferSize: the size of the buffer to receive the data of the response; defaults to 1024 bytes.

      • nettyChannelOptions: overwrites the options of the Netty channel, use with care.

      • noDelay: disables the Nagle Algorithm; defaults to true.

      • tls: configuration for SSL/TLS

    • disableCertificationVerification: set to true if the certificate on the server should not be verified (for example for a self-signed certificate).

    • protocols: supported protocols; defaults to the Netty default.

    • ciphers: supported cipher algorithms; defaults to the Netty default.

      • proxy: configuration to exchange data via a proxy; defaults to no configuration.

    • address: hostname or IP address and port of the proxy server.

    • type: type of the proxy to use: SOCKS4, SOCKS5; defaults to SOCKS4.

    • authenticate: username and optional password to authenticate to the proxy, if the proxy supports and requires it.

  • pool: configuration of the pool of connections; defaults to disabled.

    • size: number of constant connections maintained in the pool.

    • checkHealthBeforeUse: verifies the health of the connection when polled from the pool and before it is used.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

TCP WITH step

The TCP WITH step within the Netty plugin sends a TCP payload, reusing the connection created by a precedent step.

Ancestor

Step

Functionality

  • The TCP WITH step is a direct or indirect follower of a TCP step. TCP WITH specifies the name of the step that owns the connection, and reuses a connection from that step as it was created (either from a pool or the connection attached to the minion).

  • TCP WITH cannot configure the connection differently than the TCP step it relates to.

Example
netty().tcpWith("name-of-a-previous-HTTP-step") {
    request { context, input -> input.encodeToByteArray() }
}
Notable parameters

  • request: a factory to create the request to exchange, using the step context and the input as arguments.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

CLOSE TCP step

The CLOSE TCP step within the Netty plugin closes the TCP connection, attached to a minion and created by a direct or indirect ancestor step, when the tail of the minion workflow is received.

It is best practice to use a CLOSE TCP step when reusing a TCP connection across several steps without a pool of connections.

Ancestor

Step

Functionality

The CLOSE TCP step is a direct or indirect follower of an HTTP step which keeps the connection open. CLOSE TCP simply forwards the input it receives to the next step(s).

Example
netty().closeHttp("name-of-a-previous-HTTP-step")

Notable parameter: * closeHttp (required): specifies the name of the HTTP step to close.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.

UDP step

The UDP step within the Netty plugin is a client for UDP.

It is best practice to use the UDP step when reusing a TCP connection across several steps without a pool of connections.

Ancestor: Scenario or Step

Functionality

The UDP step creates a UDP Datagram packet from the request and sends it to the remote endpoint and expects a response.

Example
netty().udp {
    request { context, input -> input.encodeToByteArray() }
    connect {
        address("remote-server.acme.com", 9000)
        connectTimeout = Duration.ofMinutes(1)
    }
}
Notable parameters

  • request: a factory to create the request to exchange, using the step context and the input as arguments, the request is an array of bytes

  • connect: configures the connection to the remote endpoint

    • address: hostname or IP address and port of the remote endpoint

    • connectTimeout: limit duration to fully establish the connection; defaults to 10 seconds.

    • readTimeout: limit duration to receive a response and read it totally after a request; defaults to 10 seconds.

    • shutdownTimeout: limit duration to close a connection; defaults to 10 seconds.

    • sendBufferSize: the size of the buffer used to prepare the data to send; defaults to 1024 bytes.

    • receiveBufferSize: the size of the buffer to receive the data of the response; defaults to 1024 bytes.

    • nettyChannelOptions: overwrites the options of the Netty channel, use with care.

Reference Documentation

  • Refer to the Netty documentation for further configuration and parameter information.