Testing a REST API with QALIPSIS

In this quickstart guide, you will learn how to test a very simple echo-like HTTP service using QALIPSIS.

Before you get started

Before starting, you need:

  • A general understanding of the QALIPSIS Core Concepts.

  • Java 11 SDK or above installed on your system.

  • Docker installed on your system.

What this guide covers

This guide will take you through the following steps for testing a simple HTTP service:

  1. Start an HTTP server and a docker environment for the project.

  2. Create a project.

  3. Configure a scenario.

    1. Configure the scenario basics

    2. Add steps to the scenario.

      1. An HTTP step to send requests to the server.

      2. A verify step that asserts that the server’s response is matches the expectations and to verify the server’s responsiveness.

      3. An HTTP WITH step that reuses the HTTP connection to send further requests.

      4. A second assertion step to verify the server’s responses and responsiveness.

  4. Execute the scenario using Gradle and view the report.

  5. Create a Java archive of your project so that you can transport it as needed and run it anywhere.

  6. Enable event logging to allow you to view errors in the report.

Start an HTTP server and a docker environment

  1. Create a file named docker-compose.yml in your working directory with the following content:

    version: '3.5'
services:
  http-punching-ball:
    image: aerisconsulting/http-punching-ball
    command:
      - "--https=true"
      - "--ssl-key=http-server.key"
      - "--ssl-cert=http-server.crt"
    ports:
      - "18080:8080"
      - "18443:8443"

  2. Start the created docker environment with the command docker compose up -d.

Create a project

Start by creating a new project using the QALIPSIS bootstrap project.

  1. Configure the project to use the plugins you need for your scenario:

    qalipsis {
    plugins {
        apacheKafka()
        netty()
        elasticsearch()
        influxDb()
        r2dbcJasync()
    }
}

  2. Add the dependencies for the data assertions.

    dependencies{
    implementation("com.willowtreeapps.assertk:assertk:0.+")
}

  3. Reload the Gradle project if your IDE or editor provides this functionality.

  4. Rename the file src/main/kotlin/my/bootstrap/MyBootstrapScenario.kt to better represent your project.

  5. Rename the docker images in the docker {} area as appropriate for your project.

Configure a scenario

Edit the downloaded scenario skeleton

  1. Open the downloaded project archive in your IDE.

  2. Edit the file that you renamed above.

The scenario skeleton should be displayed:

package quickstart
import io.qalipsis.api.annotations.Scenario
class Quickstart{
    @Scenario("quickstart")
    fun quickstart() {
        scenario {
            minionsCount = 1
            profile {
            }
        }
    .start()
    }
}

Configure the scenario basics

Configure the scenario to inject 10,000 minions at a rate of 100 minions every 500 ms.

        scenario {
            minionsCount = 10000
            profile {
                regular(periodMs = 500, minionsCountProLaunch = 100)
            }
        }
        .start()

Add steps to the scenario

Add and configure steps within the scenario that will send an HTTP request to the server and verify that the server’s response is as expected.

Send an HTTP request

Add an http step to the scenario that will send a POST request to the server.

        .start() (1)
        .netty() (2)
        .http {  (3)
            name = "quickstart-http-post"  (4)
            connect {  (5)
                url("http://localhost:18080")
                version = HttpVersion.HTTP_1_1
            }
            request { _,_ ->  (6)
                    SimpleHttpRequest(HttpMethod.POST, "/echo").body(
                    "Hello World!",
                    HttpHeaderValues.TEXT_PLAIN
                    )
            }
            report {
                reportErrors = true  (7)
            }
        }
1 Specifies the starting point for load injection.
2 Accesses the netty namespace.
3 Calls the the http operator
4 Gives the http step a name.
5 Configures the HTTP connection to the server.
6 Configures the HTTP request (HTTP method, path and body) that will be sent to the server.
7 Enables reporting of errors for the http step.

Execute the scenario

Execute the scenario using Gradle: ./gradlew run. This allows you to observe requests logged by the HTTP server. However, it does not verify that the server is returning what you expect.

Verify the server’s response

To verify that the server is returning what you expect, add a verify step to the scenario, just after the http step.

        .start()
        .netty()
        .http {  (1)
            // ...
        }
        .verify { result ->
            assertSoftly {
                result.asClue {
                    it.response shouldNotBe null
                    it.response!!.asClue { response ->
                        response.status shouldBe HttpResponseStatus.OK  (2)
                        response.body shouldContainOnlyOnce "Hello World!"  (3)
                        response.body shouldContainOnlyOnce "POST"
                    }
                    it.meters.asClue { meters ->
                        meters.timeToFirstByte!! shouldBeLessThanOrEqualTo Duration.ofSeconds(1)
                        meters.timeToLastByte!! shouldBeLessThanOrEqualTo Duration.ofSeconds(2)
                    }
                }
            }
        }.configure {
            name = "verify-post" (3)
        }
1 Place holder for the http step (quickstart-http-post) configuration parameters.
2 Verifies that the HTTP status of the response is OK; the step uses io.netty.handler.codec.http.HttpResponseStatus for the verification.
3 Verifies that the body of the response contains the string ("Hello World!") sent in the request.

At this point, if you execute the updated scenario, a log file will display showing the status of each response and confirmation that that server is posting the "Hello World!" string.

Reuse the HTTP connection

You can strengthen the relevance of your test by:

  1. Adding an httpWith step to the scenario that reuses the original HTTP connection (quickstart-http-post).

    The same minions that passed through the first http step will pass through the httpWith step.

  2. Adding a second verify step to assert that the response from the server is as expected.

...
            configure {
                name = "verify-post"
            }
            .netty()  (1)
            .httpWith("quickstart-http-post") {  (2)
                report {
                    reportErrors = true
                }
                request { _, _ ->
                    SimpleHttpRequest(HttpMethod.PATCH, "/echo").body(
                        "Hello World!",
                        HttpHeaderValues.TEXT_PLAIN
                    )
                }
            }.verify { result ->
                assertSoftly {
                    result.asClue {
                        it.response shouldNotBe null
                        it.response!!.asClue { response ->
                            response.status shouldBe HttpResponseStatus.OK (3)
                            response.body shouldContainOnlyOnce "Hello World!" (4)
                        }
                        it.meters.asClue { meters ->
                            meters.timeToFirstByte!! shouldBeLessThanOrEqualTo Duration.ofMillis(100)
                            meters.timeToLastByte!! shouldBeLessThanOrEqualTo Duration.ofMillis(200)
                        }
                    }
                }
            }.configure {
                name = "verify-patch"
            }
1 Accesses the netty namespace.
2 Calls the httpWith operator and specifies the previous http connection to reuse ("quickstart-http-post").

Note that there is no need to specify parameters for the`httpWith` connection since it reuses the parameters from the previous http connection.
3 Verifies that the HTTP status of the response is OK; the step uses io.netty.handler.codec.http.HttpResponseStatus for the verification.
4 Verifies that the body of the response contains the string ("Hello World!") sent in the request.

Execute the scenario

Execute your updated scenario as a Gradle project.

A couple of seconds after execution, a report (similar to the one below) will display on your console.

============================================================
=====================  CAMPAIGN REPORT =====================
============================================================

Campaign...........................quickstart-http-4fb6244cd3
Start..............................2022-03-30T16:41:02.597973Z
End................................2022-03-30T16:41:53.411602Z
Duration...........................50 seconds
Configured minions.................10000
Completed minions..................10000
Successful steps executions........39832
Failed steps executions............42
Status.............................FAILED



=====================  SCENARIO REPORT =====================
Scenario...........................quickstart-http
Start..............................2022-03-30T16:41:02.597973Z
End................................2022-03-30T16:41:53.411602Z
Duration...........................50 seconds
Configured minions.................10000
Completed minions..................10000
Successful steps executions........39832
Failed steps executions............42
Status.............................FAILED
Messages:
- ERROR:  step 'quickstart-http-post' - "Success: 9958, Execution errors: 42"
- INFO:   step 'verify-post' - "Success: 9958, Failures (verification errors): 0, Errors (execution errors): 0"
- INFO:   step '_622niod959' - "Success: 9958, Execution errors: 0"
- ERROR:  step 'verify-patch' - "Success: 9926, Failures (verification errors): 32, Errors (execution errors): 0"

  • Note that the information for the whole campaign as well as for each scenario is displayed.

  • In the example above, there are errors, which explains why the execution failed.

  • To understand the errors, refer to Enable Event Logging .

Create an archive of your scenario

Execute the statement ./gradlew assemble.

A JAR archive ending with -qalipsis.jar is created in the folder build/libs. This archive contains all dependencies and is self-sufficient to execute your scenario on any machine with a Java Runtime Environment installed.

To execute your scenario: . Open a terminal and navigate to the folder where the JAR archive is stored. . Move the JAR archive out of the build folder; this will avoid accidental deletion of the archive. . Execute the command java -jar quickstart-<your-version>-qalipsis.jar.

Enable event logging

In the current directory, create a folder called config. In this new folder, create a file called qalipsis.yml with the following content:

events:
  root: warn
  export:
    slf4j:
      enabled: true

This enables the logging of the events with severity WARN and higher to a file called qalipsis-events.log.

The qalipsis-events.log contains a contains the details of the individual failed assertions.

For example, the details of a failed assertion may be displayed as:

2022-03-31T06:33:55.937Z  WARN --- step.assertion.failure;io.qalipsis.plugins.netty.tcp.ConnectionAndRequestResult@1e869276

The pertinent parts of the above message are:

  • The exact instant when the failed assertion occurred:`2022-03-31T06:33:55`.

  • The reason for the failure: PT0.10696475S should be < PT0.1S (the durations are displayed in the ISO-8601 format).

  • The campaign (quickstart-campaign-4d39026fd7), scenario (quickstart-http), and verification step (verify-patch) where the failed assertion occurred.