HTTP Validator - Usage guide & API reference

1. Introduction

This project aims to ensure the reliability and consistency of a web application or HTTP-accessible resource by performing automated checks on various web endpoints. It does this by periodically sending requests to these endpoints and validating the responses based on predefined criteria.

1.1. Purpose

Automate the monitoring and validation of web application endpoints to quickly detect and respond to any issues. This can help maintain the integrity and performance of the web application, ensuring that it functions correctly and meets expected standards.

1.2. Features

Automated Validation Tasks:
- The system periodically sends HTTP requests to specified web endpoints.
- It checks the responses against predefined criteria to ensure they are correct and expected.
Configurable Task Management:
- Validation tasks are defined in an XML file, which can be updated independently of the application code.
- This file can be modified either manually or through a web API, without needing to restart the application.
Notification System:
- If any validation task detects an issue, the system sends an email notification.
- This ensures that responsible parties are alerted promptly to any problems.
Scheduling Flexibility:
- The schedule for running validation tasks can be set through the API or a separate configuration file.

1.3. Summary

In essence, the HTTP Validator can provide a robust yet flexible solution for monitoring web application endpoints and content or systems reachable via HTTP. By automating the process of sending requests and validating responses, it helps ensure the application remains reliable and performant.

The ability to configure tasks and schedules easily, combined with prompt email notifications for issues and low resource usage, make it a practical tool for monitoring and maintaining application health.

2. Constructing an XML File for Validation Tasks

An XML file for defining validation tasks, which is used to specify the details of the tasks that the system will perform, follows a structured format.

2.1. Steps to create the XML file

Start with the Root Element: Begin your XML file with a root element called <validations>. This will contain all your individual validation tasks.
Add Validation Elements: Inside the <validations> element, you will add multiple <validation> elements. Each <validation> element represents a single task.
Define each Task: For each <validation> element, you need to specify several sub-elements and attributes:
- <url>: The URL to which the request will be sent to (up to 2000 characters). Required.
  - method: Attribute with the indicator of the request method (use 0 for GET requests, 1 for POST). Required.
- <header>: Any headers that should be included in the request. These should be formatted as HeaderName|HeaderValue, separated by pipe (|, up to 3000 characters per element). Optional.
- <reqbody>: JSON content to be sent in the body of a POST request (up to 1000 characters). Optional.
- <response>: The content expected to be present in the body of the response (up to 1000 characters). Required.
  - statuscode: Attribute with the expected HTTP status code of the response. Required.

2.2. Example XML Tasks

Below are some examples of validation tasks in XML format:

<?xml version="1.0" encoding="UTF-8"?>
<validations>
    <!-- Example Task 1 -->
    <validation>
        <url method="0">http://example.com/api/status</url>
        <header>Authorization|Bearer token4534354X.9+yjo</header>
        <header>Accept|application/json</header>
        <response statuscode="200">A string in a plaintext response</response>
    </validation>

    <!-- Example Task 2 -->
    <validation>
        <url method="0">http://example.com/api/data</url>
        <response statuscode="200">"data":"some JSON data"</response>
    </validation>

    <!-- Example Task 3 -->
    <validation>
        <url method="1">http://example.com/api/health</url>
        <header>Content-Type|application/json</header>
        <reqbody>{ "id": "AX1000T" }</reqbody>
        <response statuscode="200">"health":87876854354</response>
    </validation>

    <!-- Example Task 4 -->
    <validation>
        <url method="0">http://example.com/oldpath</url>
        <response statuscode="404" />
    </validation>
</validations>

2.3. Explanation of Examples

Example Task 1:
- Request Method: GET (indicated by method="0")
- Request URL: http://example.com/api/status
- Request Headers: Two headers, Authorization with a bearer token and Accept with value application/json Keys and values are separated by | (pipe character)
- Expected Status Code: 200
- Expected Response Body: The string expected to be present in a plaintext response
Example Task 2:
- Request Method: GET
- Request URL: http://example.com/api/data
- Request Headers: No headers (no <header> element present below the <validation> element)
- Expected Status Code: 200
- Expected Response Body: "data":"some JSON data"
Example Task 3:
- Request Method: POST (indicated by method="1")
- Request URL: http://example.com/api/health
- Request Headers: One header, Content-Type with value application/json
- Request Body: JSON data to be sent in the body of the request, inside the <reqbody> element
- Expected Status Code: 200
- Expected Response Body: "health":87876854354
Example Task 4:
- Request Method: GET
- Request URL: http://example.com/oldpath
- Request Headers: No headers
- Request Body: None (no <reqbody> element present within the <validation> element)
- Expected Status Code: 404
- Expected Response Body: Body contents are ignored (indicated by empty <response> element)

Besides these common cases most text editors and many online tools can read an input URL or string and highlight any conflicting characters in XML and provide alternatives to escape them.

By following this structure, you can easily define multiple validation tasks in an XML file, which the system will use to perform automated checks on your web application endpoints.

3. Running the application

3.1. Compiling the executable jar file

git clone https://github.com/lfir/http-validator.git
cd http-validator
./mvnw package

Or download one of the precompiled artifacts from the releases site and place it in the same location as the config directory.

3.2. Starting the application from the BASH terminal

3.2.1. Environment variables that need to be set

SENDGRID_APIKEY - An api key from a SendGrid user with permission to send emails
NOTIF_FROM - Email address to be used as sender of the notifications
NOTIF_TO - Destination address for email notifications
JWT_SECRET - Encoded secret key for signing JWTs

3.2.2. Important files used by the application

./config/application.properties - Used to set the run schedule and location of the datafile
./config/validations.xml - File with the validation task definitions

These two files need to be present in the config folder before starting the app.

Finally, run

java -jar target/http-validator-x.y.z.jar

NOTE: If more than one version of Java is installed on the host a suitable one (v21+) might need to be specified instead of the default before running the commands above, i.e.

export JAVA_HOME=/usr/lib/jvm/java-21-openjdk
./mvnw...
/path/to/java -jar...

4. API - Information endpoints

4.1. Application Status

This endpoint provides the status of the web application, including start time, data file status, and configuration status.

4.1.1. Request Structure

GET /api/status HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA
Accept: application/json
Host: localhost:8080

4.1.2. Request Headers

Name Description

Name	Description
`Authorization`	Bearer token for authentication

Authorization

Bearer token for authentication

4.1.3. Response Structure

Path Type Description

Path	Type	Description
`start_time`	`String`	The start time of the application in a formatted string
`datafile_status`	`String`	`OK`: Data file status is okay, i.e. it exists and is readable `ERROR`: Data file status is not okay
`config_status`	`String`	`OK`: Both mail and validation configurations are okay (run schedule, credentials, from and to addresses) `ERROR`: Either mail or validation configuration is not okay

start_time

String

The start time of the application in a formatted string

datafile_status

String

OK: Data file status is okay, i.e. it exists and is readable

ERROR: Data file status is not okay

config_status

String

OK: Both mail and validation configurations are okay (run schedule, credentials, from and to addresses)

ERROR: Either mail or validation configuration is not okay

4.1.4. Sample Response

HTTP/1.1 200 OK
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Content-Length: 93

{"start_time":"Sat, 12 Oct 2024 23:04:00 GMT","datafile_status":"OK","config_status":"ERROR"}

4.1.5. CURL request

$ curl 'http://localhost:8080/api/status' -i -X GET \
    -H 'Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA' \
    -H 'Accept: application/json'

4.2. Validation Tasks - Last Run data

This endpoint provides summary information about results of the last run of the validation tasks, including start time, time elapsed, number of total tasks, tasks ok and tasks failed.

4.2.1. Request Structure

GET /api/validator/lastrun HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA
Accept: application/json
Host: localhost:8080

4.2.2. Request Headers

Name Description

Name	Description
`Authorization`	Bearer token for authentication

Authorization

Bearer token for authentication

4.2.3. Response Structure

The following fields will be included if data is available (at least one run of validations has already completed) when the endpoint is called:

Path Type Description

Path	Type	Description
`start_time`	`String`	The start time of the last run in a formatted string
`time_elapsed`	`String`	Duration of the last run
`tasks_total`	`String`	The total number of tasks to be processed in the last run
`tasks_ok`	`String`	The number of tasks with expected results
`tasks_failed`	`String`	The number of tasks with invalid results in the last run

start_time

String

The start time of the last run in a formatted string

time_elapsed

String

Duration of the last run

tasks_total

String

The total number of tasks to be processed in the last run

tasks_ok

String

The number of tasks with expected results

tasks_failed

String

The number of tasks with invalid results in the last run

If there’s no data yet an error response like the one below will be returned.

Path Type Description

Path	Type	Description
`error`	`String`	No validation tasks have been completed yet

error

String

No validation tasks have been completed yet

4.2.4. Sample Response

HTTP/1.1 503 Service Unavailable
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Content-Length: 55

{"error":"No validation tasks have been completed yet"}

HTTP/1.1 200 OK
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Content-Length: 117

{"start_time":"Sat, 12 Oct 2024 23:04:00 GMT","tasks_failed":"1","tasks_total":"1","tasks_ok":"0","time_elapsed":"0"}

4.2.5. CURL request

$ curl 'http://localhost:8080/api/validator/lastrun' -i -X GET \
    -H 'Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA' \
    -H 'Accept: application/json'

5. API - Configuration management endpoints

5.1. Update Validation Task definitions file

This endpoint updates the validator’s data file with new validation tasks. The file should be an XML file that adheres to the predefined schema.

5.1.1. Request Structure

PUT /api/validator/datafile HTTP/1.1
Content-Type: multipart/form-data; boundary=6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA
Host: localhost:8080

--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=file; filename=validations.xml
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?><validations><validation><url method="0">http://example.com/api/test</url><response statuscode="200" /></validation></validations>
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm--

5.1.2. Request Headers

Name Description

Name	Description
`Authorization`	Bearer token for authentication

Authorization

Bearer token for authentication

5.1.3. Response Structure

If the new XML provided is valid according to the schema the body will be empty. Otherwise, an error message will be returned indicating whether the problem was in the contents of the new file or the operation of writing it to disk.

Path Type Description

Path	Type	Description
`error`	`String`	Invalid Data File or Error updating Data File

error

String

Invalid Data File or Error updating Data File

5.1.4. Sample Response

HTTP/1.1 400 Bad Request
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Content-Length: 29

{"error":"Invalid Data File"}

HTTP/1.1 200 OK
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers

5.1.5. CURL request

$ curl 'http://localhost:8080/api/validator/datafile' -i -X PUT \
    -H 'Content-Type: multipart/form-data' \
    -H 'Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA' \
    -F 'file=@validations.xml;type=application/xml'

5.2. Update Run Schedule

This endpoint updates the validator’s run schedule with a new cron expression.

5.2.1. Request Structure

PUT /api/validator/runschedule HTTP/1.1
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA
Content-Length: 29
Host: localhost:8080

{"cron_expression":"* 2 * ?"}

5.2.2. Request Headers

Name Description

Name	Description
`Authorization`	Bearer token for authentication

Authorization

Bearer token for authentication

5.2.3. Response Structure

If the new cron expression provided is valid the body will be empty. Otherwise, an error message will be returned.

Path Type Description

Path	Type	Description
`error`	`String`	Invalid Cron Expression

error

String

Invalid Cron Expression

5.2.4. Sample Response

HTTP/1.1 400 Bad Request
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Content-Length: 35

{"error":"Invalid Cron Expression"}

HTTP/1.1 200 OK
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers

5.2.5. CURL request

$ curl 'http://localhost:8080/api/validator/runschedule' -i -X PUT \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJjZi5tYXliZWxhbWJkYS5odHRwdmFsaWRhdG9yLnNwcmluZ2Jvb3Quc2VydmljZS5Kd3RBdXRoZW50aWNhdGlvblNlcnZpY2UiLCJpYXQiOjE3Mjg3NzQyNDAsImV4cCI6MTcyODc3Nzg0MH0.pTh7H3XB1Y_GuRi11lnOXo_HpmPuCsbPKfP_inxRW2dDcO_tmk-JYyK-yGyCTwpe4Ya_PnBqzDyWUg1zAh9qTA' \
    -d '{"cron_expression":"* 2 * ?"}'

6. Limitations and other relevant information

At this stage of the project Validation Tasks can only validate responses with empty body or discrete, text-only data like plaintext, CSV, HTML, JSON, etc.
The main log file of the application is ./logs/spring-framework.log.
When the run schedule is updated from the API WebApplicationContext is restarted automatically to begin using it (PID remains the same, beans & variables like start_time are re-initialized).
Configuration changes done by updating ./config/application.properties require manually restarting the application so they enter in effect.
If a network error occurs that prevents a task from completing it is considered failed and a notification email is sent about that, but at this point no more details are provided regarding the issue (i.e. if the connection was rejected or it timed out).