HTTP Protocol Compliance Tests¶

Smithy is a protocol-agnostic IDL that tries to abstract the serialization format of request and response messages sent between a client and server. Protocol specifications define the serialization format of a protocol, and protocol compliance tests help to ensure that implementations correctly implement a protocol specification.

Table of contents

Overview
- Parameter format
httpRequestTests
- HTTP request example
httpResponseTests
- HTTP response example
- HTTP error response example

Overview¶

This specification defines two traits in the smithy.test namespace that are used to make assertions about client and server protocol implementations.

smithy.test#httpRequestTests: Used to define how an HTTP request is serialized given a specific protocol, authentication scheme, and set of input parameters.
smithy.test#httpResponseTests: Used to define how an HTTP response is serialized given a specific protocol, authentication scheme, and set of output or error parameters.

Protocol implementation developers use these traits to ensure that their implementation is correct. This can be done through code generation of test cases or by dynamically loading test cases at runtime. For example, a Java implementation could generate JUnit test cases to assert that the expectations defined in a model match the behavior of a generated client or server.

Parameter format¶

The params property used in both the httpRequestTests trait and httpResponseTests trait test cases represents parameters that are used to serialize HTTP requests and responses. In order to compare implementation specific results against the expected result of each test case across different programming languages, parameters are defined in the same format specified in Trait JSON values with the following additional constraints:

Timestamp values must be converted to a Unix timestamp represented as an integer.
Client implementations that automatically provide values for members marked with the idempotencyToken trait MUST use a constant value of 00000000-0000-4000-8000-000000000000.

httpRequestTests¶

Summary

The httpRequestTests trait is used to define how an HTTP request is serialized given a specific protocol, authentication scheme, and set of input parameters.

Trait selector

operation

Value type

list of HttpRequestTestCase structures

The httpRequestTests trait is a list of HttpRequestTestCase structures that support the following members:

Property	Type	Description
id	`string`	Required. The identifier of the test case. This identifier can be used by protocol test implementations to filter out unsupported test cases by ID, to generate test case names, etc. The provided `id` MUST match Smithy's `identifier` ABNF. No two `httpRequestTests` test cases can share the same ID.
protocol	`string`	Required. The name of the protocol to test. Because Smithy services can support multiple protocols, each test MUST specify which protocol is under test.
method	`string`	Required. The expected serialized HTTP request method.
uri	`string`	Required. The request-target of the HTTP request, not including the query string (for example, "/foo/bar").
authScheme	`string`	The optional authentication scheme to assume. It's possible that specific authentication schemes might influence the serialization logic of an HTTP request.
queryParams	`list<string>`	A list of the expected serialized query string parameters. Each element in the list is a query string key value pair that starts with the query string parameter name optionally followed by "=", optionally followed by the query string parameter value. For example, "foo=bar", "foo=", and "foo" are all valid values. Note This kind of list is used instead of a map so that query string parameter values for lists can be represented using repeated key-value pairs. The query string parameter name and the value MUST appear in the format in which it is expected to be sent over the wire; if a key or value needs to be percent-encoded, then it MUST appear percent-encoded in this list. A serialized HTTP request is not in compliance with the protocol if any query string parameter defined in `queryParams` is not defined in the request or if the value of a query string parameter in the request differs from the expected value. `queryParams` applies no constraints on additional query parameters.
forbidQueryParams	`list<string>`	A list of query string parameter names that must not appear in the serialized HTTP request. Each value MUST appear in the format in which it is sent over the wire; if a key needs to be percent-encoded, then it MUST appear percent-encoded in this list.
requireQueryParams	`list<string>`	A list of query string parameter names that MUST appear in the serialized request URI, but no assertion is made on the value. Each value MUST appear in the format in which it is sent over the wire; if a key needs to be percent-encoded, then it MUST appear percent-encoded in this list.
headers	`map<string, string>`	A map of expected HTTP headers. Each key represents a header field name and each value represents the expected header value. An HTTP request is not in compliance with the protocol if any listed header is missing from the serialized request or if the expected header value differs from the serialized request value. `headers` applies no constraints on additional headers.
forbidHeaders	[`string`]	A list of header field names that must not appear in the serialized HTTP request.
requireHeaders	[`string`]	A list of header field names that must appear in the serialized HTTP message, but no assertion is made on the value. Headers listed in `headers` do not need to appear in this list.
body	`string`	The expected HTTP message body. If no request body is defined, then no assertions are made about the body of the message.
bodyMediaType	`string`	The media type of the `body`. This is used to help test runners to parse and validate the expected data against generated data. Binary media type formats require that the contents of `body` are base64 encoded.
params	`document`	Defines the input parameters used to generate the HTTP request. These parameters MUST be compatible with the input of the operation.
vendorParams	`document`	Defines vendor-specific parameters that are used to influence the request. For example, some vendors might utilize environment variables, configuration files on disk, or other means to influence the serialization formats used by clients or servers.
documentation	`string`	A description of the test and what is being asserted defined in CommonMark.

HTTP request example¶

The following example defines a protocol compliance test for a JSON protocol that uses HTTP binding traits.

namespace smithy.example

use smithy.test#httpRequestTests

@http(method: "POST", uri: "/")
@httpRequestTests([
    {
        id: "say_hello",
        protocol: "example",
        params: {
            "greeting": "Hi",
            "name": "Teddy",
            "query": "Hello there"
        },
        method: "POST",
        uri: "/",
        queryParams: [
            "Hi=Hello%20there"
        ],
        headers: {
            "X-Greeting": "Hi",
        },
        body: "{\"name\": \"Teddy\"}",
        bodyMediaType: "application/json"
    }
])
operation SayHello {
    input: SayHelloInput
}

structure SayHelloInput {
    @httpHeader("X-Greeting")
    greeting: String,

    @httpQuery("Hi")
    query: String,

    name: String
}

{
    "smithy": "0.5.0",
    "shapes": {
        "smithy.example#SayHello": {
            "type": "operation",
            "input": {
                "target": "smithy.example#SayHelloInput"
            },
            "traits": {
                "smithy.api#http": {
                    "method": "POST",
                    "uri": "/",
                    "code": 200
                },
                "smithy.test#httpRequestTests": [
                    {
                        "id": "say_hello",
                        "protocol": "example",
                        "method": "POST",
                        "uri": "/",
                        "headers": {
                            "X-Greeting": "Hi"
                        },
                        "queryParams": [
                            "Hi=Hello%20there"
                        ],
                        "body": "{\"name\": \"Teddy\"}",
                        "bodyMediaType": "application/json"
                        "params": {
                            "greeting": "Hi",
                            "name": "Teddy",
                            "query": "Hello there"
                        }
                    }
                ]
            }
        },
        "smithy.example#SayHelloInput": {
            "type": "structure",
            "members": {
                "greeting": {
                    "target": "smithy.api#String",
                    "traits": {
                        "smithy.api#httpHeader": "X-Greeting"
                    }
                },
                "name": {
                    "target": "smithy.api#String"
                }
            }
        }
    }
}

httpResponseTests¶

Summary

The httpResponseTests trait is used to define how an HTTP response is serialized given a specific protocol, authentication scheme, and set of output or error parameters.

Trait selector

:each(operation, structure[trait|error])

Value type

list of HttpResponseTestCase structures

The httpResponseTests trait is a list of HttpResponseTestCase structures that support the following members:

Property	Type	Description
id	`string`	Required. The identifier of the test case. This identifier can be used by protocol test implementations to filter out unsupported test cases by ID, to generate test case names, etc. The provided `id` MUST match Smithy's `identifier` ABNF. No two `httpResponseTests` test cases can share the same ID.
protocol	`string`	Required. The name of the protocol to test. Because Smithy services can support multiple protocols, each test MUST specify which protocol is under test.
code	`integer`	Required. The expected HTTP response status code.
authScheme	`string`	The optional authentication scheme to assume. It's possible that specific authentication schemes might influence the serialization logic of an HTTP response.
headers	`map<string, string>`	A map of expected HTTP headers. Each key represents a header field name and each value represents the expected header value. An HTTP response is not in compliance with the protocol if any listed header is missing from the serialized response or if the expected header value differs from the serialized response value. `headers` applies no constraints on additional headers.
forbidHeaders	`list<string>`	A list of header field names that must not appear in the serialized HTTP response.
requireHeaders	`list<string>`	A list of header field names that must appear in the serialized HTTP response, but no assertion is made on the value. Headers listed in `headers` do not need to appear in this list.
body	`string`	The expected HTTP message body. If no response body is defined, then no assertions are made about the body of the message.
bodyMediaType	`string`	The media type of the `body`. This is used to help test runners to parse and validate the expected data against generated data. Binary media type formats require that the contents of `body` are base64 encoded.
params	`document`	Defines the output or error parameters used to generate the HTTP response. These parameters MUST be compatible with the targeted operation's output or the targeted error structure.
vendorParams	`document`	Defines vendor-specific parameters that are used to influence the response. For example, some vendors might utilize environment variables, configuration files on disk, or other means to influence the serialization formats used by clients or servers.
documentation	`string`	A description of the test and what is being asserted defined in CommonMark.

HTTP response example¶

The following example defines a protocol compliance test for a JSON protocol that uses HTTP binding traits.

namespace smithy.example

use smithy.test#httpResponseTests

@http(method: "POST", uri: "/")
@httpResponseTests([
    {
        id: "say_goodbye",
        protocol: "example",
        params: {farewell: "Bye"},
        code: 200,
        headers: {
            "X-Farewell": "Bye",
            "Content-Length": "0"
        }
    }
])
operation SayGoodbye {
    output: SayGoodbyeOutput
}

structure SayGoodbyeOutput {
    @httpHeader("X-Farewell")
    farewell: String,
}

{
    "smithy": "0.5.0",
    "shapes": {
        "smithy.example#SayGoodbye": {
            "type": "operation",
            "output": {
                "target": "smithy.example#SayGoodbyeOutput"
            },
            "traits": {
                "smithy.api#http": {
                    "method": "POST",
                    "uri": "/",
                    "code": 200
                },
                "smithy.test#httpResponseTests": [
                    {
                        "id": "say_goodbye",
                        "protocol": "example",
                        "headers": {
                            "Content-Length": "0",
                            "X-Farewell": "Bye"
                        },
                        "params": {
                            "farewell": "Bye"
                        },
                        "code": 200
                    }
                ]
            }
        },
        "smithy.example#SayGoodbyeOutput": {
            "type": "structure",
            "members": {
                "farewell": {
                    "target": "smithy.api#String",
                    "traits": {
                        "smithy.api#httpHeader": "X-Farewell"
                    }
                }
            }
        }
    }
}

HTTP error response example¶

The httpResponseTests trait can be applied to error structures to define how an error HTTP response is serialized. Client protocol compliance test implementations SHOULD ensure that each error with the httpResponseTests trait associated with an operation can be properly deserialized.

The following example defines a protocol compliance test for a JSON protocol that uses HTTP binding traits.

namespace smithy.example

use smithy.test#httpResponseTests

@error("client")
@httpError(400)
@httpResponseTests([
    {
        id: "invalid_greeting",
        protocol: "example",
        params: {foo: "baz", message: "Hi"},
        code: 400,
        headers: {"X-Foo": "baz"},
        body: "{\"message\": \"Hi\"}",
        bodyMediaType: "application/json",
    }
])
structure InvalidGreeting {
    @httpHeader("X-Foo")
    foo: String,

    message: String,
}

{
    "smithy": "0.5.0",
    "shapes": {
        "smithy.example#InvalidGreeting": {
            "type": "structure",
            "members": {
                "foo": {
                    "target": "smithy.api#String",
                    "traits": {
                        "smithy.api#httpHeader": "X-Foo"
                    }
                },
                "message": {
                    "target": "smithy.api#String"
                }
            },
            "traits": {
                "smithy.api#error": "client",
                "smithy.api#httpError": 400,
                "smithy.test#httpResponseTests": [
                    {
                        "id": "invalid_greeting",
                        "protocol": "example",
                        "body": "{\"message\": \"Hi\"}",
                        "bodyMediaType": "application/json",
                        "headers": {
                            "X-Foo": "baz"
                        },
                        "params": {
                            "foo": "baz",
                            "message": "Hi"
                        },
                        "code": 400
                    }
                ]
            }
        }
    }
}