Smoke Tests#

Smoke tests are small, simple tests intended to uncover large issues by ensuring core functionality works as expected. In Smithy, these tests are used to make sure clients can successfully make a request to a live service, and that the service responds with the right kind of response.


This specification defines a single trait in the smithy.test namespace that is used to make basic assertions about how the service should respond to a given request from the client.

Defines a set of test cases to send to a live service to ensure that a client can successfully connect to a service and get the right kind of response.

This trait can be used by client code generators to generate test cases that test generated clients against live services to ensure core functionality is working, and continues to work as the client and service evolve.


The smokeTests trait is used to define a set of test cases to send to a live service to ensure that a client can successfully connect to a service and receives the right kind of response.
Trait selector
Value type
list of SmokeTestCase structures

The smokeTests trait is a list of SmokeTestCase structures.


A structure defining a smoke test case.

Property Type Description
id string Required. The identifier of the test case. This identifier may be used by smoke test implementations to generate test case names. The provided id MUST match Smithy's smithy:Identifier ABNF. No two test cases can share the same ID, including test cases defined for other operations bound to the same service.
params document

Defines the input parameters used to generate the request. These parameters MUST be compatible with the input of the operation.

Parameter values that contain binary data MUST be defined using values that can be represented in plain text as the plain text representation (for example, use "foo" and not "Zm9vCg==").

vendorParams document

Defines vendor-specific parameters that are used to influence the request. For example, some vendors might utilize environment variables, configuration files on disk, or other means to influence the serialization formats used by clients or servers.

If vendorParamsShape is set, these parameters MUST be compatible with that shape's definition.

vendorParamsShape string

The ID of the shape that should be used to validate the vendorParams member contents.

If set, the parameters in vendorParams MUST be compatible with this shape's definition.

expect Expectation Required. Defines the kind of response that is expected from the service call.
tags [string] Attaches a list of tags that can be used to categorize and group test cases. If a test case uses a feature that requires special configuration, it should be tagged.


A union describing the different kinds of expectations that can be made for a test case. Exactly one member must be set.

Property Type Description
success Unit Indicates that the call is expected to not throw an error. No other assertions are made about the response.
failure FailureExpectation Indicates that the call is expected to throw an error.
Property Type Description
errorId string Indicates that the call is expected to throw a specific type of error matching the targeted shape. If not specified, the error can be of any type.

Smoke tests example#

The following example defines smoke test cases for an operation which should return a specific InvalidMessageError response when given an invalid input, and return a successful response otherwise.

$version: "2"

namespace smithy.example

use smithy.test#smokeTests

            id: "FooSuccess"
            params: {bar: "2"}
            expect: {
                success: {}
            id: "FooInvalidMessageError"
            params: {bar: "föö"}
            expect: {
                failure: {errorId: InvalidMessageError}
operation Foo {
    input := {
        bar: String
    errors: [

structure InvalidMessageError {}