software.amazon.smithy.model.validation.testrunner

Class SmithyTestSuite

java.lang.Object

software.amazon.smithy.model.validation.testrunner.SmithyTestSuite

public final class SmithyTestSuite
extends java.lang.Object

Runs test cases against a directory of models and error files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SmithyTestSuite.Error Thrown when errors are encountered in the test runner.
static class	SmithyTestSuite.Result Value result of executing the test suite.

Method Summary

Modifier and Type	Method and Description
SmithyTestSuite	addTestCase(SmithyTestCase testCase) Adds a test case to the test suite.
SmithyTestSuite	addTestCasesFromDirectory(java.nio.file.Path modelDirectory) Adds test cases by crawling a directory and looking for model files that end with ".json" and ".smithy".
SmithyTestSuite	addTestCasesFromUrl(java.net.URL url) Convenience method for supplying a directory using a class loader.
static java.util.stream.Stream<java.lang.Object[]>	defaultParameterizedTestSource(java.lang.Class<?> contextClass) Factory method used to easily created a JUnit 5 ParameterizedTest MethodSource based on the given Class.
SmithyTestSuite.Result	run() Executes the test runner.
SmithyTestSuite.Result	run(java.util.concurrent.ExecutorService executorService) Executes the test runner with a specific ExecutorService.
static SmithyTestSuite	runner() Creates a new Smithy test suite.
SmithyTestSuite	setModelAssemblerFactory(java.util.function.Supplier<ModelAssembler> modelAssemblerFactory) Sets a custom ModelAssembler factory to use to create a ModelAssembler for each test case.
java.util.stream.Stream<java.util.concurrent.Callable<SmithyTestCase.Result>>	testCaseCallables() Creates a Stream of Callable objects that can be used to execute each test case.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

runner

public static SmithyTestSuite runner()

Creates a new Smithy test suite.

Returns:

Returns the created test suite.

defaultParameterizedTestSource

public static java.util.stream.Stream<java.lang.Object[]> defaultParameterizedTestSource(java.lang.Class<?> contextClass)

Factory method used to easily created a JUnit 5 ParameterizedTest MethodSource based on the given Class.

This method assumes that there is a resource named errorfiles relative to the given class that contains test cases. It also assumes validators and traits should be loaded using the ClassLoader of the given contextClass, and that model discovery should be used using the given contextClass.

Each returns Object[] contains the filename of the test as the first argument, followed by a Callable<SmithyTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

For example, the following can used as a unit test:

 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestCase;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable<SmithyTestCase.Result> callable) throws Exception {
         callable.call();
     }

     public static Stream<?> source() {
         return SmithyTestSuite.defaultParameterizedTestSource(TestRunnerTest.class);
     }
 }
 

Parameters:

contextClass - The class to use for loading errorfiles and model discovery.

Returns:

Returns the Stream that should be used as a JUnit 5 MethodSource return value.

addTestCase

public SmithyTestSuite addTestCase(SmithyTestCase testCase)

Adds a test case to the test suite.

Parameters:

testCase - Test case to add.

Returns:

Returns the test suite.

addTestCasesFromDirectory

public SmithyTestSuite addTestCasesFromDirectory(java.nio.file.Path modelDirectory)

Adds test cases by crawling a directory and looking for model files that end with ".json" and ".smithy". A corresponding ".errors" file is expected to be found for each found model file.

See SmithyTestCase.fromModelFile(java.lang.String) for a description of how the errors file is expected to be formatted.

Parameters:

modelDirectory - Directory that contains models.

Returns:

Returns the test suite.

See Also:

SmithyTestCase.fromModelFile(java.lang.String)

addTestCasesFromUrl

public SmithyTestSuite addTestCasesFromUrl(java.net.URL url)

Convenience method for supplying a directory using a class loader.

Parameters:

url - URL that contains models.

Returns:

Returns the test suite.

Throws:

java.lang.IllegalArgumentException - if a non-file scheme URL is provided.

See Also:

addTestCasesFromDirectory(java.nio.file.Path)

setModelAssemblerFactory

public SmithyTestSuite setModelAssemblerFactory(java.util.function.Supplier<ModelAssembler> modelAssemblerFactory)

Sets a custom ModelAssembler factory to use to create a ModelAssembler for each test case.

The supplier must return a new instance of a Model assembler each time it is called. Model assemblers are mutated and execute in parallel.

Parameters:

modelAssemblerFactory - Model assembler factory to use.

Returns:

Returns the test suite.

testCaseCallables

public java.util.stream.Stream<java.util.concurrent.Callable<SmithyTestCase.Result>> testCaseCallables()

Creates a Stream of Callable objects that can be used to execute each test case.

The SmithyTestCase.Result.unwrap() method must be called on the result of each callable in order to actually assert that the test case result is OK.

Returns:

Returns a stream of test case callables.

run

public SmithyTestSuite.Result run()

Executes the test runner.

Returns:

Returns the test case result object on success.

Throws:

java.lang.Error - if the validation events do not match expectations.

run

public SmithyTestSuite.Result run(java.util.concurrent.ExecutorService executorService)

Executes the test runner with a specific ExecutorService.

Tests ideally should use JUnit 5's ParameterizedTest as described in parameterizedTestSource(). However, this method can be used to run tests in parallel in other scenarios (like if you aren't using JUnit, or not running tests cases during unit tests).

Parameters:

executorService - Executor service to execute tests with.

Returns:

Returns the test case result object on success.

Throws:

java.lang.Error - if the validation events do not match expectations.