Package software.amazon.smithy.model.validation.testrunner

Class SmithyTestSuite

  • public final class SmithyTestSuite
extends java.lang.Object
    Runs test cases against a directory of models and error files.

    • Method Detail

      • runner

        public static SmithyTestSuite runner()
        Creates a new Smithy test suite.
        Returns:
        Returns the created test suite.

      • defaultParameterizedTestSource

        public static java.util.stream.Stream<java.lang.Object[]> defaultParameterizedTestSource​(java.lang.Class<?> contextClass)
        Factory method used to easily create a JUnit 5 ParameterizedTest MethodSource based on the given Class.

        This method assumes that there is a resource named errorfiles relative to the given class that contains test cases. It also assumes validators and traits should be loaded using the ClassLoader of the given contextClass, and that model discovery should be used using the given contextClass.

        Each returned Object[] contains the filename of the test as the first argument, followed by a Callable<SmithyTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

        For example, the following can be used as a unit test: 

        
 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestCase;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable&lt;SmithyTestCase.Result&gt; callable) throws Exception {
         callable.call();
     }

     public static Stream&lt;?&gt; source() {
         return SmithyTestSuite.defaultParameterizedTestSource(TestRunnerTest.class);
     }
 }
        Parameters:
        contextClass - The class to use for loading errorfiles and model discovery.
        Returns:
        Returns the Stream that should be used as a JUnit 5 MethodSource return value.

      • parameterizedTestSource

        public java.util.stream.Stream<java.lang.Object[]> parameterizedTestSource()
        Factory method used to create a JUnit 5 ParameterizedTest MethodSource.

        Test cases need to be added to the test suite before calling this, for example by using addTestCasesFromDirectory(Path).

        Each returned Object[] contains the filename of the test as the first argument, followed by a Callable<SmithyTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

        For example, the following can be used as a unit test: 

        
 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestCase;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable&lt;SmithyTestCase.Result&gt; callable) throws Exception {
         callable.call();
     }

     public static Stream&lt;?&gt; source() {
         ModelAssembler assembler = Model.assembler(TestRunnerTest.class.getClassLoader());
         return SmithyTestSuite.runner()
                 .setModelAssemblerFactory(assembler::copy)
                 .addTestCasesFromUrl(TestRunnerTest.class.getResource("errorfiles"))
                 .parameterizedTestSource();
     }
 }
        Returns:
        Returns the Stream that should be used as a JUnit 5 MethodSource return value.

      • addTestCase

        public SmithyTestSuite addTestCase​(SmithyTestCase testCase)
        Adds a test case to the test suite.
        Parameters:
        testCase - Test case to add.
        Returns:
        Returns the test suite.

      • addTestCasesFromDirectory

        public SmithyTestSuite addTestCasesFromDirectory​(java.nio.file.Path modelDirectory)
        Adds test cases by crawling a directory and looking for model files that end with ".json" and ".smithy". A corresponding ".errors" file is expected to be found for each found model file.

        See SmithyTestCase.fromModelFile(java.lang.String) for a description of how the errors file is expected to be formatted.

        Parameters:
        modelDirectory - Directory that contains models.
        Returns:
        Returns the test suite.
        See Also:
        SmithyTestCase.fromModelFile(java.lang.String)

      • addTestCasesFromUrl

        public SmithyTestSuite addTestCasesFromUrl​(java.net.URL url)
        Convenience method for supplying a directory using a class loader.
        Parameters:
        url - URL that contains models.
        Returns:
        Returns the test suite.
        Throws:
        java.lang.IllegalArgumentException - if a non-file scheme URL is provided.
        See Also:
        addTestCasesFromDirectory(java.nio.file.Path)

      • setModelAssemblerFactory

        public SmithyTestSuite setModelAssemblerFactory​(java.util.function.Supplier<ModelAssembler> modelAssemblerFactory)
        Sets a custom ModelAssembler factory to use to create a ModelAssembler for each test case.

        The supplier must return a new instance of a Model assembler each time it is called. Model assemblers are mutated and execute in parallel.

        Parameters:
        modelAssemblerFactory - Model assembler factory to use.
        Returns:
        Returns the test suite.

      • testCaseCallables

        public java.util.stream.Stream<java.util.concurrent.Callable<SmithyTestCase.Result>> testCaseCallables()
        Creates a Stream of Callable objects that can be used to execute each test case.

        The SmithyTestCase.Result.unwrap() method must be called on the result of each callable in order to actually assert that the test case result is OK.

        Returns:
        Returns a stream of test case callables.

      • run

        public SmithyTestSuite.Result run​(java.util.concurrent.ExecutorService executorService)
        Executes the test runner with a specific ExecutorService.

        Tests ideally should use JUnit 5's ParameterizedTest as described in parameterizedTestSource(). However, this method can be used to run tests in parallel in other scenarios (like if you aren't using JUnit, or not running tests cases during unit tests).

        Parameters:
        executorService - Executor service to execute tests with.
        Returns:
        Returns the test case result object on success.
        Throws:
        SmithyTestSuite.Error - if the validation events do not match expectations.