Package software.amazon.smithy.model.validation.testrunner

Class SmithyTestSuite

java.lang.Object
software.amazon.smithy.model.validation.testrunner.SmithyTestSuite
public final class SmithyTestSuite extends Object
Runs test cases against a directory of models and error files.

  • Method Details

    • runner

      public static SmithyTestSuite runner()
      Creates a new Smithy test suite.
      Returns:
      Returns the created test suite.

    • defaultParameterizedTestSource

      public static Stream<Object[]> defaultParameterizedTestSource(Class<?> contextClass)
      Factory method used to easily create a JUnit 5 ParameterizedTest MethodSource based on the given Class.

      This method assumes that there is a resource named errorfiles relative to the given class that contains test cases. It also assumes validators and traits should be loaded using the ClassLoader of the given contextClass, and that model discovery should be used using the given contextClass.

      Each returns Object[] contains the filename of the test as the first argument, followed by a Callable<SmithyTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

      For example, the following can used as a unit test: 

      
 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestCase;
 import software.amazon.smithy.model.validation.testrunner.SmithyTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable&lt;SmithyTestCase.Result&gt; callable) throws Exception {
         callable.call();
     }

     public static Stream&lt;?&gt; source() {
         return SmithyTestSuite.defaultParameterizedTestSource(TestRunnerTest.class);
     }
 }
      Parameters:
      contextClass - The class to use for loading errorfiles and model discovery.
      Returns:
      Returns the Stream that should be used as a JUnit 5 MethodSource return value.

    • addTestCase

      public SmithyTestSuite addTestCase(SmithyTestCase testCase)
      Adds a test case to the test suite.
      Parameters:
      testCase - Test case to add.
      Returns:
      Returns the test suite.

    • addTestCasesFromDirectory

      public SmithyTestSuite addTestCasesFromDirectory(Path modelDirectory)
      Adds test cases by crawling a directory and looking for model files that end with ".json" and ".smithy". A corresponding ".errors" file is expected to be found for each found model file.

      See SmithyTestCase.fromModelFile(java.lang.String) for a description of how the errors file is expected to be formatted.

      Parameters:
      modelDirectory - Directory that contains models.
      Returns:
      Returns the test suite.
      See Also:

    • addTestCasesFromUrl

      public SmithyTestSuite addTestCasesFromUrl(URL url)
      Convenience method for supplying a directory using a class loader.
      Parameters:
      url - URL that contains models.
      Returns:
      Returns the test suite.
      Throws:
      IllegalArgumentException - if a non-file scheme URL is provided.
      See Also:

    • setModelAssemblerFactory

      public SmithyTestSuite setModelAssemblerFactory(Supplier<ModelAssembler> modelAssemblerFactory)
      Sets a custom ModelAssembler factory to use to create a ModelAssembler for each test case.

      The supplier must return a new instance of a Model assembler each time it is called. Model assemblers are mutated and execute in parallel.

      Parameters:
      modelAssemblerFactory - Model assembler factory to use.
      Returns:
      Returns the test suite.

    • testCaseCallables

      public Stream<Callable<SmithyTestCase.Result>> testCaseCallables()
      Creates a Stream of Callable objects that can be used to execute each test case.

      The SmithyTestCase.Result.unwrap() method must be called on the result of each callable in order to actually assert that the test case result is OK.

      Returns:
      Returns a stream of test case callables.

    • run

      public SmithyTestSuite.Result run()
      Executes the test runner.
      Returns:
      Returns the test case result object on success.
      Throws:
      SmithyTestSuite.Error - if the validation events do not match expectations.

    • run

      public SmithyTestSuite.Result run(ExecutorService executorService)
      Executes the test runner with a specific ExecutorService.

      Tests ideally should use JUnit 5's ParameterizedTest as described in parameterizedTestSource(). However, this method can be used to run tests in parallel in other scenarios (like if you aren't using JUnit, or not running tests cases during unit tests).

      Parameters:
      executorService - Executor service to execute tests with.
      Returns:
      Returns the test case result object on success.
      Throws:
      SmithyTestSuite.Error - if the validation events do not match expectations.