Package software.amazon.smithy.diff.testrunner

Class SmithyDiffTestSuite

java.lang.Object
software.amazon.smithy.diff.testrunner.SmithyDiffTestSuite
public final class SmithyDiffTestSuite extends Object
Runs diff test cases against corresponding model `a`, model `b`, and validation `events` files.

  • Method Details

    • runner

      public static SmithyDiffTestSuite runner()
      Creates a new Smithy diff test suite.
      Returns:
      Returns the created test suite.

    • defaultParameterizedTestSource

      public static Stream<Object[]> defaultParameterizedTestSource(Class<?> contextClass)
      Factory method used to easily create a JUnit 5 ParameterizedTest MethodSource based on the given Class.

      This method assumes that there is a resource named diffs relative to the given class that contains test cases. It also assumes validators and traits should be loaded using the ClassLoader of the given contextClass, and that model discovery should be used using the given contextClass.

      Each returned Object[] contains the filename of the test as the first argument, followed by a Callable<SmithyDiffTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

      For example, the following can be used as a unit test: 

      
 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.diff.testrunner.SmithyDiffTestCase;
 import software.amazon.smithy.diff.testrunner.SmithyDiffTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable&lt;SmithyDiffTestCase.Result&gt; callable)
         throws Exception {
         callable.call();
     }

     public static Stream&lt;?&gt; source() {
         return SmithyDiffTestSuite.defaultParameterizedTestSource(TestRunnerTest.class);
     }
 }
      Parameters:
      contextClass - The class to use for loading diffs and model discovery.
      Returns:
      Returns the Stream that should be used as a JUnit 5 MethodSource return value.

    • parameterizedTestSource

      public Stream<Object[]> parameterizedTestSource()
      Factory method used to create a JUnit 5 ParameterizedTest MethodSource.

      Test cases need to be added to the test suite before calling this, for example by using addTestCasesFromDirectory(Path).

      Each returned Object[] contains the name of the test as the first argument, followed by a Callable<SmithyDiffTestCase.Result> as the second argument. All a parameterized test needs to do is call call on the provided Callable to execute the test and fail if the test case is invalid.

      For example, the following can be used as a unit test: 

      
 import java.util.concurrent.Callable;
 import java.util.stream.Stream;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import software.amazon.smithy.diff.testrunner.SmithyDiffTestCase;
 import software.amazon.smithy.diff.testrunner.SmithyDiffTestSuite;

 public class TestRunnerTest {
     \@ParameterizedTest(name = "\{0\}")
     \@MethodSource("source")
     public void testRunner(String filename, Callable&lt;SmithyDiffTestCase.Result&gt; callable)
         throws Exception {
         callable.call();
     }

     public static Stream&lt;?&gt; source() {
         ModelAssembler assembler = Model.assembler(TestRunnerTest.class.getClassLoader());
         return SmithyDiffTestSuite.runner()
                 .setModelAssemblerFactory(assembler::copy)
                 .addTestCasesFromUrl(TestRunnerTest.class.getResource("errorfiles"))
                 .parameterizedTestSource();
     }
 }
      Returns:
      Returns the Stream that should be used as a JUnit 5 MethodSource return value.

    • addTestCase

      public SmithyDiffTestSuite addTestCase(SmithyDiffTestCase testCase)
      Adds a test case to the test suite.
      Parameters:
      testCase - Test case to add.
      Returns:
      Returns the test suite.

    • addTestCasesFromDirectory

      public SmithyDiffTestSuite addTestCasesFromDirectory(Path modelDirectory)
      Adds test cases by crawling a directory and looking for events files that end with ".events". Corresponding ".a.(json|smithy)" and ".b.(json|smithy)" files are expected to be found for each found events file.

      See SmithyDiffTestCase.from(java.nio.file.Path, java.lang.String) for a description of how the events file is expected to be formatted.

      Parameters:
      modelDirectory - Directory that contains diff models.
      Returns:
      Returns the test suite.
      See Also:

    • addTestCasesFromUrl

      public SmithyDiffTestSuite addTestCasesFromUrl(URL url)
      Convenience method for supplying a directory using a class loader.
      Parameters:
      url - URL that contains diff models.
      Returns:
      Returns the test suite.
      Throws:
      IllegalArgumentException - if a non-file scheme URL is provided.
      See Also:

    • setModelAssemblerFactory

      public SmithyDiffTestSuite setModelAssemblerFactory(Supplier<ModelAssembler> modelAssemblerFactory)
      Sets a custom ModelAssembler factory to use to create a ModelAssembler for each test case.

      The supplier must return a new instance of a Model assembler each time it is called. Model assemblers are mutated and execute in parallel.

      Parameters:
      modelAssemblerFactory - Model assembler factory to use.
      Returns:
      Returns the test suite.

    • testCaseCallables

      public Stream<Callable<SmithyDiffTestCase.Result>> testCaseCallables()
      Creates a Stream of Callable objects that can be used to execute each test case.

      The SmithyDiffTestCase.Result.unwrap() method must be called on the result of each callable in order to actually assert that the test case result is OK.

      Returns:
      Returns a stream of test case callables.

    • run

      public SmithyDiffTestSuite.Result run()
      Executes the test runner.
      Returns:
      Returns the test case result object on success.
      Throws:
      SmithyDiffTestSuite.Error - if the validation events do not match expectations.

    • run

      public SmithyDiffTestSuite.Result run(ExecutorService executorService)
      Executes the test runner with a specific ExecutorService.

      Tests ideally should use JUnit 5's ParameterizedTest as described in parameterizedTestSource(). However, this method can be used to run tests in parallel in other scenarios (like if you aren't using JUnit, or not running tests cases during unit tests).

      Parameters:
      executorService - Executor service to execute tests with.
      Returns:
      Returns the test case result object on success.
      Throws:
      SmithyDiffTestSuite.Error - if the validation events do not match expectations.