Package software.amazon.smithy.model.loader

Interface IdlTokenizer

All Superinterfaces:
Iterator<IdlToken>
public interface IdlTokenizer extends Iterator<IdlToken>
Iterates over a Smithy IDL model as a series of tokens.

  • Method Details

    • create

      static IdlTokenizer create(CharSequence model)
      Create a tokenizer for the given model.
      Parameters:
      model - IDL model contents to parse.
      Returns:
      Returns the tokenizer.

    • create

      static IdlTokenizer create(String filename, CharSequence model)
      Create a tokenizer for the given filename and model.
      Parameters:
      filename - Filename being parsed.
      model - IDL model contents to parse.
      Returns:
      Returns the tokenizer.

    • getSourceFilename

      String getSourceFilename()
      Get the filename of the content being tokenized.
      Returns:
      Returns the filename used in source locations.

    • getModel

      CharSequence getModel()
      Get the model being tokenized.
      Returns:
      Returns the model.

    • getModel

      default CharSequence getModel(int start, int end)
      Get a borrowed slice of the model being tokenized.
      Parameters:
      start - Start position to get, inclusive.
      end - End position to stop at, exclusive.
      Returns:
      Returns the slice.

    • getPosition

      int getPosition()
      Get the current position of the tokenizer.
      Returns:
      Returns the absolute position.

    • getLine

      int getLine()
      Get the current line number of the tokenizer, starting at 1.
      Returns:
      Get the current line number.

    • getColumn

      int getColumn()
      Get the current column number of the tokenizer, starting at 1.
      Returns:
      Get the current column number.

    • getCurrentToken

      IdlToken getCurrentToken()
      Get the current IdlToken.
      Returns:
      Return the current token type.

    • getCurrentTokenLine

      int getCurrentTokenLine()
      Get the line of the current token.
      Returns:
      Return the line of the current token.

    • getCurrentTokenColumn

      int getCurrentTokenColumn()
      Get the column of the current token.
      Returns:
      Return the column of the current token.

    • getCurrentTokenStart

      int getCurrentTokenStart()
      Get the start position of the current token.
      Returns:
      Return the 0-based start position of the current token.

    • getCurrentTokenEnd

      int getCurrentTokenEnd()
      Get the end position of the curren token.
      Returns:
      Return the 0-based end position of the current token.

    • getCurrentTokenSpan

      default int getCurrentTokenSpan()
      Get the length of the current token.
      Returns:
      Return the current token span.

    • getCurrentTokenLocation

      default SourceLocation getCurrentTokenLocation()
      Get the source location of the current token.
      Returns:
      Return the current token source location.

    • getCurrentTokenLexeme

      default CharSequence getCurrentTokenLexeme()
      Get the lexeme of the current token.
      Returns:
      Returns the lexeme of the current token.

    • getCurrentTokenStringSlice

      CharSequence getCurrentTokenStringSlice()
      If the current token is a string or text block, get the parsed content as a CharSequence. If the current token is an identifier, the lexeme of the identifier is returned.
      Returns:
      Returns the parsed string content associated with the current token.
      Throws:
      ModelSyntaxException - if the current token is not a string, text block, or identifier.

    • getCurrentTokenNumberValue

      Number getCurrentTokenNumberValue()
      If the current token is a number, get the associated parsed number.
      Returns:
      Returns the parsed number associated with the current token.
      Throws:
      ModelSyntaxException - if the current token is not a number.

    • getCurrentTokenError

      String getCurrentTokenError()
      If the current token is an error, get the error message associated with the token.
      Returns:
      Returns the associated error message.
      Throws:
      ModelSyntaxException - if the current token is not an error.

    • expect

      default void expect(IdlToken token)
      Assert that the current token is token.

      The tokenizer is not advanced after validating the current token.

      Parameters:
      token - Token to expect.
      Throws:
      ModelSyntaxException - if the current token is unexpected.

    • expect

      default IdlToken expect(IdlToken... tokens)
      Assert that the current token is one of tokens.

      The tokenizer is not advanced after validating the current token.

      Parameters:
      tokens - Assert that the current token is one of these tokens.
      Returns:
      Returns the current token.
      Throws:
      ModelSyntaxException - if the current token is unexpected.

    • isCurrentLexeme

      default boolean isCurrentLexeme(CharSequence chars)
      Test if the current token lexeme is equal to the give chars.
      Parameters:
      chars - Characters to compare the current lexeme against.
      Returns:
      Returns true if the current lexeme is equal to chars.