Package software.amazon.smithy.syntax

Class CapturedToken

  • All Implemented Interfaces:
    FromSourceLocation, ToSmithyBuilder<CapturedToken>
    public final class CapturedToken
extends java.lang.Object
implements FromSourceLocation, ToSmithyBuilder<CapturedToken>
    A persisted token captured from an IdlTokenizer.

    For performance, IdlTokenizer does not create new tokens types for each encountered token. Instead, it updates the current state of the tokenizer and allows the caller to inspect the tokenizer for information about each token. Because smithy-syntax needs to create a token-tree rather than go directly to an AST, it requires arbitrary lookahead of tokens, requiring it to persist tokens in memory.

    • Method Detail

      • from

        public static CapturedToken from​(IdlTokenizer tokenizer)
        Persist the current token of an IdlTokenizer.
        Parameters:
        tokenizer - Tokenizer to capture.
        Returns:
        Returns the persisted token.

      • from

        public static CapturedToken from​(IdlTokenizer tokenizer,
                                 java.util.function.Function<java.lang.CharSequence,​java.lang.String> stringTable)
        Persist the current token of an IdlTokenizer.
        Parameters:
        tokenizer - Tokenizer to capture.
        stringTable - String table that caches previously created strings.
        Returns:
        Returns the persisted token.

      • getIdlToken

        public IdlToken getIdlToken()
        Get the token IDL token of the captured token.
        Returns:
        Returns the underlying token type.

      • getFilename

        public java.lang.String getFilename()

      • getPosition

        public int getPosition()

      • getStartLine

        public int getStartLine()

      • getStartColumn

        public int getStartColumn()

      • getEndLine

        public int getEndLine()

      • getEndColumn

        public int getEndColumn()

      • getSpan

        public int getSpan()

      • getLexeme

        public java.lang.CharSequence getLexeme()
        Get the raw lexeme of the current token.
        Returns:
        Returns the underlying lexeme of the token.

      • getStringContents

        public java.lang.String getStringContents()
        Get the associated String contents of the token if it's a string, text block, or identifier.
        Returns:
        Returns the string contents of the lexeme, or null if not a string|text block|identifier.

      • getErrorMessage

        public java.lang.String getErrorMessage()
        Gets the associated error message with the token if it's an error.
        Returns:
        Returns the error message or null if not an error.

      • getNumberValue

        public java.lang.Number getNumberValue()
        Get the computed Number of the current token if it's a number.
        Returns:
        Returns the computed Number or null if not a number.