Package com.pnfsoftware.jeb.util.encoding.jflex.lexers

Class PythonLexer

java.lang.Object
com.pnfsoftware.jeb.util.encoding.jflex.DefaultJFlexLexer
com.pnfsoftware.jeb.util.encoding.jflex.lexers.PythonLexer
public final class PythonLexer extends DefaultJFlexLexer

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    ML_STRING
     
    static final int
    SQML_STRING
     
    static final int
    SQSTRING
     
    static final int
    STRING
     
    static final int
    YYEOF
    This character denotes the end of file.
    static final int
    YYINITIAL
     

    Fields inherited from class com.pnfsoftware.jeb.util.encoding.jflex.DefaultJFlexLexer

    offset, tokenLength, tokenStart

  • Constructor Summary

    Constructors
    Constructor
    Description
    PythonLexer()
    Create an empty lexer, yyrset will be called later to reset and assign the reader
    PythonLexer(Reader in)
    Creates a new scanner

  • Method Summary

    Modifier and Type
    Method
    Description
    final boolean
    yyatEOF()
    Returns whether the scanner has reached the end of the reader it reads from.
    final void
    yybegin(int newState)
    Enters a new lexical state.
    long
    yychar()
    Return the char number from beginning of input stream.
    final char
    yycharat(int position)
    Returns the character at the given position from the matched text.
    final void
    yyclose()
    Closes the input reader.
    final int
    yylength()
    How many characters were matched.
    Token
    yylex()
    Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
    void
    yypushback(int number)
    Pushes the specified amount of characters back into the input stream.
    final void
    yyreset(Reader reader)
    Resets the scanner to read from a new input stream.
    final int
    yystate()
    Returns the current lexical state.
    final String
    yytext()
    Returns the text matched by the current regular expression.

    Methods inherited from class com.pnfsoftware.jeb.util.encoding.jflex.DefaultJFlexLexer

    parse, parse, token, token, token, token

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • PythonLexer

      public PythonLexer()
      Create an empty lexer, yyrset will be called later to reset and assign the reader

    • PythonLexer

      public PythonLexer(Reader in)
      Creates a new scanner
      Parameters:
      in - the java.io.Reader to read input from.

  • Method Details

    • yychar

      public long yychar()
      Description copied from class: DefaultJFlexLexer
      Return the char number from beginning of input stream. This is NOT implemented by JFlex, so the code must be added to create this and return the private yychar field
      Specified by:
      yychar in class DefaultJFlexLexer

    • yyclose

      public final void yyclose() throws IOException
      Closes the input reader.
      Throws:
      IOException - if the reader could not be closed.

    • yyreset

      public final void yyreset(Reader reader)
      Resets the scanner to read from a new input stream.

      Does not close the old reader.

      All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.

      Internal scan buffer is resized down to its initial length, if it has grown.

      Specified by:
      yyreset in class DefaultJFlexLexer
      Parameters:
      reader - The new input stream.

    • yyatEOF

      public final boolean yyatEOF()
      Returns whether the scanner has reached the end of the reader it reads from.
      Returns:
      whether the scanner has reached EOF.

    • yystate

      public final int yystate()
      Returns the current lexical state.
      Returns:
      the current lexical state.

    • yybegin

      public final void yybegin(int newState)
      Enters a new lexical state.
      Parameters:
      newState - the new lexical state

    • yytext

      public final String yytext()
      Returns the text matched by the current regular expression.
      Specified by:
      yytext in class DefaultJFlexLexer
      Returns:
      the matched text.

    • yycharat

      public final char yycharat(int position)
      Returns the character at the given position from the matched text.

      It is equivalent to yytext().charAt(pos), but faster.

      Specified by:
      yycharat in class DefaultJFlexLexer
      Parameters:
      position - the position of the character to fetch. A value from 0 to yylength()-1.
      Returns:
      the character at position.

    • yylength

      public final int yylength()
      How many characters were matched.
      Specified by:
      yylength in class DefaultJFlexLexer
      Returns:
      the length of the matched text region.

    • yypushback

      public void yypushback(int number)
      Pushes the specified amount of characters back into the input stream.

      They will be read again by then next call of the scanning method.

      Parameters:
      number - the number of characters to be read again. This number must not be greater than yylength().

    • yylex

      public Token yylex() throws IOException
      Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
      Specified by:
      yylex in class DefaultJFlexLexer
      Returns:
      the next token.
      Throws:
      IOException - if any I/O-Error occurs.