# Class: com.pnfsoftware.jeb.util.format.TokenExtractor

A customizable string token extractor. Standard delimiters are provided: see `DF_xxx` objects. 

 Example: Consider the line `"hello world"`. If you use a [wsp token extractor](#DF_WhiteSpace), asking for which token is at:
 \- offset 0,1,2,3,4 would return "hello".
 \- offset 5 would return null \(it falls on the space char\).
 \- offset 6,7,8,9,10 would return "world".
 \- anywhere else, null would be returned.

## Constructor: TokenExtractor
- parameter: `delimiterFinder`, type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`

Description: Create a token extractor.
parameter: delimiterFinder: a delimiter finder; client code may either create their own or use one            of the standard finders, see `DF_*`

## Static Field: DF_Assignment
Type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`
Description: Delimiter= '=' character. In particular on decompilation comment with pattern when x=myvalue

## Static Field: DF_CommonFullSymbolChars
Type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`
Description: Delimiter= any character that is generally not legal in a symbol name, in particular: 
 
- any character allowed in java identifier
-  
- additional '/' character for java signature
-  
- additional '\+' character for native relative signature
-

## Static Field: DF_CommonSymbolChars
Type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`
Description: Delimiter= any character that is generally not legal in a symbol name, in particular: 
 
- any character allowed in java identifier
-  
- all other characters are delimiters
-

## Static Field: DF_NonAlphaNum
Type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`
Description: Delimiter= any character that is neither a letter nor a digit.

## Static Field: DF_WhiteSpace
Type: `com.pnfsoftware.jeb.util.format.TokenExtractor.IDelimiterFinder`
Description: Delimiter= whitespace characters \(per [isWhitespace](Character#isWhitespace(char))\)

## Method: extract
- parameter: `s`, type: `java.lang.String`
- parameter: `o`, type: `int`
- return type: `java.lang.String`

Description: Extract the token spanning over the provided offset of the input string.
parameter: s: input string
parameter: o: start offset
return: the token as a substring of s or null if not found.

## Method: extractCoordinates
- parameter: `s`, type: `java.lang.String`
- parameter: `o`, type: `int`
- return type: `int[]`

Description: Extract the token spanning over the provided offset of the input string.
parameter: s: input string
parameter: o: start offset
return: an array with 2 indexes \[start, end\] of the token, or null if not found.

## Protected Method: match
- parameter: `s`, type: `java.lang.String`
- parameter: `index`, type: `int`
- return type: `boolean`


## Static Method: getGenericTokenList
- return type: `java.util.List<com.pnfsoftware.jeb.util.format.TokenExtractor>`

Description: Retrieve the generic list of Token Extractor, based on DF\_\* [IDelimiterFinder](IDelimiterFinder). List is sorted from least to most restrictive.
return: token extractor list