Tokenizer<TSymbol,TSymbolType> Class
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.
public abstract class Tokenizer<TSymbol,TSymbolType> : System.Web.Razor.StateMachine<TSymbol>, System.Web.Razor.Tokenizer.ITokenizer where TSymbol : SymbolBase<TSymbolType>
type Tokenizer<'Symbol, 'SymbolType (requires 'Symbol :> SymbolBase<'SymbolType>)> = class
inherit StateMachine<'Symbol (requires 'Symbol :> SymbolBase<'SymbolType>)>
interface ITokenizer
Public MustInherit Class Tokenizer(Of TSymbol, TSymbolType)
Inherits StateMachine(Of TSymbol)
Implements ITokenizer
Type Parameters
- TSymbol
The type for the language symbol.
- TSymbolType
The enumeration type for the language symbol.
- Inheritance
- Derived
- Implements
Constructors
Tokenizer<TSymbol,TSymbolType>(ITextDocument) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Initializes a new instance of the Tokenizer<TSymbol,TSymbolType> class. |
Properties
Buffer |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets or sets the buffer for the tokenizer. |
CurrentCharacter |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the current character in the tokenizer. |
CurrentErrors |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets a list of the current razor errors. |
CurrentLocation |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the current source location. |
CurrentStart |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the current start of the source location. |
CurrentState |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets or sets the current state of the machine. (Inherited from StateMachine<TReturn>) |
EndOfFile |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets a value whether the tokenizer current location is at the end of the file. |
HaveContent |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets a value whether the tokenizer have content. |
RazorCommentStarType |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the star type for the razor comment. |
RazorCommentTransitionType |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the transition type for the razor comment. |
RazorCommentType |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the type of razor comment. |
Source |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the source of the text document. |
StartState |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Gets the starting state of the machine. (Inherited from StateMachine<TReturn>) |
Methods
AfterRazorCommentTransition() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Returns a result after the razor comment transition. |
At(String, Boolean) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Determines whether the lookahead buffer contains the expected string. |
CharOrWhiteSpace(Char) |
Returns a function delegate, that accepts a character parameter and returns a value that indicates whether the character parameter is equal to specified character or white space. |
CreateSymbol(SourceLocation, String, TSymbolType, IEnumerable<RazorError>) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Creates a language symbol type for the tokenizer with the specified content. |
EndSymbol(SourceLocation, TSymbolType) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Returns the language end symbol type used by the tokenizer. |
EndSymbol(TSymbolType) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Returns the language end symbol type used by the tokenizer. |
MoveNext() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Reads to the next character from the code reader. |
NextSymbol() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Shows the next symbol to be used. |
Peek() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Reads the next symbol in the code. |
RazorCommentBody() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Parses the Razor comment body. |
Reset() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Sets the tokenizer status to its initial state. |
ResumeSymbol(TSymbol) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Resumes using the previous language symbol type. |
Single(TSymbolType) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Uses a single type of symbol. |
StartSymbol() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Returns the start symbol used in this class. |
Stay() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Stays into the machine during the transition. (Inherited from StateMachine<TReturn>) |
Stay(TReturn) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Stays into the machine during the transition with the specified output. (Inherited from StateMachine<TReturn>) |
Stop() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Disables the machine upon transition. (Inherited from StateMachine<TReturn>) |
TakeAll(String, Boolean) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Takes the string if found in the lookahead buffer into the tokenizer buffer. |
TakeCurrent() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Accepts the current character into the buffer. |
TakeString(String, Boolean) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Accepts the given input string into the buffer. |
TakeUntil(Func<Char,Boolean>) |
Parses the source document until the condition specified by predicate is met or end file is reached. |
Transition(StateMachine<TReturn>.State) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Indicates the new transition of the state. (Inherited from StateMachine<TReturn>) |
Transition(TReturn, StateMachine<TReturn>.State) |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Indicates the new transition of the state with the specified output. (Inherited from StateMachine<TReturn>) |
Turn() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Describes the turning process of the state. (Inherited from StateMachine<TReturn>) |
Explicit Interface Implementations
ITokenizer.NextSymbol() |
This type/member supports the .NET Framework infrastructure and is not intended to be used directly from your code.Returns the next language symbol type. |