JML

org.multijava.mjc
Class ParsingController

java.lang.Object
  extended byorg.multijava.mjc.ParsingController

public class ParsingController
extends Object

This class manages the parsing of source code. It supports the use of multiple lexers and parsers to process nested languages (as in javadoc comments in Java programs). It includes most of the functionality of antlr.TokenStreamSelector, though for efficiency it duplicates the necessary code rather than inheriting it. This class buffers the tokens produced by multiple lexers and allows client parsers to accept tokens from just a single lexer.


Nested Class Summary
static class ParsingController.ConfigurationException
          This inner class represents exceptions that can be thrown because of misconfiguration of the parsing controller.
 class ParsingController.FilteredTokenStream
          This inner class provides a token stream that accesses one of the lexers managed by the controller.
(package private)  class ParsingController.IndexedTokenBuffer
          This inner class buffers tokens of a nested language and allows selection and discarding of a subset of these buffered tokens based on a reference to an index token of the surrounding language.
(package private) static class ParsingController.InputState
          This nested class is a hack to expose the line and column fields of LexerSharedInputState for reading.
static class ParsingController.KeyException
          This inner class represents exceptions that can be thrown because of bad key values passed as arguments to the outer classes methods.
(package private)  class ParsingController.StreamBufferPair
          This inner class encapsulates a buffer and an input stream so they can be the stored as a single value in a map.
static class ParsingController.TokenWrapper
          This inner class wraps a single token.
 
Field Summary
private  String activeKey
          The key of the currently active input stream.
private  ParsingController.StreamBufferPair activeStreamBuffer
          The currently active input stream and token buffer.
private  boolean discarding
          Indicates whether tokens for the currently active stream are to be discarded.
private  boolean endOfFileReached
          Indicates whether an end-of-file token has been received on some input stream.
private  boolean fetching
          Indicates that a token fetch request is in progress for the active input stream.
static antlr.Token IGNORE
          A token that may be returned by lexers upon detecting a lexical pattern that triggers a switch between lexers (e.g., /** for switching to javadoc).
private  ParsingController.FilteredTokenStream initialFilteredStream
          The output stream for the initially selected lexer.
private  boolean initialSet
          Indicates whether the initial input stream has been selected.
private  ParsingController.InputState inputState
          The input state that should be shared by the lexers bound to this parsing controller.
private  Stack keyStack
          For efficiency, used to track the keys when switching between input streams.
private  Set keysToDiscard
          The set of keys corresponding to lexers whose output are being discarded.
private  Map keyStreamBufferMap
          Maps keys to lexers and buffers
private  int ltCount
           
private  ParsingController.IndexedTokenBuffer nestedLanguageBuffer
          The active buffer before the last pop operation.
private static int NONE
          When operationPending equals this, then no operations are pending.
private  int operationPending
          Indicates whether any input stream switching operations (i.e., push, pop) are pending on completion of the current call to bufferNextToken.
private  String pendingKey
          The key of any pending push operation.
private static int POP
          When operationPending equals this, then a pop operation is pending.
private static int PUSH
          When operationPending equals this, then a push operation is pending.
private  Stack streamBufferStack
          Tracks the active input streams.
private  int unmatchedTypeLT
          Number of LT tokens that have not been matched by GT in Type Arguments or Type Parameters
private  Set usedKeys
          The set of keys corresponding to lexers whose output are being discarded or have been bound to output token streams.
 
Constructor Summary
ParsingController(Reader r, File file)
          Establish a parsing controller for the given reader.
 
Method Summary
 void addInputStream(antlr.TokenStream stream, String key)
          Adds the given input stream to the controller and binds it to the given key.
private  void bufferNextToken()
          Retrieves the next token from the currently active lexer and adds it to the appropriate buffer or discards it if necessary.
 int column()
          Returns the current column number in the input.
 void discardAllTokensFor(String key)
          Indicates that tokens generated by the lexer bound to the given key should be discarded instead of buffered.
 boolean discardTokensBefore(String key, antlr.Token index)
          Discards all tokens from the lexer bound to the given key that precede the given token of the surrounding language.
 File file()
          Returns the file name of the input.
 antlr.TokenStream get(String key)
          Returns the TokenStream corresponding to a given name.
 antlr.TokenStream getCurrentStream()
          Returns the currently active input stream.
 antlr.TokenStream getStream(String sname)
          Returns the named input stream (previously added with addInputStream).
 int getUnmatchedTypeLT()
          get the value of unmatchedTypeLT
 ParsingController.FilteredTokenStream initialOutputStream()
          Returns a token stream that provides tokens generated by the initial lexer.
 int line()
          Returns the current line number in the input.
private  ParsingController.IndexedTokenBuffer newBlockedEmptyBuffer()
          Factory method creates a new token buffer that is blocked and empty.
 void pop()
          Pops a stream from the stack of input streams.
private  void processPop()
          Pops a stream from the stack of input streams.
private  void processPush(String key)
          Switches to a new input stream, maintaining a stack of the old input streams.
 void push(String key)
          Switches to a new input stream for future token requests, maintaining a stack of the old input streams.
 void selectInitial(String key)
          Makes the lexer bound to the given key the initial lexer.
private  void setActive(String key, ParsingController.StreamBufferPair sbp)
          Sets the given key, input stream, and token buffer to be active.
 void setUnmatchedTypeLT(int count)
          set the value of unmatchedTypeLT
 antlr.LexerSharedInputState sharedInputState()
          Returns an input state that can be shared by the lexers attached to this controller.
 ParsingController.FilteredTokenStream streamForBefore(String key, antlr.Token index)
          Returns a token stream from the lexer bound to the given key and containing all the tokens preceding the given token of the surrounding language.
private  void useKey(String key)
          Registers that tokens generated by the lexer bound to the given key will be used or discarded.
private  void verifyHasKey(String key)
          Verifies that the given key is bound to a lexer.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

inputState

private ParsingController.InputState inputState
The input state that should be shared by the lexers bound to this parsing controller.


initialSet

private boolean initialSet
Indicates whether the initial input stream has been selected.


initialFilteredStream

private ParsingController.FilteredTokenStream initialFilteredStream
The output stream for the initially selected lexer.


keyStreamBufferMap

private Map keyStreamBufferMap
Maps keys to lexers and buffers


usedKeys

private Set usedKeys
The set of keys corresponding to lexers whose output are being discarded or have been bound to output token streams.


keysToDiscard

private Set keysToDiscard
The set of keys corresponding to lexers whose output are being discarded.


discarding

private boolean discarding
Indicates whether tokens for the currently active stream are to be discarded.


keyStack

private Stack keyStack
For efficiency, used to track the keys when switching between input streams. The top of the the stack is the previous active key, not the current active key.


streamBufferStack

private Stack streamBufferStack
Tracks the active input streams. The top of the the stack is the previous active input stream and token buffer, not the current.


activeKey

private String activeKey
The key of the currently active input stream.


activeStreamBuffer

private ParsingController.StreamBufferPair activeStreamBuffer
The currently active input stream and token buffer.


nestedLanguageBuffer

private ParsingController.IndexedTokenBuffer nestedLanguageBuffer
The active buffer before the last pop operation. Used to store index tokens into the buffer of a nested language.


endOfFileReached

private boolean endOfFileReached
Indicates whether an end-of-file token has been received on some input stream.


fetching

private boolean fetching
Indicates that a token fetch request is in progress for the active input stream.


operationPending

private int operationPending
Indicates whether any input stream switching operations (i.e., push, pop) are pending on completion of the current call to bufferNextToken.


ltCount

private int ltCount

pendingKey

private String pendingKey
The key of any pending push operation.

See Also:
operationPending

unmatchedTypeLT

private int unmatchedTypeLT
Number of LT tokens that have not been matched by GT in Type Arguments or Type Parameters


NONE

private static final int NONE
When operationPending equals this, then no operations are pending.

See Also:
operationPending

POP

private static final int POP
When operationPending equals this, then a pop operation is pending.

See Also:
operationPending

PUSH

private static final int PUSH
When operationPending equals this, then a push operation is pending.

See Also:
operationPending

IGNORE

public static final antlr.Token IGNORE
A token that may be returned by lexers upon detecting a lexical pattern that triggers a switch between lexers (e.g., /** for switching to javadoc). This token is not passed to any client parsers but is swallowed by the parsing controller. Using this token in a lexer allows the associated parser to ignore the lexical triggers. Note that using a "skip" token in ANTLR doesn't work for this purpose because the generated lexer code will not return from a nextToken call until a non-skip token is found. Thus using a skip token causes the lexer switch to be delayed.

Constructor Detail

ParsingController

public ParsingController(Reader r,
                         File file)
Establish a parsing controller for the given reader. The reader is used to generate shared lexer state for the various lexers tied to this. A lexer tied to this parser should be instantiated by passing it this. The lexer's constructor will call a super constructor passing it the result of calling sharedInputState() on this.

See Also:
sharedInputState()
Method Detail

sharedInputState

public antlr.LexerSharedInputState sharedInputState()
Returns an input state that can be shared by the lexers attached to this controller.


addInputStream

public void addInputStream(antlr.TokenStream stream,
                           String key)
                    throws ParsingController.KeyException
Adds the given input stream to the controller and binds it to the given key.

Throws:
ParsingController.KeyException - the given key is already bound or if either argument is null

getCurrentStream

public antlr.TokenStream getCurrentStream()
Returns the currently active input stream.


getStream

public antlr.TokenStream getStream(String sname)
Returns the named input stream (previously added with addInputStream).


selectInitial

public void selectInitial(String key)
                   throws ParsingController.KeyException,
                          ParsingController.ConfigurationException
Makes the lexer bound to the given key the initial lexer.

Throws:
ParsingController.KeyException - if the given key is not bound
ParsingController.ConfigurationException - if the initial lexer has already been selected

initialOutputStream

public ParsingController.FilteredTokenStream initialOutputStream()
                                                          throws ParsingController.ConfigurationException
Returns a token stream that provides tokens generated by the initial lexer.

Throws:
ParsingController.ConfigurationException - if the initial lexer has not been selected.
See Also:
selectInitial(String)

discardAllTokensFor

public void discardAllTokensFor(String key)
                         throws ParsingController.KeyException
Indicates that tokens generated by the lexer bound to the given key should be discarded instead of buffered. This allows the controller to make more efficient use of memory.

Throws:
ParsingController.KeyException - if the given key is not bound to a lexer or if a TokenStream for the key has already been exposed

streamForBefore

public ParsingController.FilteredTokenStream streamForBefore(String key,
                                                             antlr.Token index)
                                                      throws ParsingController.KeyException
Returns a token stream from the lexer bound to the given key and containing all the tokens preceding the given token of the surrounding language.

Throws:
ParsingController.KeyException - if the given key is not bound to a lexer

discardTokensBefore

public boolean discardTokensBefore(String key,
                                   antlr.Token index)
                            throws ParsingController.KeyException
Discards all tokens from the lexer bound to the given key that precede the given token of the surrounding language.

Returns:
true if any tokens are actually discarded
Throws:
ParsingController.KeyException - if the given key is not bound to a lexer

pop

public void pop()
         throws antlr.TokenStreamException
Pops a stream from the stack of input streams. Called when exiting a nested language block.

Throws:
antlr.TokenStreamException - if a pending operation prevents popping

get

public antlr.TokenStream get(String key)
                      throws ParsingController.KeyException,
                             antlr.TokenStreamException
Returns the TokenStream corresponding to a given name.

Parameters:
key - the name of the TokenStream to return
Returns:
the matching TokenStream
Throws:
ParsingController.KeyException
antlr.TokenStreamException

push

public void push(String key)
          throws ParsingController.KeyException,
                 antlr.TokenStreamException
Switches to a new input stream for future token requests, maintaining a stack of the old input streams. Called when entering a nested language block.

Parameters:
key - the unique identifier of the lexer to switch to
Throws:
antlr.TokenStreamException - if a pending operation prevents pushing
ParsingController.KeyException - if the given key is not bound to a lexer

file

public File file()
Returns the file name of the input.


line

public int line()
Returns the current line number in the input.


column

public int column()
Returns the current column number in the input.


bufferNextToken

private void bufferNextToken()
                      throws antlr.TokenStreamException
Retrieves the next token from the currently active lexer and adds it to the appropriate buffer or discards it if necessary.

Throws:
antlr.TokenStreamException - passed on from underlying lexer

processPush

private void processPush(String key)
Switches to a new input stream, maintaining a stack of the old input streams. Called when entering a nested language block.

Parameters:
key - the unique identifier of the lexer to switch to

processPop

private void processPop()
Pops a stream from the stack of input streams. Called when exiting a nested language block.


useKey

private void useKey(String key)
             throws ParsingController.KeyException
Registers that tokens generated by the lexer bound to the given key will be used or discarded. Handles checks to prevent assigning a lexer to multiple output streams or to both discard and output.

Parameters:
key - the unique identifier of the lexer whose tokens are to be discarded or passed to an output stream
Throws:
ParsingController.KeyException - if the given key is not bound to a lexer or if a TokenStream for the key has already been exposed

verifyHasKey

private void verifyHasKey(String key)
                   throws ParsingController.KeyException
Verifies that the given key is bound to a lexer.

Throws:
ParsingController.KeyException - if the given key is not bound

setActive

private void setActive(String key,
                       ParsingController.StreamBufferPair sbp)
Sets the given key, input stream, and token buffer to be active.


newBlockedEmptyBuffer

private ParsingController.IndexedTokenBuffer newBlockedEmptyBuffer()
Factory method creates a new token buffer that is blocked and empty.


getUnmatchedTypeLT

public int getUnmatchedTypeLT()
get the value of unmatchedTypeLT


setUnmatchedTypeLT

public void setUnmatchedTypeLT(int count)
set the value of unmatchedTypeLT


JML

JML is Copyright (C) 1998-2002 by Iowa State University and is distributed under the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This release depends on code from the MultiJava project and is based in part on the Kopi project Copyright (C) 1990-99 DMS Decision Management Systems Ges.m.b.H.