(I tried to continue Aravinda's thread "empty token sequence error"
but that discussion is closed and no reply are there allowed).
Well, there is very few do***entation around (in this group, too)
about the "empty token sequence" issue.
Thus, let's take again the Aravinda's thread.
Weel, after the definition of the "canonical" ID lexical rule, I added
to Gpl.jj (the grammar has been previously mentioned, see
Avoid SKIPping while producing another terminal / Unit tests for
JavaCC tokens and productions
http://groups.google.com/group/comp.compilers.tools.javacc/browse_thread/thread/e33401b7c7c81b66
)
the lexical rule for the group identifiers. A group identifier is an
identifier optionally followed by zero or more identifiers, being
separated by a given string (currently a string composed by a single
dot character)
[I] <GROUP_ID : <ID> ( <GROUP_ID_SEPARATOR> <ID> )* >
[II] <GROUP_ID : <ID> <GROUP_ID_SEPARATOR> <ID>
( <GROUP_ID_SEPARATOR> <ID> )* >
The second version [II] is an attempted workaround to clean the
warning described below.
Here is the Gpl.jj
------------------------
options
{
STATIC = true;
UNICODE_INPUT = true;
}
PARSER_BEGIN(Gpl)
package it.finmatica.gpj.ec.istruzioni.gpl;
public class Gpl
{
public static
void
main( String args[] ) throws ParseException
{
Gpl lParser = new Gpl( System.in );
lParser.ExpressionList();
}
} // end class Gpl
PARSER_END(Gpl)
SKIP :
{ " "
| "\t"
| "\n"
| "\r"
}
TOKEN : // Identifiers
{ <ID : ["a"-"z", "A"-"Z","_"] ( ["a"-"z", "A"-"Z", "_", "0"-"9"] )* >
}
TOKEN : // Group identifiers
{ <GROUP_ID : <ID> ( <GROUP_ID_SEPARATOR> <ID> )* >
| < #GROUP_ID_SEPARATOR: "." >
}
/***
TOKEN : // Group identifiers
{ <GROUP_ID : <ID> <GROUP_ID_SEPARATOR> <ID> ( <GROUP_ID_SEPARATOR>
<ID> )* >
| < #GROUP_ID_SEPARATOR: "." >
}
***/
void
ExpressionList() :
{
String lString;
}
{
{
System.out.println( "Type in an expression followed by a \";\" or ^D
to quit:" );
System.out.println( "" );
}
( lString = Expression() ";"
{
System.out.println( lString );
System.out.println("");
System.out.println("Type in another expression followed by a \";\"
or ^D to quit:" );
System.out.println("");
}
)*
<EOF>
} // end Gpl.ExpressionList()
String
Expression() :
{
java.util.Vector<String> lTermImage = new java.util.Vector<String>();
String lResult;
}
{
lResult = Term()
{
lTermImage.addElement( lResult );
}
( "+" lResult = Term()
{
lTermImage.addElement( lResult );
}
)*
{
if ( lTermImage.size() == 1 )
{
lResult = lTermImage.elementAt(0);
return lResult;
}
else
{
lResult = "the sum of " + lTermImage.elementAt(0);
for ( int i = 1; i < lTermImage.size() - 1; i++ )
{
lResult += ", " + lTermImage.elementAt(i);
}
if ( lTermImage.size() > 2 )
lResult += ",";
lResult += " and " + lTermImage.elementAt( lTermImage.size() -
1 );
return lResult;
}
}
} // end Gpl.Expression()
String
Term() :
{
java.util.Vector<String> lFactorImage = new
java.util.Vector<String>();
String lResult;
}
{
lResult = Factor()
{
lFactorImage.addElement( lResult );
}
(
"*" lResult = Factor()
{
lFactorImage.addElement( lResult );
}
)*
{
if ( lFactorImage.size() == 1 )
{
lResult = lFactorImage.elementAt(0);
return lResult;
}
else
{
lResult = "the product of " + lFactorImage.elementAt(0);
for ( int i = 1; i < lFactorImage.size() - 1; i++ )
{
lResult += ", " + lFactorImage.elementAt(i);
}
if ( lFactorImage.size() > 2 )
lResult += ",";
lResult += " and " + lFactorImage.elementAt( lFactorImage.size() -
1 );
return lResult;
}
}
} // end Gpl.Term()
String
Factor() :
{
Token lToken;
String lResult;
}
{
{
lResult = Id();
return lResult;
}
| {
lResult = GroupId();
return lResult;
}
} // end Gpl.Factor()
String
Id() :
{
Token lToken;
String lResult;
}
{
lToken = <ID>
{
lResult = lToken.image;
return lResult;
}
} // end Gpl.Id()
String
GroupId() :
{
Token lToken;
String lResult;
}
{
lToken = <GROUP_ID>
{
lResult = lToken.image;
return lResult;
}
|
lToken = <ID>
{
lResult = lToken.image;
return lResult;
}
} // end Gpl.GroupId()
// EOF, in unit testing allows to test a given input and not a prefix
// (see also:
http://groups.google.com/group/comp.compilers.tools.javacc/browse_thread/thread/e33401b7c7c81b66/cede39f37111bc73#cede39f37111bc73)
String
Eof() :
{
Token lToken;
String lResult;
}
{
lToken = <EOF>
{
lResult = lToken.image;
return lResult;
}
} // end Gpl.Eof()
------------------------
Both <ID> and <GROUP_ID> share a prefix for the input.
Following the suggestion
3.3. What if more than one regular expression matches a prefix of the
remaining input?
http://www.engr.mun.ca/~theo/JavaCC-FAQ/
I wrote the non terminal GroupId() (see above).
Notwithstanding this I got the following warning (switching between
the [I] and [II] definition does not "clean" the warning)
Warning: Line 153, Column 9: This choice can expand to the empty
token sequence and will therefore always be taken in favor of the
choices appearing later.
javacc complains that within Factro() non terminal the Id() might be
choosen to the detriment of the following choice (GroupId())
What does it mean? what's wrong, if anything is wrong?
Excuse me if the questions might be naive, I've found only a brief
mention about the empty token sequence issue in the
http://www.engr.mun.ca/~theo/JavaCC-Tutorial/.
p. 25, "Maximal munch")
Is there any do***entation about the empty token sequence issue?
Thanks in advance for any clarification.
ciao
Cesare


|