Talk About Network

Google


Register and Login
Nick
Password
Register create new account Sign up is FREE and you can post replies, new topics, bookmark posts and more!
Recover lost password


Programming > Compiler Tools JavaCC > Actual vs symbo...
Latest [ Topics | Posts ] Archive Post A New Topic Post a Reply
<< Topic < Post Post 1 of 3 Topic 472 of 548
Post > Topic >>

Actual vs symbolic token image

by Cesare Zecca <Cesare.Zecca.prof@[EMAIL PROTECTED] > Sep 11, 2007 at 03:43 PM

Hi all

Theodore Norvell recommends token reuse when a given regular
expression is repeatedly used in other expressions (cf. A draft
tutorial by Theodore Norvell,  
http://www.engr.mun.ca/~theo/JavaCC-Tutorial/,
p 15).

I defined the following basic tokens

TOKEN : // Basics
{	<#LETTER : ["a"-"z", "A"-"Z"] >
|	<#DIGIT : ["0"-"9"] >
|	<PAR_OPEN : ["("] >
|	<PAR_CLOSED : [")"] >
}

in the Gpl.jj grammar file. Some of them have been made private to the
token manager to avoid that they are passed to the parser and
nonterminals.
PAR_OPEN and PAR_CLOSED have been made tokens too: currently the
language is still green and it is still possible the specific kind of
parentheses might change in the future ( e.g. change from  round  to
angular "<" / ">" or square "[" / "]" brackets). Thus I decided to
factor out them in a unique, changeable definition.

PAR_OPEN and PAR_CLOSED are public because they are used also in
nonterminal rules, such as the Factor() one

String
Factor() :
{
	Token lToken;
	String lResult;
}
{
	lResult = GroupId()
	{
		return lResult;
	}

|	<PAR_OPEN> lResult = Expression() <PAR_CLOSED>
	{
		String lParOpen = GplConstants.tokenImage[PAR_OPEN];
		String lParClosed = GplConstants.tokenImage[PAR_CLOSED];
		lResult = lParOpen + " " + lResult + " " + lParClosed;
		return lResult;
	}
} // end Gpl.Factor()

As you can see I tried to avoid to scatter throughout the grammar file
string literals such as "(" and ")" using their GplConstants' symbolic
definitions.

The String result printed out for explanation purposes for the "( (a +
b) + c)" input is some of this kind

	<PAR_OPEN> the sum of <PAR_OPEN>the sum of a and b<PAR_CLOSED> and c
<PAR_CLOSED>

The same holds when I tried to use them in the exceptions to make the
diagnostic message, that contains <PAR_OPEN> and <PAR_CLOSED> (instead
of their actual images, not so well readable by humans)

I would like to get something similar to

	( the sum of ( the sum of a and b ) and c )

using the actual token terminal (again, without being forced to
directly use any string literal such as "(" and ")" in the code)
In other words I would like to access the ACTUAL image of the terminal
instead of its simbolic name defined by GplConstants.tokenImage

Is there anyway to get the actual image of a (token) terminal?

(initially I used the term "token category" instead of (token)
terminal that, perhaps, in some way helps to distinguish between the
actual token "(" and its symbolic name <PAR_OPEN>)

Thanks in advance for any reply.
ciao
Cesare


P.S.

I thougth also to to factor out in a Java String constant the string
literal and to reuse it in the token definition

PARSER_BEGIN(Gpl)

package it.finmatica.gpj.ec.istruzioni.gpl;

public class Gpl
{
	public static
	void
	main( String args[] ) throws ParseException
	{
		Gpl lParser = new Gpl( System.in );
		lParser.ExpressionList();
	}

	public static final String sParOpen = new String( "(" );	// <-- here!
} // end class Gpl

PARSER_END(Gpl)


....


TOKEN :	// Basics
{	<#LETTER : ["a"-"z", "A"-"Z"] >
|	<#DIGIT : ["0"-"9"] >
|	<PAR_OPEN : [ sParOpen ] >
|	<PAR_CLOSED : [")"] >
}

But the attempt was naive and led to the following error issued by
JavaCC

org.javacc.parser.ParseException: Encountered "sParOpen" at line 36,
column 23.
Was expecting one of:
    <STRING_LITERAL> ...
    "]" ...
 




 3 Posts in Topic:
Actual vs symbolic token image
Cesare Zecca <Cesare.Z  2007-09-11 15:43:51 
Re: Actual vs symbolic token image
AC <user@[EMAIL PROTEC  2007-09-12 07:57:33 
Re: Actual vs symbolic token image
Cesare Zecca <Cesare.Z  2007-09-12 17:05:49 

Post A Reply:
  Go here to Signup

AddThis Feed Button


About - Advertising - Contact - Frequently Asked Questions - Privacy Policy - Terms of Use - Signup

Contact
tan12V112 Wed Jul 23 23:49:27 CDT 2008.