From: Petter Gustad
Subject: zebu: how to handle newline in long numbers?
Date: Tue, 16 Sep 2003 18:43:58 +0000
Message-ID: <871xugwp3l.fsf@zener.home.gustad.com>

I'm trying to write a zebu grammar to parse SVF files which can
contain newline and/or return characters in numbers (in order not to
exceed a line length of 256 characters per line).

I tried something like: 


(:name "svf"
 :grammar "null-grammar"
 :package "CL-USER"
 :white-space (#\Space #\Tab #\Newline #\Return)
 :lex-cats (
            (COMMENT "//.*$")
            (HEX "[0-9a-fA-F][0-9a-fA-F\\n\\r]*")
            )
 )

Here's a part of my grammar:

(defrule ARGS
    := ( ARGKEY "(" HEX ")" ARGS ) ;
    := () ;
    )

This results in the following error message:

Error in function ZEBU::NEXT-TOKEN:
   Unrecognized Token at: "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d53ff000000000000000120028000000000000000000000002000180006000180006000180006000180"
Expected: ")"

Is there a contradiction due to newline in whitespace as well as in my
hex number definition? 

Petter
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

• Re: zebu: how to handle newline in long numbers? Matthew Danish
  • Re: zebu: how to handle newline in long numbers? Petter Gustad

From: Matthew Danish
Subject: Re: zebu: how to handle newline in long numbers?
Date: Wed, 17 Sep 2003 08:22:40 +0000
Message-ID: <20030917082240.GY1454@mapcar.org>

On Tue, Sep 16, 2003 at 08:43:58PM +0200, Petter Gustad wrote:
> I'm trying to write a zebu grammar to parse SVF files which can
> contain newline and/or return characters in numbers (in order not to
> exceed a line length of 256 characters per line).
> I tried something like: 
> 
> (:name "svf"
>  :grammar "null-grammar"
>  :package "CL-USER"
>  :white-space (#\Space #\Tab #\Newline #\Return)
>  :lex-cats (
>             (COMMENT "//.*$")
>             (HEX "[0-9a-fA-F][0-9a-fA-F\\n\\r]*")
>             )
>  )

I am starting to suspect the lex-cats mechanism of having a few bugs.
Try these options instead:

:case-sensitive nil
:intern-identifier nil
:identifier-start-chars "0123456789abcdefABCDEF"
:identifier-continue-chars #.(format nil "0123456789abcdefABCDEF~C~C"
					 #\Newline #\Return)

Don't worry about :white-space for now, I think it is okay for your
purpose.  But you could try editting it, I suppose, if this doesn't
work.

Use the IDENTIFIER terminal category instead of HEX, now.

When calling zebu:read-parser, let the following special variables:

*preserve-case* => t
*disallow-packages* => t

See the doc/ folder for more info about these options.

-- 
; Matthew Danish <·······@andrew.cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian.org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."

From: Petter Gustad
Subject: Re: zebu: how to handle newline in long numbers?
Date: Wed, 17 Sep 2003 21:18:37 +0000
Message-ID: <87n0d315ci.fsf@zener.home.gustad.com>

Matthew Danish <·······@andrew.cmu.edu> writes:

> I am starting to suspect the lex-cats mechanism of having a few bugs.
> Try these options instead:
> 
> :case-sensitive nil

[snip]

I tried these, but I got the same result:

Error in function ZEBU::NEXT-TOKEN:
   Unrecognized Token at: "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d53ff000000000000000120028000000000000000000000002000180006000180006000180006000180"
Expected: ")"

I have a working parser based on Meta so I think I'll give up on zebu
for now...

Thanks
Petter

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?