From: Bob Felts
Subject: A Better Lisp?
Date: 
Message-ID: <1hfwbps.19x3acl9qcg8yN%wrf3@stablecross.com>
In another thread, Pascal Bourguignon made the statement, "But
researchers who are doing this kind of productivity studies are
not _able_ to make a better lisp. "

One of the things that I've observed is that Lisp isn't "very good" at
reading data from a text file.  Perhaps this is because of my newness to
Lisp (corrections gladly welcome), but:

1) Lisp isn't orthogonal.  It has (parse-integer ...) but it doesn't
have (parse-number ...).
2) (read ...) might suffice, but it reads things I don't want read (in
particular, what if I want "; ..." to not be ignored?
      (read-from-string ";;; this is a test" nil 'EOF)
   returns EOF.
3)  (read...) is dangerous since it, by default, allows for execution of
user input.
4) There is no equivalent of (format ...) for input conversion (like C's
scanf).

Yes, I can write all of this or I can google it -- but this is basic
stuff.

I have some binary files that I want to parse so I'll be giving Peter
S.'s chapter 25 a workout in the next few days as time permits.

From: Ron Garret
Subject: Re: A Better Lisp?
Date: 
Message-ID: <rNOSPAMon-B5A48D.13225425052006@news.gha.chartermi.net>
In article <···························@stablecross.com>,
 ····@stablecross.com (Bob Felts) wrote:

> In another thread, Pascal Bourguignon made the statement, "But
> researchers who are doing this kind of productivity studies are
> not _able_ to make a better lisp. "

It's not true, but it is hard to significantly improve on what currently 
exists.

> One of the things that I've observed is that Lisp isn't "very good" at
> reading data from a text file.  Perhaps this is because of my newness to
> Lisp (corrections gladly welcome), but:
> 
> 1) Lisp isn't orthogonal.  It has (parse-integer ...) but it doesn't
> have (parse-number ...).

(defun parse-number (s)
  (let ( (n (read-from-string s)) )
    (if (not (numberp n))
      (error "~A is not a number" s)
      n)))


> 2) (read ...) might suffice, but it reads things I don't want read (in
> particular, what if I want "; ..." to not be ignored?
>       (read-from-string ";;; this is a test" nil 'EOF)
>    returns EOF.

(set-syntax-from-char #\; #\a)


> 3)  (read...) is dangerous since it, by default, allows for execution of
> user input.

(defvar *safe-readtable* (copy-readtable))
(dolist (c (list #\# #\, #\`))
  (set-syntax-from-char c #\a *safe-readtable*))
(set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
                     nil *safe-readtable*)
(defun safe-read (&rest args)
  (let ( (*readtable* *safe-readtable*) )
    (ignore-errors (apply 'read args))))


> 4) There is no equivalent of (format ...) for input conversion (like C's
> scanf).

That's right.  There's a good reason for this.  Figuring out what it is 
is left as an exercise.

rg
From: Bob Felts
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1hfwwnd.1tzsqntp9uzb4N%wrf3@stablecross.com>
Ron Garret <·········@flownet.com> wrote:

> In article <···························@stablecross.com>,
>  ····@stablecross.com (Bob Felts) wrote:
> 
> > In another thread, Pascal Bourguignon made the statement, "But
> > researchers who are doing this kind of productivity studies are
> > not _able_ to make a better lisp. "
> 
> It's not true, but it is hard to significantly improve on what currently
> exists.
> 

In the all to short time I've been playing with Lisp, it's far and above
anything else; nevertheless, I'm not (yet) convinced it can't be
significantly improved.  Why, someone of some repute was recently
lobbying for a certain syntatic change... ;-)

> > One of the things that I've observed is that Lisp isn't "very good" at
> > reading data from a text file.  Perhaps this is because of my newness to
> > Lisp (corrections gladly welcome), but:
> > 
> > 1) Lisp isn't orthogonal.  It has (parse-integer ...) but it doesn't
> > have (parse-number ...).
> 
> (defun parse-number (s)
>   (let ( (n (read-from-string s)) )
>     (if (not (numberp n))
>       (error "~A is not a number" s)
>       n)))
> 

Well, yes,  But suppose the error should say:
   (error "~S is not a number at line ~A" s line-no)

Making this work well is harder for me in Lisp than it is in, say, C.
And I'm still not satisfied with my solution...

> > 2) (read ...) might suffice, but it reads things I don't want read (in
> > particular, what if I want "; ..." to not be ignored?
> >       (read-from-string ";;; this is a test" nil 'EOF)
> >    returns EOF.
> 
> (set-syntax-from-char #\; #\a)
> 

But now I have to bring in the overhead of safely modifying a readtable.
Sure, it works, but I'm not convinced it's elegant.

> > 3)  (read...) is dangerous since it, by default, allows for execution of
> > user input.
> 
> (defvar *safe-readtable* (copy-readtable))
> (dolist (c (list #\# #\, #\`))
>   (set-syntax-from-char c #\a *safe-readtable*))
> (set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
>                      nil *safe-readtable*)
> (defun safe-read (&rest args)
>   (let ( (*readtable* *safe-readtable*) )
>     (ignore-errors (apply 'read args))))
> 

All I want to do is to safely parse a text file and output user-friendly
error messages (e.g. "At <line #> <character #> expected <this> found
<that>")

> > 4) There is no equivalent of (format ...) for input conversion (like C's
> > scanf).
> 
> That's right.  There's a good reason for this.  Figuring out what it is
> is left as an exercise.
> 

Ok, I'll bite.  One reason is because Lisp uses call-by-value and not
call-by-reference.  Currently, I need to use (multiple-value-list
(read-from-string ...)), why couldn't this be extended to a scanf-type
function?
From: Novus
Subject: Re: A Better Lisp?
Date: 
Message-ID: <2006052523053716807-novus@ngoqdeorg>
On 2006-05-25 21:46:43 -0400, ····@stablecross.com (Bob Felts) said:

> All I want to do is to safely parse a text file and output user-friendly
> error messages (e.g. "At <line #> <character #> expected <this> found
> <that>")

I'm confused. Why not just use cl-yacc? Have your lexer save the line
and character number.

Novus
From: Ron Garret
Subject: Re: A Better Lisp?
Date: 
Message-ID: <rNOSPAMon-9805C7.00162126052006@news.gha.chartermi.net>
In article <···························@stablecross.com>,
 ····@stablecross.com (Bob Felts) wrote:

> > > One of the things that I've observed is that Lisp isn't "very good" at
> > > reading data from a text file.  Perhaps this is because of my newness to
> > > Lisp (corrections gladly welcome), but:
> > > 
> > > 1) Lisp isn't orthogonal.  It has (parse-integer ...) but it doesn't
> > > have (parse-number ...).
> > 
> > (defun parse-number (s)
> >   (let ( (n (read-from-string s)) )
> >     (if (not (numberp n))
> >       (error "~A is not a number" s)
> >       n)))
> > 
> 
> Well, yes,  But suppose the error should say:
>    (error "~S is not a number at line ~A" s line-no)
> 
> Making this work well is harder for me in Lisp than it is in, say, C.
> And I'm still not satisfied with my solution...

What does that have to do with Lisp's non-orthogonality and lack of a 
built-in read-number?

If you want errors with line numbers then just use read-line and 
increment a counter every time you call it.

> > > 2) (read ...) might suffice, but it reads things I don't want read (in
> > > particular, what if I want "; ..." to not be ignored?
> > >       (read-from-string ";;; this is a test" nil 'EOF)
> > >    returns EOF.
> > 
> > (set-syntax-from-char #\; #\a)
> > 
> 
> But now I have to bring in the overhead of safely modifying a readtable.

Overhead?  What overhead?  And what do you mean by "safely modifying"?

> Sure, it works, but I'm not convinced it's elegant.

<shrug> Elegance is in the eye of the beholder.

As Novus pointed out you can always use CL-YACC.

> > > 3)  (read...) is dangerous since it, by default, allows for execution of
> > > user input.
> > 
> > (defvar *safe-readtable* (copy-readtable))
> > (dolist (c (list #\# #\, #\`))
> >   (set-syntax-from-char c #\a *safe-readtable*))
> > (set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
> >                      nil *safe-readtable*)
> > (defun safe-read (&rest args)
> >   (let ( (*readtable* *safe-readtable*) )
> >     (ignore-errors (apply 'read args))))
> > 
> 
> All I want to do is to safely parse a text file and output user-friendly
> error messages (e.g. "At <line #> <character #> expected <this> found
> <that>")

OK.  So what's the problem?

> > > 4) There is no equivalent of (format ...) for input conversion (like C's
> > > scanf).
> > 
> > That's right.  There's a good reason for this.  Figuring out what it is
> > is left as an exercise.
> > 
> 
> Ok, I'll bite.  One reason is because Lisp uses call-by-value and not
> call-by-reference.  Currently, I need to use (multiple-value-list
> (read-from-string ...)), why couldn't this be extended to a scanf-type
> function?

That's one reason, but it's not the main reason.  C is call-by-value 
too.  That's why you have to pass pointers to scanf.  You could do the 
same thing in Lisp.  That's not the reason it isn't done.

Here's a hint: why doesn't C have a library function equivalent to 
Lisp's READ function?

rg
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: A Better Lisp?
Date: 
Message-ID: <8764jt89tj.fsf@qrnik.zagroda>
Ron Garret <·········@flownet.com> writes:

> That's one reason, but it's not the main reason.  C is call-by-value 
> too.  That's why you have to pass pointers to scanf.  You could do the 
> same thing in Lisp.

No. Lisp doesn't support first-class places. They can be emulated by
   (cons #'(lambda () place) #'(lambda (value) (setf place value)))
but then the place is evaluated on each access.

Only the setter is needed by scanf though, and plain variables rather
than arbitrary places might suffice.

Not that it's best suitable for scanf; perhaps it should bind new
variables rather than setting them. And the scanf interface is poor,
it should better support some regexps.

-- 
   __("<         Marcin Kowalczyk
   \__/       ······@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/
From: sross
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1148642573.204341.184820@j33g2000cwa.googlegroups.com>
> No. Lisp doesn't support first-class places. They can be emulated by
>    (cons #'(lambda () place) #'(lambda (value) (setf place value)))
> but then the place is evaluated on each access.

Or take a look at locatives which uses the setf machinary to emulate
first class places.

http://article.gmane.org/gmane.lisp.sources.code/17

Sean.
From: Ron Garret
Subject: Re: A Better Lisp?
Date: 
Message-ID: <rNOSPAMon-4B378C.09394226052006@news.gha.chartermi.net>
In article <··············@qrnik.zagroda>,
 Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote:

> Ron Garret <·········@flownet.com> writes:
> 
> > That's one reason, but it's not the main reason.  C is call-by-value 
> > too.  That's why you have to pass pointers to scanf.  You could do the 
> > same thing in Lisp.
> 
> No.

Yes.

> Lisp doesn't support first-class places.

Not natively, but it's easy to roll your own, you yourself point out in 
the very next sentence:

> They can be emulated by
>    (cons #'(lambda () place) #'(lambda (value) (setf place value)))
> but then the place is evaluated on each access.

So what?

> Only the setter is needed by scanf though, and plain variables rather
> than arbitrary places might suffice.
>
> Not that it's best suitable for scanf; perhaps it should bind new
> variables rather than setting them. And the scanf interface is poor,
> it should better support some regexps.

Now you're thinking.

rg
From: Rob Thorpe
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1148637898.063992.142900@i39g2000cwa.googlegroups.com>
Bob Felts wrote:
> Ron Garret <·········@flownet.com> wrote:
>
> > In article <···························@stablecross.com>,
> >  ····@stablecross.com (Bob Felts) wrote:
> >
> > > In another thread, Pascal Bourguignon made the statement, "But
> > > researchers who are doing this kind of productivity studies are
> > > not _able_ to make a better lisp. "
> >
> > It's not true, but it is hard to significantly improve on what currently
> > exists.
> >
>
> In the all to short time I've been playing with Lisp, it's far and above
> anything else; nevertheless, I'm not (yet) convinced it can't be
> significantly improved.  Why, someone of some repute was recently
> lobbying for a certain syntatic change... ;-)
>
> > > One of the things that I've observed is that Lisp isn't "very good" at
> > > reading data from a text file.  Perhaps this is because of my newness to
> > > Lisp (corrections gladly welcome), but:
> > >
> > > 1) Lisp isn't orthogonal.  It has (parse-integer ...) but it doesn't
> > > have (parse-number ...).
> >
> > (defun parse-number (s)
> >   (let ( (n (read-from-string s)) )
> >     (if (not (numberp n))
> >       (error "~A is not a number" s)
> >       n)))
> >
>
> Well, yes,  But suppose the error should say:
>    (error "~S is not a number at line ~A" s line-no)
>
> Making this work well is harder for me in Lisp than it is in, say, C.
> And I'm still not satisfied with my solution...
>
> > > 2) (read ...) might suffice, but it reads things I don't want read (in
> > > particular, what if I want "; ..." to not be ignored?
> > >       (read-from-string ";;; this is a test" nil 'EOF)
> > >    returns EOF.
> >
> > (set-syntax-from-char #\; #\a)
> >
>
> But now I have to bring in the overhead of safely modifying a readtable.

It's only really a little mental overhead.  The reader does not become
slower.

> Sure, it works, but I'm not convinced it's elegant.

If I were re-writing common lisp today I would have a separate
pre-knobbled readtable in it and a function like (read-untrusted ...).
But theres really no problem with doing this.

> > > 3)  (read...) is dangerous since it, by default, allows for execution of
> > > user input.
> >
> > (defvar *safe-readtable* (copy-readtable))
> > (dolist (c (list #\# #\, #\`))
> >   (set-syntax-from-char c #\a *safe-readtable*))
> > (set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
> >                      nil *safe-readtable*)
> > (defun safe-read (&rest args)
> >   (let ( (*readtable* *safe-readtable*) )
> >     (ignore-errors (apply 'read args))))
> >
>
> All I want to do is to safely parse a text file and output user-friendly
> error messages (e.g. "At <line #> <character #> expected <this> found
> <that>")

In most languages there is one type of file reading problem.  Lets call
that problem A.

Problem A is ...
Read a certain file format from a file, extract data from that format,
perhaps are a tree.
The solution to this problem is almost always the same, write some form
of lexer which passes data to some form of parser.  Actions within the
parser put the data into program data structures.

In Lisp there are really 2 problems, problem A and problem B.  Problem
B is "read something that is lisp".  In Lisp problem B is easy to
solve, to solve it you use (read ...), and if you need read to be
secure you knobble it as others have shown.  Problem A though remains
the same as it does in other languages, read does not help very much.
The approach should be roughly the same as in other languages.

Your problem falls squarely into class A.
Where you are using (parse-integer) above you should probably be
treating the problem as lexing the tokens of the file format in
question.  This problem is best treated in two parts, first find the
boundaries of each token, then extract the contents.

Finding the boundaries is the job of a lexer, which may be as simple as
a couple of regular expressions.  Once the boundaries have been found
you may be able to use the reader to read the token.

;; This function occurs after boundaries have been found
;; token is the token contents as a string, token-type it's type
(defun extract-object (token token-type)
  (if (equal token-type 'string)
      (let (n (read-from-string token))
        ...)
  (if (equal token-type 'symbol)
      (let (n (read-from-string token)) ;; Tokens not suitable use a
custom fn here
        ...))
;; Above should use cond

So, in a small way the reader can help with problem A.  But only
because the form of the tokens in the file format.

There are many gotchas in this subject.  It is quite often not possible
to read some tokens in a file with C scanf.  If an integer could be
larger than the size of long then using scanf("%d", x); is troublesome
because it won't work, one must check it's return value and attempt
recovery.  This may be needed even if values >sizeof(long) are not
allowed in the file format, but where there is a possiblity that in
error a space could be deleted between two number, e.g. "45678 9987653"
-> "456789987653".  This error is not caught unless the return value of
scanf is used.  Using %lf is even worse, since there are many subtly
different formats for floating point numbers. Lisp read is similar, it
is not appropriate for every job.  Each file format must be examined
individually.


I agree with you that Common Lisp is not perfect.  There are many parts
that are not orthogonal.  Many parts of the language that could be much
improved upon.  I don't think its worthwhile though to write a whole
new language to solve the problem though.  It would be nice to see some
additions though.  I'd really welcome a kind of CL + "stuff everyone
uses/does" standard.
From: Frank Buss
Subject: Re: A Better Lisp?
Date: 
Message-ID: <uncjswqqj2m9$.1pz75n3jk8bsr.dlg@40tude.net>
Ron Garret wrote:

> (defvar *safe-readtable* (copy-readtable))
> (dolist (c (list #\# #\, #\`))
>   (set-syntax-from-char c #\a *safe-readtable*))
> (set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
>                      nil *safe-readtable*)
> (defun safe-read (&rest args)
>   (let ( (*readtable* *safe-readtable*) )
>     (ignore-errors (apply 'read args))))

Why not just (setf *READ-EVAL* nil) ?

-- 
Frank Buss, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
From: Ron Garret
Subject: Re: A Better Lisp?
Date: 
Message-ID: <rNOSPAMon-EABB94.14115425052006@news.gha.chartermi.net>
In article <·······························@40tude.net>,
 Frank Buss <··@frank-buss.de> wrote:

> Ron Garret wrote:
> 
> > (defvar *safe-readtable* (copy-readtable))
> > (dolist (c (list #\# #\, #\`))
> >   (set-syntax-from-char c #\a *safe-readtable*))
> > (set-macro-character #\: (lambda (stream c) (declare (ignore stream)) c)
> >                      nil *safe-readtable*)
> > (defun safe-read (&rest args)
> >   (let ( (*readtable* *safe-readtable*) )
> >     (ignore-errors (apply 'read args))))
> 
> Why not just (setf *READ-EVAL* nil) ?

Oh, that would be too easy.

Seriously, if you just set *READ-EVAL* to nil the reader would throw 
errors for certain kinds of input that you may or may not want.  Rolling 
your own readtable gives you more flexibility handling errors in the 
input.

rg
From: Peter Lewerin
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1148596732.851284.84300@i40g2000cwc.googlegroups.com>
> One of the things that I've observed is that Lisp isn't "very good" at
> reading data from a text file.

Reading data from any old text file is often hard: Lisp is probably no
better or worse than most other programming languages in that regard.

However, Lisp has an advantage that makes it *excel* at reading (or
writing) data from text files, compared to other languages: built-in
and ubiquitous serialization of data.  Values and structures can be
written as plain text and read back seamlessly.  This even extends to
user-defined types.  All in all, I think it's fair to say that Lisp is
very good indeed at handling data as text.
From: ··············@gmail.com
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1148734401.072512.123960@j55g2000cwa.googlegroups.com>
Has anyone heard of JMC's talk about beyond lisp? If anyone actually
attended and listened I was wondering what the content of the stuff
was.

<a
href="http://www.international-lisp-conference.org/2005/speakers.html#john_mccarthy">
John McCarthy's talk</a>

I have brainstormed some potential modifications to lisp but they were
written when I was a little bit naive.
http://community.schemewiki.org/?Beyond-lisp
From: Thomas Schilling
Subject: Re: A Better Lisp?
Date: 
Message-ID: <4dteasF1c6s8cU1@news.dfncis.de>
··············@gmail.com wrote:
> I have brainstormed some potential modifications to lisp but they were
> written when I was a little bit naive.
> http://community.schemewiki.org/?Beyond-lisp

I don't understand what problem you are trying to solve?  I assume you
don't want to embed Prolog into Lisp, since this has already been done.
 So are you trying to prove properties of Lisp programs?  I don't get
this PLs-as-groups-idea.  Don't you usually take a semantics of your
language and a program and prove properties of you program?  Sure, the
default semantics of Scheme/CL is probably very hard to formalize.  So
you limit the language to make reasoning about it easier.  Then,
however, you don't go "beyond" lisp, but create "a verifiable subset of
Lisp" which should be simply embeddable.  Do you know ACL2? (Not
AllegroCL 2.0).

I'm not saying that what you're doing is bad--just that you have to be a
bit more specific about what your actual goals are.

-ts
From: ··············@gmail.com
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1149197775.795970.20520@i39g2000cwa.googlegroups.com>
Yes, I have been busy with that to not improve the description. I am
going to grok the reasoned schemer and prolog style programming before
I tackle that problem.
Thomas Schilling wrote:
> ··············@gmail.com wrote:
> > I have brainstormed some potential modifications to lisp but they were
> > written when I was a little bit naive.
> > http://community.schemewiki.org/?Beyond-lisp
>
> I don't understand what problem you are trying to solve?  I assume you
> don't want to embed Prolog into Lisp, since this has already been done.
>  So are you trying to prove properties of Lisp programs?  I don't get
> this PLs-as-groups-idea.  Don't you usually take a semantics of your
> language and a program and prove properties of you program?  Sure, the
> default semantics of Scheme/CL is probably very hard to formalize.  So
> you limit the language to make reasoning about it easier.  Then,
> however, you don't go "beyond" lisp, but create "a verifiable subset of
> Lisp" which should be simply embeddable.  Do you know ACL2? (Not
> AllegroCL 2.0).
>
> I'm not saying that what you're doing is bad--just that you have to be a
> bit more specific about what your actual goals are.
> 
> -ts
From: Kaz Kylheku
Subject: Re: A Better Lisp?
Date: 
Message-ID: <1149377666.304421.134540@u72g2000cwu.googlegroups.com>
Bob Felts wrote:
> 4) There is no equivalent of (format ...) for input conversion (like C's
> scanf).

CL-PPCRE