From: Vassili Bykov
Subject: Re: Parenthesized syntax challenge
Date: 
Message-ID: <45bijr$9u@tandem.CAM.ORG>
········@expernet26.expernet.com (Peter Ludemann) wrote:
>[...]
>Because Smalltalk, C, Java, etc. programmers don't have access to the
>entire source in a convenient form, it's much harder for them to build
>sophisticated preprocessors, without cranking up something heavy,
>using yacc/lex.  (Lisp programmers do get a reasonable macro expansion
>facility.)

Excuse me, but you definitely don't know enough about Smalltalk to include
it in this statement.  I am a LISP adept, but I do Smalltalk for living, 
and here Smalltalk does not go in one row with C and Java.

Smalltalk does provide access to the entire source text.  (I do mean 
programmatical access).  Smalltalk compiler is also written in Smalltalk
and is accessible from a program.  It means in your code you can retrieve 
a compiled method of a class.  Then you can send a message to the method to 
get its source text.  Then you can feed the text into the scanner and get the 
stream of tokens for this method, or into the parser and get a parse tree.  
All nouns mentioned above are first-class Smalltalk object, which means you 
can do whatever you want with them.  Like manipulate the parse tree and then 
send it a message to generate compiled code.  Or preprocess the stream of 
tokens and *then* feed it into the parser.

Also, since scanner, parser and parse tree nodes are all first-class objects,
you can easily build your own customized versions of them and thus either
extend the language with new constructs or build you own.  Adding namespaces
to Smalltalk took me about 5 minutes.

You are right, Smalltalk syntax does not look like syntax for any Smalltalk
data structures.  But the source *is* convenient to access, and easy to
manipulate at any compilation stage.  Also, you have an analog of lambda
in Smalltalk.  It is called BlockClosure, it retains its creation context
(environment, that is), and is a first-class object you can pass around and
evaluate as many times as you want.

--Vassili