I'm really enjoying Lisp. Coming from perl and Python, it is certainly
liberating.
However, a couple of questions. I've heard that it is possible to
write your own language in Lisp. I want to see how far that can go.
(1) Is it possible to make whitespace significant? In other words, is
it possible to have the reader put in some kind of token when it
detects an indent or dedent like Python does?
(2) Is it possible to give special meaning for certain characters,
such as ':', ';', or even ','? A case would be to write a hash-table
creator macro that would parse like Python:
{key-expr: value-expr, key-expr: value-expr, etc...}
In article
<····································@u12g2000prd.googlegroups.com>,
Jonathan Gardner <········@jonathangardner.net> wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
Yes. But that would be widely considered to be a Bad Idea (tm).
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
>
> {key-expr: value-expr, key-expr: value-expr, etc...}
That's a little harder. The problem is that the meaning of the colon
character is hard-coded deep within the reader and is impossible to
override without re-writing the reader. You can do that (there are
open-source versions available) but IMHO it would be more trouble than
it's worth.
You can do a quick-and-dirty hack version if you don't need to support
package syntax: just read the string to the next close curly-brace (you
need to be a little more clever than that if you want to support nested
dictionaries), delete all the colon and commas, and then just call
read-from-string to read out the tokens.
Note that all those colons and commas don't really add any information,
they're just eye-candy (and carpal-tunnel inducers). The LISPy way
would be to just do:
{ key value key value ... }
or
#D(key value key value ...)
rg
Jonathan Gardner <········@jonathangardner.net> writes:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
Well, the general idea behind a so-called domain-specific language (DSL)
in lisp is that they tend to use lisp syntax. That plus macros lets you
very quickly come up with your own DSL and then use it to solve a
problem. Deviating a lot from lisp syntax is not well-supported by the
tools.
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
I suppose it might be, although I would expect it to be tricky to
arrange just by using the reader. You would have to keep track of state
globally, and use macros on the whitespace to manage it.
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
>
> {key-expr: value-expr, key-expr: value-expr, etc...}
Well, it is usually hard to impossible to change the behavior of the
package marker ":" character. That is the one character that is often
hard-wired into the reader. Also, the reader is forward looking, so
that it would also be pretty much impossible to have the last character
of a symbol affect the interpretation and reading of that symbol,
because by the time the character is detected, the symbol has already
been read.
--
Thomas A. Russ, USC/Information Sciences Institute
On 8 apr, 19:30, Jonathan Gardner <········@jonathangardner.net>
wrote:
> (1) Is it possible to make whitespace significant?
As others have said, this is not the Lisp way. Stay within normal Lisp
syntax, and use macros to make extensions. You will find that the
regularity in syntax makes your macros easily work together with
everyone else's, which IMHO invalidates much of the scared
Pythonista's reasoning that a language offering macros to everyone
unavoidably results in its users being spread over multiple
incompatible dialects.
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
Take a look at this post by Erik Naggum:
http://groups.google.com/group/comp.lang.lisp/msg/eb5555cad1c7f085
- Willem
In article
<····································@b9g2000prh.googlegroups.com>,
Willem Broekema <········@gmail.com> wrote:
> On 8 apr, 19:30, Jonathan Gardner <········@jonathangardner.net>
> wrote:
> > (1) Is it possible to make whitespace significant?
>
> As others have said, this is not the Lisp way. Stay within normal Lisp
> syntax, and use macros to make extensions. You will find that the
> regularity in syntax makes your macros easily work together with
> everyone else's, which IMHO invalidates much of the scared
> Pythonista's reasoning that a language offering macros to everyone
> unavoidably results in its users being spread over multiple
> incompatible dialects.
>
> > (2) Is it possible to give special meaning for certain characters,
> > such as ':', ';', or even ','? A case would be to write a hash-table
> > creator macro that would parse like Python:
>
> Take a look at this post by Erik Naggum:
> http://groups.google.com/group/comp.lang.lisp/msg/eb5555cad1c7f085
Or:
http://www.flownet.com/ron/lisp/dictionary.lisp
for real abstract associative maps.
rg
Jonathan Gardner wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
>
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
>
> {key-expr: value-expr, key-expr: value-expr, etc...}
You are allowed to think about such things after programming Lisp for
ten years.
hth, kenny
ps. Aren't you supposed to be learning Cello? k
--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/
"I've never read the rulebook. My job is to catch the ball."
-- Catcher Josh Bard after making a great catch on a foul ball
and then sliding into the dugout, which by the rules allowed the
runners to advance one base costing his pitcher a possible shutout
because there was a runner on third base.
"My sig is longer than most of my articles."
-- Kenny Tilton
On Apr 8, 10:30 am, Jonathan Gardner <········@jonathangardner.net>
wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant?
Yes, but this is not what I would even remotely imagine as testing the
``limits'' of redefining syntax.
The interesting limits lie in the area of semantics rather than
tokenization.
How far can you twist the semantics of the embedded sub-language
compared ot Lisp, and yet compile together with surrounding Lisp
seamlessly?
> In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
These tokens would interfere with the Lisp syntax, so you would have
to have custom syntax handling these tokens also.
You might be better off handling the indentatio in the reader and put
out a nested list in which the whitespace is no longer represented.
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
At this point, your custom reader has to take on a lot of
responsibilities. You can no longer recursively rely on the normal
Lisp reader to handle tokens for you.
When the : character occurs as the constituent of an identifier token,
it is interpreted specially as a package marker. There is no way to
change this; package marking isn't a character property in the read
table.
Thus you cannot read KEY-EXPR: using the normal reader. It is in fact
undefined syntax. (See 2.3.5 of the CLHS). Some Lisps interpret it as
the symbol named "" (empty string) in the package KEY-EXPR.
> {key-expr: value-expr, key-expr: value-expr, etc...}
So this entire syntax would have to be analyzed by your custom lexer
and parser.
If you do your own scanning, you of course handle any language for
which you can get your hands on a specification.
On Apr 8, 12:30 pm, Jonathan Gardner <········@jonathangardner.net>
wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
Taking a different slant, of what kind of language are you thinking:
a domain-specific language, a Lisp dialect, or a completely different
beast with its own syntax and semantics? Coming from the languages
you mention, and even more so if you've worked with C and things like
Lexx/Yacc, you might be predisposed to jump right into the third one.
However, with Lisp, you have some slicker options. Joe Marshall
outlined it nicely...
http://groups.google.com/group/plt-scheme/browse_thread/thread/3da987b97656a2/f4782a0aa34eca41?lnk=gst&q=learning#f4782a0aa34eca41
-Mike
P� Tue, 08 Apr 2008 19:30:48 +0200, skrev Jonathan Gardner
<········@jonathangardner.net>:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
>
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
>
> {key-expr: value-expr, key-expr: value-expr, etc...}
>
>
>
Yes, it is. Unfortunately it isn't well documented anywhere. It is called
reader macros, and I expect to write a chapter in the CL-COOKBOOK soon. In
the meantime feel free to browse the rest. (I didn't write it.)
--------------
John Thingstad
JG> (2) Is it possible to give special meaning for certain characters,
JG> such as ':', ';', or even ','? A case would be to write a hash-table
JG> creator macro that would parse like Python:
JG> {key-expr: value-expr, key-expr: value-expr, etc...}
you can do this via macro-char.
this means when reader encouters { (for example) it gives control to your
function, and it reads from the stream whatever it wants -- it defines
syntax.
but available charset is quite limited, so often people use dispatch-macro
char -- kinda combination of two chars.
i.e. you can hook #{ combination.
see examples here:
http://www.lisp.org/HyperSpec/Body/fun_set-macro_ro-character.html
http://www.lisp.org/HyperSpec/Body/fun_set-dispa_ro-character.html
also, an interesting example -- XML reader. here we dispatch on < character
and check if it's followed by a letter. if it is, we read XML via a call to
S-XML library.
otherwise, we call normal reader.
(defparameter *normal-readtable* (copy-readtable *readtable*))
(defun install-xml-reader (readtable)
(set-macro-character #\< #'try-read-xml t readtable))
(defun try-read-xml (stream char)
;;check it's an xml tag, otherwise redirect to normal reader
(unless (alpha-char-p (peek-char nil stream t nil t))
(let ((*readtable* *normal-readtable*))
(return-from try-read-xml (read stream t nil t))))
(unread-char char stream)
(s-xml:parse-xml stream))
then we enable it: (install-xml-reader *readtable*)
and use it:
CL-USER> (print '<a href="bbb">hhh</a>)
((:|a| :|href| "bbb") "hhh") ((:|a| :|href| "bbb") "hhh")
you see, it's well-integrated with Lisp.
note: i've implemented this myself and didn't test a lot, so maybe i'm doing
something wrong.. you can find better examples in some libs, like yaclml,
clsql
From: Robert Maas, see http://tinyurl.com/uh3t
Subject: Re: Limits of the Redefinition of Lisp Syntax
Date:
Message-ID: <rem-2008apr08-002@yahoo.com>
> From: Jonathan Gardner <········@jonathangardner.net>
> I'm really enjoying Lisp. Coming from perl and Python, it is
> certainly liberating.
Agree!
>However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
As far as you want. All you have to do is:
- Define precisely the syntax of the language. At this point it's
possible to write a recognizer for the syntax. If you define the
syntax via BNF, there may already be a tool to build your
recognizer directly from the BNF.
- Define precisely the mapping from the syntax definition to an
abstract parse tree composed entirely of CONS cells and atoms
(symbols and strings and numbers for example). At this point
it's possible to write a parser that takes the syntax as input
and produces the parse tree as a returned value.
- Define precisely the mapping from the parse tree to ordinary Lisp
function-calls. At this point it's possible to write a
translator from your parse tree to Lisp source, which can then
be passed to EVAL or to the compiler.
Note that you do *not* need to write the Lisp source out to a file
as text and then read it back into the compiler later, since the
compiler works directly on internal structures. This is where Lisp
beats out C and those other traditional languages in a way that's
relevant to this kind of task.
> (1) Is it possible to make whitespace significant?
Yes. It's possible to define both the syntax and the parser in that way.
> (2) Is it possible to give special meaning for certain characters,
Yes. When you define your own syntax, just about anything like that
is possible.
Perhaps you should try writing a parser+translator for some
already-existing totally-weird programming language, such as one of
these: <http://esolangs.org/wiki/Hello_world_program_in_esoteric_languages>
On Apr 8, 7:30 pm, Jonathan Gardner <········@jonathangardner.net>
wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
You are not liberated enough.
Cheers
--
Marco
PS. Ok. This is a "scare-the-newby-post" :)
Marco Antoniotti <·······@gmail.com> writes:
> On Apr 8, 7:30�pm, Jonathan Gardner <········@jonathangardner.net>
> wrote:
>> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
>> liberating.
>>
>> However, a couple of questions. I've heard that it is possible to
>> write your own language in Lisp. I want to see how far that can go.
>>
>> (1) Is it possible to make whitespace significant? In other words, is
>> it possible to have the reader put in some kind of token when it
>> detects an indent or dedent like Python does?
>
> You are not liberated enough.
On the contrary, it seems he's liberated too much and feels the
vertigo of freedom. He wants to go back to the syntactic jail.
--
__Pascal Bourguignon__
Marco Antoniotti wrote:
> On Apr 8, 7:30 pm, Jonathan Gardner <········@jonathangardner.net>
> wrote:
>
>>I'm really enjoying Lisp. Coming from perl and Python, it is certainly
>>liberating.
>>
>>However, a couple of questions. I've heard that it is possible to
>>write your own language in Lisp. I want to see how far that can go.
>>
>>(1) Is it possible to make whitespace significant? In other words, is
>>it possible to have the reader put in some kind of token when it
>>detects an indent or dedent like Python does?
>
>
> You are not liberated enough.
Yeah, I took the wrong tack.
Jonathan, when you heard it was possible to create your own language in
lisp, the idea was to create a /good/ language (meaning it will look and
feel like lisp), not recreate the crap you came from. ie, it would seem
you are not truly enjoying Lisp or you would not have as your first Lisp
goal to escape Lisp, you would instead have realized, oh, crap, Python
is the language with retarded syntax.
btw, I am not joking. Once you have done a serious amount of Lisp and
have a feel for the idioms and conventions and find your self without
trying (you were trying) creating a new language, make sure it works
like the stuff you found when you got here. Functions should do
something nice when given nil. Iterators should take test and key args
and maybe even start/end. etc etc...
hth, kenny
--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/
"I've never read the rulebook. My job is to catch the ball."
-- Catcher Josh Bard after making a great catch on a foul ball
and then sliding into the dugout, which by the rules allowed the
runners to advance one base costing his pitcher a possible shutout
because there was a runner on third base.
"My sig is longer than most of my articles."
-- Kenny Tilton
On Apr 8, 7:30 pm, Jonathan Gardner <········@jonathangardner.net>
wrote:
> I'm really enjoying Lisp. Coming from perl and Python, it is certainly
> liberating.
>
> However, a couple of questions. I've heard that it is possible to
> write your own language in Lisp. I want to see how far that can go.
>
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
Yes it is, the problem is that it will introduce unnecessary syntax.
>
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
>
> {key-expr: value-expr, key-expr: value-expr, etc...}
Of course but that will be even more syntax, few more staff and
macroprogramming will be lost, aka avaialable only to gods.
thus spoke Jonathan Gardner <········@jonathangardner.net>:
> (1) Is it possible to make whitespace significant? In other words, is
> it possible to have the reader put in some kind of token when it
> detects an indent or dedent like Python does?
It's not just macros that make uniform syntax good but structure editing
as well.
Paredit introduces a number of keystrokes for editing sexps.
Take a look at the impressive list of commands available:
http://mumble.net/~campbell/emacs/paredit.html
This wouldn't have been at all possible in Python. Vanilla Emacs already
includes a number of structure-editing commands:
(foo |(bar baz) 42)
C-M-k
(foo |42)
; "(bar baz)" got added to the kill ring
Enough to say that after binding []-brackets to
paredit-open-parenthesis/close-paren-and-newline I need much less
neurons firing for writing up the code than for appeasing the parser in
Algol-derived languages.
> (2) Is it possible to give special meaning for certain characters,
> such as ':', ';', or even ','? A case would be to write a hash-table
> creator macro that would parse like Python:
> {key-expr: value-expr, key-expr: value-expr, etc...}
Why the colon, why the comma? There's no need for such markers. Other
than that, it's pretty simple:
CL-USER> (progn
(setq *readtable* (copy-readtable))
(set-macro-character
#\{
(lambda (stream char)
(declare (ignore char))
(let ((list (read-delimited-list #\} stream))
(sym (gensym)))
`(let ((,sym (make-hash-table)))
,@(loop for (key value) on list by #'cddr
collect `(setf (gethash ,key ,sym) ,value))
,sym)))))
T
CL-USER> (read-from-string "{ :foo 42 :bar 69 \"xenu\" 666 }")
(LET ((#:G1930 (MAKE-HASH-TABLE)))
(SETF (GETHASH :FOO #:G1930) 42)
(SETF (GETHASH :BAR #:G1930) 69)
(SETF (GETHASH "xenu" #:G1930) 666)
#:G1930)
CL-USER> { :foo 42 :bar 69 "xenu" 666 }
#<HASH-TABLE :TEST EQL :COUNT 3 {B589921}>
CL-USER> (describe *)
#<HASH-TABLE :TEST EQL :COUNT 3 {B589921}> is an EQL hash table.
Its SIZE is 16.
Its REHASH-SIZE is 1.5. Its REHASH-THRESHOLD is 1.0.
It holds 3 key/value pairs: (:FOO 42) (:BAR 69) ("xenu" 666)
; No value
Note that alists outperform hash tables for less than 20 elements. Also
consider using CLOS or structures.
--
Nawet świnka wejdzie na drzewo kiedy ją chwalą.