How do Lisp Compilers handle Code as Data and vice-versa?
I presume they have to imbed a Lisp interpreter into the application?
Do all Common Lisp compilers handle this correctly?
Le Wed, 23 Aug 2006 05:44:29 -0700, Sean SCC a écrit :
> How do Lisp Compilers handle Code as Data and vice-versa?
Easy: code IS data. You can't tell the difference without trying to
execute it.
> I presume they have to imbed a Lisp interpreter into the application?
You don't need an interpreter, but a *reader*. When it sees an opening
parenthesis, it begins building a singly-linked list. For each item in
the data that it reads, it creates a link (called a cons cell). The
process is recursive, so nested lists are allowed.
For the reader, those are lists:
(do (somethin useful) and (maybe) (bad stuff))
(1 2 3 4)
(dolist (i (list 1 2 3)) (print i))
Then, lists given in a source file or at the REPL are considered code,
and as such, given to either the compiler or the interpreter to be
executed, except in some special cases like macros. List that are
conformant code, that is code that can succesfully be executed, are said
to be "Lisp forms" (among the three previous lists, only the last is a
Lisp form).
As code, the (list 1 2 3) list is a Lisp form that yields the list (1 2
3), that is a list containing three numbers. As data, (list 1 2 3) is a
list containing a symbol and three numbers.
> Do all Common Lisp compilers handle this correctly?
This is probably the easiest part of handling CL... That's precisly a
strength of all Lisp dialects. I would expect all CL implementations to
handle it perfectly!
Quickly,
Nowhere man
--
···········@levallois.eu.org
OpenPGP 0xD9D50D8A
From: Pascal Bourguignon
Subject: Re: How do Lisp Compilers handle Code as Data and vice-versa?
Date:
Message-ID: <874pw34l82.fsf@informatimago.com>
"Sean SCC" <···········@googlemail.com> writes:
> How do Lisp Compilers handle Code as Data and vice-versa?
Have a look at this small example:
http://groups.google.com/group/comp.lang.lisp/msg/2108939e399c5c4e
> I presume they have to imbed a Lisp interpreter into the application?
Rather, a compiler. All lisp application include a compiler. So they
can do powerfull things at run-time, like modifying themselves, adding
to themselves new functions, new behavior, learn and adapt to changing
environement, like any intelligent being.
> Do all Common Lisp compilers handle this correctly?
Yes. It's rather trivial. It was invented in 1959. It's well known
technology.
--
__Pascal Bourguignon__
===============================================
== All lisp application include a compiler. So they can do
== powerfull things at run-time, like modifying themselves,
== adding to themselves new functions, new behavior, learn
== and adapt to changing environement, like any intelligent being.
==
== ( Pascal Bourguignon )
===============================================
WOW, All I can say is WOW. I kind of knew this already but never seen
it put so clearly. Now if THAT isn't reason enough to learn and try to
master Lisp then I don't know what is, especially when you realize what
else Lisp can do, how quickly one develop in it, how maintainable the
result is and how quick the application can execute!
I am taking some of this on faith - sure hope you guys are right!
:-)
Let's say that you are in a running instance of a standalone Lisp
application. This application is a game and you bring up an in-game
console window. You type in a small lisp function and press enter...
> (defun get-gravity () 9.8)
1) The application feeds that form to its internal compiler. The new
function is created in freshly allocated memory from the heap.
2) The symbol GET-GRAVITY's function pointer is changed to point to the
new function. The old function that was pointed to is now garbage, to
be collected at a later time.
3) You exit the console and let the game resume. The next frame,
GET-GRAVITY is called. This is done by looking up GET-GRAVITY in the
symbol hashtable. The returned function is the new one, not the old
one.
That's it. I think that this is correct. :-D The overhead of the
hashtable lookups is really not bad when you consider how much
flexibility it gives you. I think that commercial lisp implementations
allow a programmer to specify that a function will not be replacable,
and will force callers to use a pointer. Corrections encouraged.
From: Ari Johnson
Subject: Re: How do Lisp Compilers handle Code as Data and vice-versa?
Date:
Message-ID: <m2zmdv1nk1.fsf@hermes.theari.com>
"pTymN" <·········@gmail.com> writes:
> Let's say that you are in a running instance of a standalone Lisp
> application. This application is a game and you bring up an in-game
> console window. You type in a small lisp function and press enter...
>
>> (defun get-gravity () 9.8)
>
> 1) The application feeds that form to its internal compiler. The new
> function is created in freshly allocated memory from the heap.
>
> 2) The symbol GET-GRAVITY's function pointer is changed to point to the
> new function. The old function that was pointed to is now garbage, to
> be collected at a later time.
>
> 3) You exit the console and let the game resume. The next frame,
> GET-GRAVITY is called. This is done by looking up GET-GRAVITY in the
> symbol hashtable. The returned function is the new one, not the old
> one.
>
> That's it. I think that this is correct. :-D The overhead of the
> hashtable lookups is really not bad when you consider how much
> flexibility it gives you. I think that commercial lisp implementations
> allow a programmer to specify that a function will not be replacable,
> and will force callers to use a pointer. Corrections encouraged.
Since the symbol-function of the GET-GRAVITY symbol is changed, when
GET-GRAVITY is called in the next frame there is no need look up the
symbol as its address is coded into the compiled frame-generation
code. You just have to look up the symbol-function of the symbol each
frame, which is fast and easy to do - it's the equivalent of a
constant-index array access.
pTymN wrote:
> Let's say that you are in a running instance of a standalone Lisp
> application. This application is a game and you bring up an in-game
> console window. You type in a small lisp function and press enter...
>
> > (defun get-gravity () 9.8)
>
> 1) The application feeds that form to its internal compiler. The new
> function is created in freshly allocated memory from the heap.
>
> 2) The symbol GET-GRAVITY's function pointer is changed to point to the
> new function. The old function that was pointed to is now garbage, to
> be collected at a later time.
>
> 3) You exit the console and let the game resume. The next frame,
> GET-GRAVITY is called. This is done by looking up GET-GRAVITY in the
> symbol hashtable. The returned function is the new one, not the old
> one.
This is generally correct, however actually making it work requires
additional magic on many architectures and operating systems. You have
to both:
1. Ensure that the instruction cache is changed as well as the data
cache. This is frequently expensive. No problem for debugging, but
not something you want a program to do too often.
2. Ensure that the security, anti-virus, etc. stuff in the OS will let
you write data into memory then execute it. This is a common cracking
technique and Lisp is not that common so some of these protections
throw the baby out with the bath water.
Neither of these are likely to be problems if you're using an existing
Lisp, but they are interesting for the implementor...
On Wed, 23 Aug 2006 07:54:29 -0700, pTymN wrote:
> Let's say that you are in a running instance of a standalone Lisp
> application. This application is a game and you bring up an in-game
> console window. You type in a small lisp function and press enter...
>
>> (defun get-gravity () 9.8)
>
> 1) The application feeds that form to its internal compiler. The new
> function is created in freshly allocated memory from the heap.
>
> 2) The symbol GET-GRAVITY's function pointer is changed to point to the
> new function. The old function that was pointed to is now garbage, to
> be collected at a later time.
>
> 3) You exit the console and let the game resume. The next frame,
> GET-GRAVITY is called. This is done by looking up GET-GRAVITY in the
> symbol hashtable. The returned function is the new one, not the old
> one.
>
> That's it. I think that this is correct. :-D The overhead of the
> hashtable lookups is really not bad when you consider how much
> flexibility it gives you. I think that commercial lisp implementations
> allow a programmer to specify that a function will not be replacable,
> and will force callers to use a pointer. Corrections encouraged.
cl-user> (let ((fun 'it))
(defun do-it ()
(funcall fun)))
do-it
cl-user> (defun it ()
"first version")
it
cl-user> (do-it)
"first version"
cl-user> (defun it ()
"second version")
style-warning: redefining it in DEFUN
it
cl-user> (do-it)
"second version"
cl-user>
; No value
cl-user> (let ((fun #'it))
(defun do-it ()
(funcall fun)))
style-warning: redefining do-it in DEFUN
do-it
cl-user> (do-it)
"second version"
cl-user> (defun it ()
"third version")
style-warning: redefining it in DEFUN
it
cl-user> (do-it)
"second version"
cl-user>
#'it will refer directly.
--
Lars Rune Nøstdal
http://lars.nostdal.org/
Well, not sure you're asking about something like this .. O_o .. i just
remember bumping into an issue with 'func versus #'func that seemed
similar to what you describe. :)
--
Lars Rune Nøstdal
http://lars.nostdal.org/
Sean SCC wrote:
> ===============================================
> == All lisp application include a compiler. So they can do
> == powerfull things at run-time, like modifying themselves,
> == adding to themselves new functions, new behavior, learn
> == and adapt to changing environement, like any intelligent being.
> ==
> == ( Pascal Bourguignon )
> ===============================================
>
> WOW, All I can say is WOW. I kind of knew this already but never seen
> it put so clearly. Now if THAT isn't reason enough to learn and try to
> master Lisp then I don't know what is, especially when you realize what
> else Lisp can do, how quickly one develop in it, how maintainable the
> result is and how quick the application can execute!
>
> I am taking some of this on faith - sure hope you guys are right!
Well you can certainly modify the behaviour of the program, it's quite
an advanced technique. Most programs don't need it.
It can be useful though. Imagine if your program contains a special
purpose language that describes something specialised, say a 3D shape.
You could write a piece of lisp code that converts the special purpose
language into lisp. Then it could be compiled at run-time and
executed.
I think it's most useful for debugging. You run the program until a
problem is found, then you can change the bit that's wrong in the
running program. There isn't a need to recompile everything.
Sean SCC wrote:
> ===============================================
> == All lisp application include a compiler. So they can do
> == powerfull things at run-time, like modifying themselves,
> == adding to themselves new functions, new behavior, learn
> == and adapt to changing environement, like any intelligent being.
> ==
> == ( Pascal Bourguignon )
> ===============================================
>
> WOW, All I can say is WOW. I kind of knew this already but never seen
> it put so clearly. Now if THAT isn't reason enough to learn and try to
> master Lisp then I don't know what is, especially when you realize what
> else Lisp can do, how quickly one develop in it, how maintainable the
> result is and how quick the application can execute!
>
> I am taking some of this on faith - sure hope you guys are right!
>
> :-)
>
Of course, the programmer has to have some real skill to take advantage
of this capability; I can't say I'm there yet myself. Also, getting
Common Lisp to execute a form constructed at run-time in-context (ie:
with lexical environments) is impossible, so I'd only build
pure-functional code at run-time.
Of course, there could be secrets and spells which remain unknown.
--
The science of economics is the cleverest proof of free will yet
constructed.
In article <························@p79g2000cwp.googlegroups.com>,
"Sean SCC" <···········@googlemail.com> wrote:
> How do Lisp Compilers handle Code as Data and vice-versa?
>
> I presume they have to imbed a Lisp interpreter into the application?
> Do all Common Lisp compilers handle this correctly?
The only time that data becomes code is if you call a function like
EVAL. That function is either an interpreter or a JIT compiler. So
either an interpreter or compiler is included as part of the Lisp
runtime library.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***