From: Colin Walters
Subject: opening files/sockets in macros?
Date: 
Message-ID: <adrq87ryw8.church.of.emacs@localhost.verbum.private>
Hello,

I'm writing a paper which compares different macro systems like the C
Preprocessor, the Dylan macro system, the Lisp defmacro, and the
Camlp4 preprocessor, and I'm trying to describe how incredibly
powerful Lisp macros are.  I've included discussion about neat stuff
like anaphoric macros, context-building macros, even whole embedded
languages.

Now, on to my question: has anyone ever found a use for the ability to
open a file or network socket to compute the expansion of a macro?  Or
other "crazy" things like that?  Note I'm *not* talking about what the
generated code does.

I have looked through a fair amount of Lisp source, and I've never
seen anyone do something like that.  Right now I've written that
although there is no current use for this ability, I wouldn't bet
against it being useful in the future.  But I'd love to be proven
wrong on the first part...

From: Geoff Summerhayes
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <J6jy8.38052$zj6.1052873@news2.calgary.shaw.ca>
"Colin Walters" <·······@debian.org> wrote in message
·······························@localhost.verbum.private...
> Hello,
>
> I'm writing a paper which compares different macro systems like the C
> Preprocessor, the Dylan macro system, the Lisp defmacro, and the
> Camlp4 preprocessor, and I'm trying to describe how incredibly
> powerful Lisp macros are.  I've included discussion about neat stuff
> like anaphoric macros, context-building macros, even whole embedded
> languages.
>

I'd be interested in reading this if you are willing/allowed to
put it on the web for public viewing after.

> Now, on to my question: has anyone ever found a use for the ability to
> open a file or network socket to compute the expansion of a macro?  Or
> other "crazy" things like that?  Note I'm *not* talking about what the
> generated code does.
>

Strictly my opinion, but this sounds like an incredibly bad idea
since the programmer has little say over when or how many times
a macro-expansion is created. I believe Graham mentions somewhere
(OnLisp?) that something along the lines of

(let ((count 0))
  (defmacro foo(...)(incf count)`(...))(defun foo-called() count))

is never going to be a reliable guide for any use. I think any
code along these lines would almost certainly be non-portable.

----------
Geoff
From: Colin Walters
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <662e87rqkg.church.of.emacs@localhost.verbum.private>
"Geoff Summerhayes" <·············@hNoOtSmPaAiMl.com> writes:

> I'd be interested in reading this if you are willing/allowed to
> put it on the web for public viewing after.

Of course.

> Strictly my opinion, but this sounds like an incredibly bad idea
> since the programmer has little say over when or how many times a
> macro-expansion is created. I believe Graham mentions somewhere
> (OnLisp?) that something along the lines of
> 
> (let ((count 0))
>   (defmacro foo(...)(incf count)`(...))(defun foo-called() count))
> 
> is never going to be a reliable guide for any use.

Right.  But can you say for sure that someone would *never* want to do
something like this?  That's my question.

> I think any code along these lines would almost certainly be
> non-portable.

It would be certainly be portable.  However, the specification does
leave a lot up to the implementation, meaning one can't rely on much
when writing macros like that.  I think it's still possible that there
are interesting things one could do that could be accomplished despite
that ambiguity.
From: Geoffrey Summerhayes
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <5TEy8.91674$QR1.3715676@news20.bellglobal.com>
"Colin Walters" <·······@debian.org> wrote in message
·······························@localhost.verbum.private...
> "Geoff Summerhayes" <·············@hNoOtSmPaAiMl.com> writes:
>
> > I think any code along these lines would almost certainly be
> > non-portable.
>
> It would be certainly be portable.  However, the specification does
> leave a lot up to the implementation, meaning one can't rely on much
> when writing macros like that.  I think it's still possible that there
> are interesting things one could do that could be accomplished despite
> that ambiguity.

OK. After looking at the other responses I can see there is
more to this than my first-blush, gut reaction. It would have
to be approached with extreme care though, the thought of
nested macros attempting access to the same file/port during
a compile give me the willies. Not to mention trying to debug
the thing under varying conditions.

----------
Geoff
From: Tim Moore
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <aacg38$gv8$0@216.39.145.192>
On 26 Apr 2002 16:24:55 -0400, Colin Walters <·······@debian.org> wrote:

>Now, on to my question: has anyone ever found a use for the ability to
>open a file or network socket to compute the expansion of a macro?  Or
>other "crazy" things like that?  Note I'm *not* talking about what the
>generated code does.

Sure.  With the cparse package you can extract FFI definitions from a
C header file.  So, though cparse doesn't provide this, it's useful to
write a macro that returns the results of cparse as a big progn.
Obviously cparse is opening a file :)

www.bricoworks.com/~moore/cparse

Tim
From: Rainer Joswig
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <joswig-089FCE.12031127042002@news.fu-berlin.de>
In article <············@216.39.145.192>,
 ······@sea-tmoore-l.dotcast.com (Tim Moore) wrote:

> On 26 Apr 2002 16:24:55 -0400, Colin Walters <·······@debian.org> wrote:
> 
> >Now, on to my question: has anyone ever found a use for the ability to
> >open a file or network socket to compute the expansion of a macro?  Or
> >other "crazy" things like that?  Note I'm *not* talking about what the
> >generated code does.
> 
> Sure.  With the cparse package you can extract FFI definitions from a
> C header file.  So, though cparse doesn't provide this, it's useful to
> write a macro that returns the results of cparse as a big progn.
> Obviously cparse is opening a file :)
> 
> www.bricoworks.com/~moore/cparse
> 
> Tim

Macintosh Common Lisp does it for its foreign function interface.

Example:
To expand the following form (using some random record type AUXDCE
from MacOS), MCL looks up the definition from the FFI database
and caches the definition.

(rlet ((foo :auxdce))
  foo
  nil)
From: Colin Walters
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <8z7887kgc6.church.of.emacs@localhost.verbum.private>
Rainer Joswig <······@lispmachine.de> writes:

> Macintosh Common Lisp does it for its foreign function interface.

Ok, generation of FFI definitions seem to be quite a legitimate use
for opening a file in a macro.

Thanks to everyone who responded!
From: Gabe Garza
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <it6evu54.fsf@kynopolis.org>
Colin Walters <·······@debian.org> writes:

> Hello,
> 
> I'm writing a paper which compares different macro systems like the C
> Preprocessor, the Dylan macro system, the Lisp defmacro, and the
> Camlp4 preprocessor, and I'm trying to describe how incredibly
> powerful Lisp macros are.  I've included discussion about neat stuff
> like anaphoric macros, context-building macros, even whole embedded
> languages.
> 
> Now, on to my question: has anyone ever found a use for the ability to
> open a file or network socket to compute the expansion of a macro?  Or
> other "crazy" things like that?  Note I'm *not* talking about what the
> generated code does.
> 
> I have looked through a fair amount of Lisp source, and I've never
> seen anyone do something like that.  Right now I've written that
> although there is no current use for this ability, I wouldn't bet
> against it being useful in the future.  But I'd love to be proven
> wrong on the first part...

A use?  How about macro designed for handling SQL queries that, during
its expansion (if it was allowed to connect and the programmer had
indicated that he wasn't going to be changing the database schema) queried
the database for the types of the selected columns so that it could write
more efficient foreign-function glue?  

For example, a macro that would transform a form like this:

(do-select ("SELECT dog, owner FROM dogs" *link*)
  ...code-that-uses-dog-and-owner-a-lot..)

Into this:

(let ((dog-place (allocate-memory 4 :type :int))
      (owner-place (allocate-memory 512 :type '(:unsigned :char))))
  (let ((handle (prepare-statement "SELECT dog, owner FROM dogs" *link*)))
    (bind-column dog-place 0 handle)       
    (bind-column owner 1 handle))
  (let ((cursor (execute-statement handle)))
    (loop while (not (empty cursor))
          do (let ((dog (create-lisp-integer dog-place))
                   (owner (create-lisp-string owner-place 512)))
                ...code-that-uses-dog-and-owner-a-lot...))))

Some C database interfaces [Oracle!] are actually that bad....

There are of course variations of this general theme: you could store
the query on the server at expansion-time, make sure you're selecting
valid columns and tables at expansion-time, etc.

Gabe Garza
From: Bob Bane
Subject: Re: opening files/sockets in macros?
Date: 
Message-ID: <3CCD7CF3.E313674A@removeme.gst.com>
Colin Walters wrote:
> 
> Now, on to my question: has anyone ever found a use for the ability to
> open a file or network socket to compute the expansion of a macro?
> ...

How about something like this?

(with-dtd "http://www.mycompany.com/dtds/ourschema.dtd"
  ;; WITH-DTD could expand into a FLET/MACROLET that gave you
  ;; conveneient accessors based on the tags in a DTD...
 )

This is similar in spirit to the FFI suggestions above, but it has to
open a socket to resolve the URL.