From: Ari Johnson
Subject: Lisp backend protocol
Date: 
Message-ID: <ari-FB507D.21144501042006@nntp.aioe.org>
I was thinking the other day about why Lisp is so hard to use.  I came 
to the conclusion that it is hard to use for different reasons in 
different areas, and I came up with an idea.  Stop me if this has been 
done. :)

Lisp is hard to use for small scripts because of the cost of loading the 
image and the fact that it is a bit of work to load up all the things 
you need.  For instance, compare the amount of code it would take in 
Lisp versus in Perl to calculate some statistics from an Apache log 
file.  In Lisp, you would have to do a bit of portability work to make 
sure that you have the right things available to you, then load up 
PPCRE, then initialize a bunch of stuff, and then finally scan the logs.  
In Perl, you can probably accomplish this with a perl -e 'code here' 
script.

Lisp is hard to use for web applications mostly because PHP is so much 
easier, but also because it is hard to interface Lisp with the web out 
of the box.  You can use mod-lisp or reverse proxying to a Lisp web 
server, but that just isn't easy.  Compare with PHP, which is a rather 
crummy programming language but a great HTML template language when used 
appropriately.  Add in the fact that PHP is just there - you can drop a 
.php file into your web directory and expect it to work on the majority 
of server installations out there.

Lisp is hard to use for heavy-lifting backend applications because it is 
hard to interface with the outside world other than through the web (see 
above).  CLIs just aren't cool anymore, and GUIs have to talk to the 
backend somehow so they don't really count here (writing a GUI 
application in Lisp is hard for its own reasons).

Time for an analogy...

"Any sufficiently complicated C or Fortran program contains an ad-hoc, 
informally-specified bug-ridden slow implementation of half of Common 
Lisp."

Let's distill this axiom into the following: "Any sufficiently 
complicated program to do X contains an ad-hoc, informally-specified, 
bug-ridden, slow implementation of half of X-language."

(let ((X "database")
      (X-language "SQL"))
  (make-analogy))

When you need to store many pieces of data, all of which have the same 
fields, you don't go around writing your own database from scratch.  You 
connect to an SQL server (be it MySQL, Postgres, Oracle, or even MSSQL) 
and let it do all the database work, because otherwise you are just 
going to end up with a buggy implementation of half of SQL anyhow.

However, SQL is utterly abysmal for writing anything other than the 
bottom-level data storage of your application in.  Only a true masochist 
tries to write an entire web application in SQL.  Instead, let's say you 
are doing MVC-style development - you write part of the M(odel) (the 
data half of it and possibly some stored-procedure logic) in SQL and the 
rest of the M plus the V(iew) and the C(ontroller) in some other 
language.  For a web application, PHP is typical enough but the actual 
language doesn't matter.  What matters is that it is an easy language to 
write the model logic, the view, and the controller in.

Sufficiently complex web applications may actually code the V and C in 
one language and split the M between two other languages - perhaps C and 
SQL.  But doing so is painful, at best.

Regardless, you use an SQL server because it does the data storage for 
you for free.  In PHP, for instance, you just write:
  $db = pg_connect($connstr);
  $res = pg_query($db, $query);
  $array = pg_fetch_array($res, null, PGSQL_ASSOC);

You simply write your database schema in SQL (mostly by creating tables) 
and then connect to it from your implementation language.

(let ((X "application logic")
      (X-language "Common Lisp"))
  (continue-analogy))

Lisp is a powerful, fun, and easy language to code application logic in.  
It is a boring and ill-suited language to code a user interface in (at 
least a web interface, which constitutes a substantial portion of all 
interfaces in present use anyhow).

Why treat Lisp any differently than SQL?  Why not let people write their 
application logic in Lisp and then connect to it from their user 
interface language of choice?
  $lisp = cl_connect($connstr);
  $res = cl_call($lisp, $fun, $args);

You could even make it look like some fancy XML-RPC libraries do, 
especially in fun languages like Ruby, where you can directly turn an 
XML-RPC interface into a Ruby class interface.
  $iface = cl_connect($connstr);
  $res = $iface->fun($arg1, $arg2);

Thoughts so far on the overall idea?

As it sits in my mind right now, the idea would be to have a protocol 
that runs over a socket and consists of the following operations:

1. Connect and authenticate with username, password, and package name.  
The package name would be a string and you would essentially log into a 
package and have access to the functions named by its exported symbols.

2. Make a function call to a function in the connected package with 
certain arguments, and get back the (multiple) values that the function 
returns.  Functions you can call would be limited to exported symbols in 
the connected package, for security purposes.  Use #'find-package in 
step 1 and #'find-symbol with that package passed to it in this step.

3. (Optional) Cancel a running process.  This of course would require 
that function calls be made in separate threads, but allows an 
application to cancel a calculation that it will be ignoring the results 
from anyhow.

The second of these would be asynchronous - you could submit your 
request and check back on the socket anytime you want for the answer.  
This means you can have both blocking and non-blocking calls from the 
user interface code (PHP's Postgres interface allows the same thing).

Is there a precedent for any such thing?  Would it be a 
best-of-both-worlds or worst-of-both-worlds feature?  Discuss. :)

Ari Johnson

From: Rob Warnock
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <wNSdneledefmp7LZnZ2dneKdnZydnZ2d@speakeasy.net>
Ari Johnson  <···@theari.com> wrote:
+---------------
| Is there a precedent for any such thing?
+---------------

Congratulations: You've just re-invented IPC and/or RPC.
See "man ipc", "man rpc" and "man rpc_secure", or given
which group this is, maybe <http://www.cliki.net/IPC>.
As always, the devil is in the details...


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <ari-E63E80.22102201042006@nntp.aioe.org>
In article <································@speakeasy.net>,
 ····@rpw3.org (Rob Warnock) wrote:

> Ari Johnson  <···@theari.com> wrote:
> +---------------
> | Is there a precedent for any such thing?
> +---------------
> 
> Congratulations: You've just re-invented IPC and/or RPC.
> See "man ipc", "man rpc" and "man rpc_secure", or given
> which group this is, maybe <http://www.cliki.net/IPC>.
> As always, the devil is in the details...

I haven't reinvented IPC.  I've suggested a layer on top of it for the 
express purpose of making Lisp backend logic available to other-language 
frontends.  Does anything already exist for *that*?

The devil is, indeed, in the details.  What I'm getting at is coding up 
the details and having them available in an easy-to-use package.

As a simple example:

--blah.lisp--
(asdf:operate 'asdf:load-op 'clipc)

(in-package "FACT")

(defun fact (n)
  (if (> n 1)
    (* n (fact (1- n)))
    1))

(defparameter *user-list* '(("dummy" . (encrypt "passwd"))))

(clipc:startup :port 4222)
(clipc:expose *package* *user-list*)
(clipc:export 'fact)
----

--blah.php--
<?php
  require("clipc");
  $lisp = clipc_connect("host=localhost port=4222 package=fact 
user=dummy password=passwd");
  list($res) = clipc_call("fact", 3);
?>
The factorial of 3 is <?= $res ?>.
----

Does anything exist that does all of this already?
From: Rob Warnock
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <P_ydnY74Nr03zrLZRVn-tQ@speakeasy.net>
Ari Johnson  <···@theari.com> wrote:
+---------------
|  ····@rpw3.org (Rob Warnock) wrote:
| > Congratulations: You've just re-invented IPC and/or RPC.
| 
| I haven't reinvented IPC.  I've suggested a layer on top of it for the 
| express purpose of making Lisp backend logic available to other-language 
| frontends.  Does anything already exist for *that*?
+---------------

It certainly *exists*, though probably not as a preconfigured
open-source package one can just slurp off the 'Net. For myself,
when I was starting to use CL seriously and was worried about
"startup time" I actually used "mod_lisp" protocol for some of
that sort of thing, since I always have a persistent Lisp-based
web server running anyway. A simple shell script wrapped around
a "sock.cgi" C program [that connects to a Unix-domain socket
and speaks "mod_lisp" protocol to it] was enough to fake up a
web request, and the answers came out of the standard-output of
the C program, so you could write script that looked like this:

    #!/bin/sh
    env HTTP_HOST=localhost \
	REQUEST_METHOD=MAGIC_LOCAL_GET \
	MAGIC_LOCAL_AUTH=cGFzc3dvcmQ9ZnJlZWJsZXNuYXR6 \
	REQUEST_URI=/ipc/run_function/foo \ | sock.cgi | whatever...

But then I found out that simple "scripting" with CMUCL is plenty
fast for me, so I stopped doing the above:

    $ cat test.lisp
    #!/usr/local/bin/cmucl -script
    (format t "hello world!~%")
    $ ./test.lisp
    hello world!
    $ time-hist ./test.lisp
    Timing 100 runs of: ./test.lisp
      54 0.016
      46 0.017
    1.156u 1.283s 0:02.88 84.3%     264+1700k 1+0io 2pf+0w
    $ 

Since 16ms is way faster than *my* reaction time, I figure it's
"fast enough"...

And if you're talking about larger programs that might want to
use a bunch of libraries, well, just save a Lisp image with all
that stuff in it already and your own startup function, and you
can run that from a script pretty quickly, too. Here's an example
using CMUCL [the "bin/" and "lib/" subdirectories are so CMUCL
can find its core file without an explicit "-core" in the script
"#!" line, since some operating systems restrict that to a single
additional argument]:

    $ mkdir -p ~/bin/cmu-images/foo/{bin,lib}
    $ ln /abs/path/to/standard/cmucl ~/bin/cmu-images/foo/bin
    $ ~/bin/cmu-images/foo/bin/lisp
    ...[lots of chatter]...
    * (asdf :cl-ppcre)
    ; loading system definition from library:local/systems/cl-ppcre.asd into
    ; #<The ASDF1497 package>
    ; Loading #p"/u/lisp/contrib/cl-ppcre-0.7.6/cl-ppcre.asd".
    ...[lots more chatter]...
    * ...[whatever else you want to load]...
    ...[lots more chatter]...
    * (defun main ()
	(format t "MAIN called:~%")
	(format t "*COMMAND-LINE-STRINGS* = ~s~%" ext:*command-line-strings*)
	(let ((script (cadr ext:*command-line-strings*)))
	  (when script
	    (format t "Script contents:~%")
	    (with-open-file (s script)
	      (loop for line = (read-line s nil nil)
		    and line-number from 1
		    while line do
		(format t "Line ~d:~8t~s~%" line-number line)))))
	(unix:unix-exit 0))

    MAIN
    *  (save-lisp "/u/rpw3/bin/cmu-images/foo/lib/lisp.core"
		  :init-function #'main
		  :load-init-file nil
		  :site-init nil
		  :print-herald nil
		  :batch-mode t)
    [Doing purification: Done.]
    [Undoing binding stack... done]
    [Saving current lisp image into /u/rpw3/bin/cmu-images/foo/lib/lisp.core:
    Writing 20608624 bytes from the Read-Only space at 0x10000000.
    Writing 3198464 bytes from the Static space at 0x28F00000.
    Writing 4096 bytes from the Dynamic space at 0x48000000.
    done.]

    $ cat >~/bin/foo
    #!/u/rpw3/bin/cmu-images/foo/bin/lisp
    First line of script
    Second line of script
    Last line of script
    ^D
    $ chmod +x ~/bin/foo
    $ foo bar baz
    MAIN called:
    *COMMAND-LINE-STRINGS* = ("/u/rpw3/bin/cmu-images/foo/bin/lisp"
			      "/u/rpw3/bin/foo" "bar" "baz")
    Script contents:
    Line 1: "#!/u/rpw3/bin/cmu-images/foo/bin/lisp"
    Line 2: "First line of script"
    Line 3: "Second line of script"
    Line 4: "Last line of script"
    $ time-hist foo bar baz
    Timing 100 runs of: foo bar baz
      38 0.016
      34 0.017
      28 0.018
    1.283u 1.218s 0:02.55 97.6%     263+1726k 0+0io 0pf+0w
    $

Note: I'm not saying having a persistent Lisp server running
to service calls from other languages is a *bad* thing -- it
indeed can be useful under some circumstances [although designing
the security protocols can be "interesting"]. But there are
much better ways of addressing the "startup time" boogieman.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <ari-44DC9B.23584101042006@nntp.aioe.org>
In article <······················@speakeasy.net>,
 ····@rpw3.org (Rob Warnock) wrote:
> Note: I'm not saying having a persistent Lisp server running
> to service calls from other languages is a *bad* thing -- it
> indeed can be useful under some circumstances [although designing
> the security protocols can be "interesting"]. But there are
> much better ways of addressing the "startup time" boogieman.

The real bogeyman I was trying to kill is the one that keeps a lot of
people from using Lisp for anything serious - they think it's a great
language but that it's impractical to interface with people in.  This
is a valid point and one that a shrink-wrapped user-friendly
all-inclusive IPC would alleviate.

Writing the heavy-lifting code in Lisp and the user interface code in
PHP would sufficiently bridge the comfortability gap to make Lisp
practical for many applications, where right now people say that
"Lisp would be great to do this in, but I don't want to deal with
writing HTML with sexps all day long.  I want to write HTML and plug
in data, and just have Lisp do the background calculations."

A system like PHP with Lisp as the internal language would work, and
I believe one or more already exist to some degree of usability,
but they tend to be rough to configure in a working fashion under
Apache, in my (admittedly limited) experience.

The idea isn't (and never should be) to solve all the world's problems
with one tool.  The idea is just to make a screwdriver with a head
that fits a particular screw.
From: Christophe Rhodes
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <sqd5g0cu0j.fsf@cam.ac.uk>
Ari Johnson <···@theari.com> writes:

> In article <······················@speakeasy.net>,
>  ····@rpw3.org (Rob Warnock) wrote:
>> Note: I'm not saying having a persistent Lisp server running
>> to service calls from other languages is a *bad* thing -- it
>> indeed can be useful under some circumstances [although designing
>> the security protocols can be "interesting"]. But there are
>> much better ways of addressing the "startup time" boogieman.
>
> The real bogeyman I was trying to kill is the one that keeps a lot of
> people from using Lisp for anything serious - they think it's a great
> language but that it's impractical to interface with people in.  This
> is a valid point 

I don't think it is valid.

> and one that a shrink-wrapped user-friendly all-inclusive IPC would
> alleviate.

No, it wouldn't.

People always find some reason not to use something they don't want to
use; there are probably all sorts of psychological observations one
could make which I will leave to someone more qualified.  Instead, I
will offer the advice not to implement something in or for Lisp
because you think other people might want it; implement it because you
need it, and that way if other people have similar needs, those needs
might be met by your software.  Implementing something because you
think it's the last thing needed for Lisp to become popular is a
recipe for frustration.

Christophe
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <ari-0636F7.09464602042006@nntp.aioe.org>
In article <··············@cam.ac.uk>,
 Christophe Rhodes <·····@cam.ac.uk> wrote:

> Ari Johnson <···@theari.com> writes:
> 
> > In article <······················@speakeasy.net>,
> >  ····@rpw3.org (Rob Warnock) wrote:
> >> Note: I'm not saying having a persistent Lisp server running
> >> to service calls from other languages is a *bad* thing -- it
> >> indeed can be useful under some circumstances [although designing
> >> the security protocols can be "interesting"]. But there are
> >> much better ways of addressing the "startup time" boogieman.
> >
> > The real bogeyman I was trying to kill is the one that keeps a lot of
> > people from using Lisp for anything serious - they think it's a great
> > language but that it's impractical to interface with people in.  This
> > is a valid point 
> 
> I don't think it is valid.

I have a counter-example: myself.  I do not do web design, and I am not
good at it.  I like to have experts at that sort of thing write the
HTML and CSS for my web applications.  I am trying to figure out the
best way to be able to write those applications in Lisp without
compromising the ability to painlessly turn HTML templates handed
to me with dummy data into active templates that get filled in with
real data.  I need to be able to do so, like I said, painlessly.
It would be even more helpful if I could provide a template system
that my non-programmer interface designers can use on their own
without my intervention.

> 
> > and one that a shrink-wrapped user-friendly all-inclusive IPC would
> > alleviate.
> 
> No, it wouldn't.
> 
> People always find some reason not to use something they don't want to
> use; there are probably all sorts of psychological observations one
> could make which I will leave to someone more qualified.  Instead, I
> will offer the advice not to implement something in or for Lisp
> because you think other people might want it; implement it because you
> need it, and that way if other people have similar needs, those needs
> might be met by your software.  Implementing something because you
> think it's the last thing needed for Lisp to become popular is a
> recipe for frustration.

I have no desire not to use Common Lisp to write application logic.
In fact, quite to the contrary, I am on the verge of reimplementing
a massive PHP application that I wrote years ago, this time in Lisp.
This application received 47,000 hits yesterday, a fairly typical
sum that is occasionally eclipsed by numbers in the 60,000's.  I am
not going to put my poor server through 50,000+ Lisp environment
initializations in a day if I can help it.  I am also not going to
tell my web designers "You must write your HTML in sexps and must
do so perfectly or else no output at all will result."

I am not harboring, and have in fact explicitly disclaimed elsewhere
on this thread, any desire to provide "the last thing needed for Lisp
to become popular."  I am merely looking at the situation I am in,
recognizing that it is likely not a unique situation, and trying to
envision the best way to solve it.  To my knowledge, no existing
software does solve it - if something does, I am all ears.  If
nothing does, then I remain all ears as to the best approach to
take in solving it.

Thanks. :)
From: Lars Rune Nøstdal
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1143992663.780103.167430@i40g2000cwc.googlegroups.com>
> I am trying to figure out the best way to be able to write those
> applications in Lisp without compromising the ability to painlessly
> turn HTML templates handed to me with dummy data into
> active templates that get filled in with real data.  I need to be
> able to do so, like I said, painlessly.

HTML-TEMPLATE is great for this kind of stuff. Check it out:
http://www.weitz.de/html-template/

-- 
Lars Rune Nøstdal
http://lars.nostdal.org/
From: Wade Humeniuk
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <qMSXf.28700$%H.2399@clgrps13>
Ari Johnson wrote:
> 
> I have no desire not to use Common Lisp to write application logic.
> In fact, quite to the contrary, I am on the verge of reimplementing
> a massive PHP application that I wrote years ago, this time in Lisp.
> This application received 47,000 hits yesterday, a fairly typical
> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
> not going to put my poor server through 50,000+ Lisp environment
> initializations in a day if I can help it.  I am also not going to
> tell my web designers "You must write your HTML in sexps and must
> do so perfectly or else no output at all will result."
> 

What else is your server going to do all day??  As 60,000 is less
than 1/sec, your machine will be idle most of the day.  Its
interesting that you ascribe feelings to your machine, as if
it is being flogged to death.  Perhaps you have been around the
piggishness of PHP and the template approach too long.

Wade
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0otvm$hnh$1@emma.aioe.org>
Wade Humeniuk wrote:
> Ari Johnson wrote:
>>
>> I have no desire not to use Common Lisp to write application logic.
>> In fact, quite to the contrary, I am on the verge of reimplementing
>> a massive PHP application that I wrote years ago, this time in Lisp.
>> This application received 47,000 hits yesterday, a fairly typical
>> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
>> not going to put my poor server through 50,000+ Lisp environment
>> initializations in a day if I can help it.  I am also not going to
>> tell my web designers "You must write your HTML in sexps and must
>> do so perfectly or else no output at all will result."
>>
> 
> What else is your server going to do all day??  As 60,000 is less
> than 1/sec, your machine will be idle most of the day.  Its
> interesting that you ascribe feelings to your machine, as if
> it is being flogged to death.  Perhaps you have been around the
> piggishness of PHP and the template approach too long.

My server does many other things.  Why would I waste cycles, disk 
accesses, and time handling each hit when they can be streamlined? 
Given the choice between each hit involving 1ms of time and no disk 
accesses or involving 50 times as long and digging through the disk to 
load a Lisp image, why would you deliberately choose the latter?
From: Rainer Joswig
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <joswig-8B149E.18363802042006@news-europe.giganews.com>
In article <············@emma.aioe.org>,
 Ari Johnson <·········@gmail.com> wrote:

> Wade Humeniuk wrote:
> > Ari Johnson wrote:
> >>
> >> I have no desire not to use Common Lisp to write application logic.
> >> In fact, quite to the contrary, I am on the verge of reimplementing
> >> a massive PHP application that I wrote years ago, this time in Lisp.
> >> This application received 47,000 hits yesterday, a fairly typical
> >> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
> >> not going to put my poor server through 50,000+ Lisp environment
> >> initializations in a day if I can help it.  I am also not going to
> >> tell my web designers "You must write your HTML in sexps and must
> >> do so perfectly or else no output at all will result."
> >>
> > 
> > What else is your server going to do all day??  As 60,000 is less
> > than 1/sec, your machine will be idle most of the day.  Its
> > interesting that you ascribe feelings to your machine, as if
> > it is being flogged to death.  Perhaps you have been around the
> > piggishness of PHP and the template approach too long.
> 
> My server does many other things.  Why would I waste cycles, disk 
> accesses, and time handling each hit when they can be streamlined? 
> Given the choice between each hit involving 1ms of time and no disk 
> accesses or involving 50 times as long and digging through the disk to 
> load a Lisp image, why would you deliberately choose the latter?

If you start up a Lisp image more than once there is a good chance
that on reload you will hit some disk cache. Some Lisps
use also shared libraries. Anyway, look for Apache - Lisp
bridges which avoid this. See mod_lisp for example.

-- 
http://lispm.dyndns.org/
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0ours$luu$1@emma.aioe.org>
Rainer Joswig wrote:
> In article <············@emma.aioe.org>,
>  Ari Johnson <·········@gmail.com> wrote:
> 
>> Wade Humeniuk wrote:
>>> Ari Johnson wrote:
>>>> I have no desire not to use Common Lisp to write application logic.
>>>> In fact, quite to the contrary, I am on the verge of reimplementing
>>>> a massive PHP application that I wrote years ago, this time in Lisp.
>>>> This application received 47,000 hits yesterday, a fairly typical
>>>> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
>>>> not going to put my poor server through 50,000+ Lisp environment
>>>> initializations in a day if I can help it.  I am also not going to
>>>> tell my web designers "You must write your HTML in sexps and must
>>>> do so perfectly or else no output at all will result."
>>>>
>>> What else is your server going to do all day??  As 60,000 is less
>>> than 1/sec, your machine will be idle most of the day.  Its
>>> interesting that you ascribe feelings to your machine, as if
>>> it is being flogged to death.  Perhaps you have been around the
>>> piggishness of PHP and the template approach too long.
>> My server does many other things.  Why would I waste cycles, disk 
>> accesses, and time handling each hit when they can be streamlined? 
>> Given the choice between each hit involving 1ms of time and no disk 
>> accesses or involving 50 times as long and digging through the disk to 
>> load a Lisp image, why would you deliberately choose the latter?
> 
> If you start up a Lisp image more than once there is a good chance
> that on reload you will hit some disk cache. Some Lisps
> use also shared libraries. Anyway, look for Apache - Lisp
> bridges which avoid this. See mod_lisp for example.
> 

See my other replies regarding mod_lisp.  Maybe things have changed in 
the past year or so, though, on that front. :)

However, as to disk caches and shared libraries - those are nice 
OS-level speedups, but I do not like relying on the OS to do my 
optimization for me.  Relying on disk caching to make your software run 
quickly enough to be usable is not wise, IMHO.
From: Tim Bradshaw
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1143998480.401744.271320@t31g2000cwb.googlegroups.com>
Ari Johnson wrote:

>
> However, as to disk caches and shared libraries - those are nice
> OS-level speedups, but I do not like relying on the OS to do my
> optimization for me.  Relying on disk caching to make your software run
> quickly enough to be usable is not wise, IMHO.

Um.  Have you ever, in your life, run a machine with disk caching
turned off?  Do you have any idea how slow that would be?  How do you
feel about all these memory caches: should we turn them off too?  And
dynamically-scheduled processors reordering your code on the fly, you
must hate that.

--tim
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p2js$bqg$1@emma.aioe.org>
Tim Bradshaw wrote:
> Ari Johnson wrote:
> 
>> However, as to disk caches and shared libraries - those are nice
>> OS-level speedups, but I do not like relying on the OS to do my
>> optimization for me.  Relying on disk caching to make your software run
>> quickly enough to be usable is not wise, IMHO.
> 
> Um.  Have you ever, in your life, run a machine with disk caching
> turned off?  Do you have any idea how slow that would be?  How do you
> feel about all these memory caches: should we turn them off too?  And
> dynamically-scheduled processors reordering your code on the fly, you
> must hate that.

You missed the point.  I never said that these are bad things.  I only 
said that I do not like relying on the OS to do the optimizations that 
the programmer should be thinking about.  If I were alone in this 
thought, then on Unix systems inetd would be the only process that ever 
calls listen().
From: Tim Bradshaw
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1144171610.123550.160870@v46g2000cwv.googlegroups.com>
Ari Johnson wrote:

>
> You missed the point.  I never said that these are bad things.  I only
> said that I do not like relying on the OS to do the optimizations that
> the programmer should be thinking about.

The point is that disk-caching is one of the things the OS *should*
think about, and which it can make a better job of than almost all
applications (because it has a much better view of the state of the
memory of the machine as a whole, and of the state of the disk than
almost all applications, and because the people who write OSs are
generally much smarter than almost all programmers).  And by the same
token it's just fine for applications to assume that things get cached
and not jump through stupid and almost always counterproductive hoops
to do their own caching.

There are cases where this is not true - the canonical example being
large databases, which often want to manage their memory and disk I/O
almost completely themselves.  But these are rather unusual
applications (though commercially important of course).

--tim
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <4433f5cb$0$15783$14726298@news.sunsite.dk>
Tim Bradshaw wrote:
> Ari Johnson wrote:
> 
>> You missed the point.  I never said that these are bad things.  I only
>> said that I do not like relying on the OS to do the optimizations that
>> the programmer should be thinking about.
> 
> The point is that disk-caching is one of the things the OS *should*
> think about, and which it can make a better job of than almost all
> applications (because it has a much better view of the state of the
> memory of the machine as a whole, and of the state of the disk than
> almost all applications, and because the people who write OSs are
> generally much smarter than almost all programmers).  And by the same
> token it's just fine for applications to assume that things get cached
> and not jump through stupid and almost always counterproductive hoops
> to do their own caching.

It's fine to assume that they get cached, but it's not fine to say "I 
don't have to optimize my code because the disk cache will take care of 
it."  Just like it's fine to assume that Scheme code will have tail 
calls eliminated, but not fine to blindly assume that all recursion will 
be optimized away.  Assuming a smart compiler and OS is only smart if 
you know exactly what it does and does not do for you.

No matter what a disk cache gets you in load time, there are still 
things that the OS must do each time you start a process and each time a 
process maps a large file into memory.
From: Tim Bradshaw
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1144264603.393565.259070@t31g2000cwb.googlegroups.com>
Ari Johnson wrote:

>
> It's fine to assume that they get cached, but it's not fine to say "I
> don't have to optimize my code because the disk cache will take care of
> it."  Just like it's fine to assume that Scheme code will have tail
> calls eliminated, but not fine to blindly assume that all recursion will
> be optimized away.  Assuming a smart compiler and OS is only smart if
> you know exactly what it does and does not do for you.

Well, yes.  So rather than saying `I do not like relying on the OS to
do my optimization for me' based on some blind prejudice, what you do
is to actually measure how fast it really is and what its
characteristics are.
>
> No matter what a disk cache gets you in load time, there are still
> things that the OS must do each time you start a process and each time a
> process maps a large file into memory.

Yes, there are, but because the people who write OS's are (often) quite
smart, they can be really surprisingly fast.  I suggest you measure
them, you might be surprised.

--tim
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <44346c2e$0$15794$14726298@news.sunsite.dk>
Tim Bradshaw wrote:
> Yes, there are, but because the people who write OS's are (often) quite
> smart, they can be really surprisingly fast.  I suggest you measure
> them, you might be surprised.

Your contention seems to remain that it is almost as fast to load a Lisp 
and run code than it is to run the code in an existing Lisp.

/--------
$ time sbcl --load test.fasl --eval '(test)' --eval '(quit)'
This is SBCL 0.8.16, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
This is a test.

real    0m0.069s
user    0m0.051s
sys     0m0.017s
---------
$ sbcl
This is SBCL 0.8.16, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (load "test.fasl")

T
* (time (test))
This is a test.
Evaluation took:
                  0.0 seconds of real time
                  0.0 seconds of user run time
                  0.0 seconds of system run time
                  0 page faults and
                  0 bytes consed.
\--------

I got a 0.001 runtime on a previous run without the load time, but never 
more than that.

I never got less than 0.069 when including the load time.  Some runs 
were as slow as 0.365, and the average was about 0.080.

These numbers may all seem small, but they are disparate enough to 
justify investigating alternatives to loading and initializing a Lisp on 
each run.

(For the record, the fastest to run "sbcl --eval '(quit)'" was 0.066.)

OS programmers may be smart, but they are not magical.
From: Rob Warnock
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <Af-dnQ_AsY687anZnZ2dnUVZ_tGdnZ2d@speakeasy.net>
Ari Johnson  <·········@gmail.com> wrote:
+---------------
| (For the record, the fastest to run "sbcl --eval '(quit)'" was 0.066.)
+---------------

Don't forget to include -- or rather, *exclude*!! -- any default
user (or other) init files you might be including. E.g. note the
difference in these two times (and why!!):

    % time cmucl -quiet -eval '(quit)'
    ; Loading #p"/u/rpw3/.cmucl-init".
    ;; Loading #p"/u/lisp/contrib/cclan/asdf/asdf.x86f".
    ;; Loading #p"/u/lisp/contrib/lib/pg.x86f".
    0.162u 0.030s 0:00.20 95.0%     114+2277k 0+0io 0pf+0w   # 0.192 total
    % time cmucl -quiet -eval '(quit)' -noinit
    0.014u 0.000s 0:00.01 100.0%    168+3436k 0+0io 0pf+0w
    % 

When I'm doing "scripting" with CMUCL, I'm very careful to avoid
including my init file, so simple scripts like this one run fast:

    % time keto 8 4 14
    grams: protein 8  carb 4  fat 14
    ketogenic ratio: 1.674897
    total calories: 174
    0.022u 0.000s 0:00.02 100.0%    126+2576k 0+0io 0pf+0w
    % 


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <44349e6f$0$15795$14726298@news.sunsite.dk>
Rob Warnock wrote:
> Ari Johnson  <·········@gmail.com> wrote:
> +---------------
> | (For the record, the fastest to run "sbcl --eval '(quit)'" was 0.066.)
> +---------------
> 
> Don't forget to include -- or rather, *exclude*!! -- any default
> user (or other) init files you might be including. E.g. note the
> difference in these two times (and why!!):

Fair enough, but:

$ ls .sbclrc /etc/sbcl*rc
ls: .sbclrc: No such file or directory
ls: /etc/sbcl*rc: No such file or directory

Already covered that. :)
From: Wade Humeniuk
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <opTXf.28834$%H.24670@clgrps13>
Ari Johnson wrote:

> 
> See my other replies regarding mod_lisp.  Maybe things have changed in 
> the past year or so, though, on that front. :)
> 
> However, as to disk caches and shared libraries - those are nice 
> OS-level speedups, but I do not like relying on the OS to do my 
> optimization for me.  Relying on disk caching to make your software run 
> quickly enough to be usable is not wise, IMHO.

(Are all you see is problems??)  On this note, just set up a RAM Disk
and be done with it.  Just code the sucker instead of trying to solve
what you see as everyone's problems.  You do not what to tell the
HTML designers to code up the Web pages in sexps, but who told you
have to do that???  You want a solution?  Here is one, get Allegro
Common Lisp, use Aserve (they have SOAP support for your IPC
requests) and satisfy your web designers by using PUBLISH-MULTI
and Webactions to create a boundary between yourself and the "Web Designers".

http://www.franz.com/support/documentation/8.0/doc/aserve/aserve.html#f-publish-multi
http://www.franz.com/support/documentation/8.0/doc/webactions.html

Wade
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p39t$fnt$1@emma.aioe.org>
Wade Humeniuk wrote:
> (Are all you see is problems??)  On this note, just set up a RAM Disk
> and be done with it.  Just code the sucker instead of trying to solve
> what you see as everyone's problems.  You do not what to tell the
> HTML designers to code up the Web pages in sexps, but who told you
> have to do that???  You want a solution?  Here is one, get Allegro
> Common Lisp, use Aserve (they have SOAP support for your IPC
> requests) and satisfy your web designers by using PUBLISH-MULTI
> and Webactions to create a boundary between yourself and the "Web 
> Designers".
> 
> http://www.franz.com/support/documentation/8.0/doc/aserve/aserve.html#f-publish-multi 
> 
> http://www.franz.com/support/documentation/8.0/doc/webactions.html

Yep! :P  I'm not trying to solve everyone's problems.  I'm trying to 
solve my own problem, with an eye toward similar problems that others 
either have already solved or share in common with me.  What's wrong 
with that?

And here is your expected response:  publish-multi is awful, because it 
requires me to write HTML at some point in Lisp, and that requires me to 
translate from web-designer HTML/CSS into Lisp-HTML.  I am not wanting 
to split things into header, footer, and generated portions.  I want to 
have a template for the page and fill in data that comes from a Lisp 
function (which itself has seen some input from a web form, cookies, 
etc.)  I want my web designers to write *all* of the HTML.

Webactions is nice, except that it requires me to run reverse proxying, 
which means that I have to put all the Lisp things on a particular web 
path and have to make any changes to that in my Apache configuration; 
and that it tries to look like HTML, which makes it harder to read a 
template.  Are there merits to this versus the PHP method of indicating 
where there will be PHP code in the file?
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87bqvjluow.fsf@thalassa.informatimago.com>
Ari Johnson <·········@gmail.com> writes:
> And here is your expected response:  publish-multi is awful, because
> it requires me to write HTML at some point in Lisp, and that requires
> me to translate from web-designer HTML/CSS into Lisp-HTML.  I am not
> wanting to split things into header, footer, and generated portions.
> I want to have a template for the page and fill in data that comes
> from a Lisp function (which itself has seen some input from a web
> form, cookies, etc.)  I want my web designers to write *all* of the
> HTML.

If you don't like what UCW proposes, you can implement a simplistic
HTML template in a few lines of Lisp.  It's rather trivial.

Have the HTML coders insert LISP tags like:
<LISP function="GENERATE-DOCUMENT-LIST" view="short" docperpage="25">
then you can just scan the HTML for such tags, build a form from it:
   (generate-document-list :view 'short :docperpage 25)
and evaluate it, collecting the output to substitute the tag.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
The mighty hunter
Returns with gifts of plump birds,
Your foot just squashed one.
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p425$jqn$1@emma.aioe.org>
Pascal Bourguignon wrote:
> Ari Johnson <·········@gmail.com> writes:
>> And here is your expected response:  publish-multi is awful, because
>> it requires me to write HTML at some point in Lisp, and that requires
>> me to translate from web-designer HTML/CSS into Lisp-HTML.  I am not
>> wanting to split things into header, footer, and generated portions.
>> I want to have a template for the page and fill in data that comes
>> from a Lisp function (which itself has seen some input from a web
>> form, cookies, etc.)  I want my web designers to write *all* of the
>> HTML.
> 
> If you don't like what UCW proposes, you can implement a simplistic
> HTML template in a few lines of Lisp.  It's rather trivial.
> 
> Have the HTML coders insert LISP tags like:
> <LISP function="GENERATE-DOCUMENT-LIST" view="short" docperpage="25">
> then you can just scan the HTML for such tags, build a form from it:
>    (generate-document-list :view 'short :docperpage 25)
> and evaluate it, collecting the output to substitute the tag.

That still requires me to do two things, neither of which I want to do 
again if I can help it (having done both more times than I like, and 
having always spent more time on them than on writing the actual 
application, and having furthermore found later on that doing so limited 
the capabilities of the application itself):

1. Generate HTML in my code rather than in the template.
2. Write my own template processor.

My most recent response to Mr. Bradshaw indicates my current turn in 
thinking about how to avoid these pitfalls. :)
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p52k$pk6$1@emma.aioe.org>
Stefan Ram wrote:
> Ari Johnson <·········@gmail.com> writes:
>> the capabilities of the application itself):
>> 1. Generate HTML in my code rather than in the template.
> 
>   If you want to output any information, it has to be emitted in
>   some language, be it HTML or some other notation. 
> 
>   So, you do not want to be it in HTML:
> 
>   Which other language do you prefer to HTML, and what keeps you
>   from actually doing so with Lisp?

I prefer that the data be exposed to the template in a way that the 
template can turn into HTML.  I see no reason that this should be any 
more complicated than:

<?php foreach($rows as $row): ?>
HTML here with <?= $row['field'] ?>
<?php endforeach; ?>

As I said from the start: PHP is a very good HTML template language and 
a not-so-good application programming language.  Why force a decision 
between a language inferior for one half of an application and a 
language inferior for the other half, if it is possible to use the best 
language available for each component?  That was the driving idea of my 
original analogy to SQL: you use SQL plus another language because SQL 
is good for data storage and the other language is good for application 
logic and user interface.  Why shouldn't it be possible to use Lisp for 
the things it's perfect at and something else for the things it is not?
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p6ee$1kh$1@emma.aioe.org>
Stefan Ram wrote:
> Ari Johnson <·········@gmail.com> writes:
>> I prefer that the data be exposed to the template in a way that the 
>> template can turn into HTML. I see no reason that this should be any 
>> more complicated than:
>> <?php foreach($rows as $row): ?>
>> HTML here with <?= $row['field'] ?>
>> <?php endforeach; ?>
> 
>   OK. Lisp could output this as well.

What could Lisp output?  PHP code?  That's patently ridiculous, so I'm 
assuming you don't mean that.  If you mean that I could write:
<?lisp (loop for row in rows do ?>
HTML here with <?= (field row) ?>
<?lisp ) ?>

...then please tell me which existing template system allows that, and 
where its thorough and novice-friendly documentation is, preferably in 
ISBN form, so I can tell my web designers just what they need to read in 
order to write their templates.


If you mean something else entirely, please let me know.  Thanks.
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p7qc$9nd$1@emma.aioe.org>
Stefan Ram wrote:
> Ari Johnson <·········@gmail.com> writes:
>> What could Lisp output? PHP code?
> 
>   Yes!

Why in the world would you ever write a web application in Lisp that 
dynamically generates PHP code, when the entire purpose is to avoid 
having your web designers have to write Lisp?  I think you misunderstand 
the problem.

>> <?lisp (loop for row in rows do ?>
>> HTML here with <?= (field row) ?>
>> <?lisp ) ?>
> 
>   You already wrote that you don't want Lisp to output HTML.

No, I wrote that I don't want to *write* HTML *in* Lisp.  There's a 
difference that really isn't all that difficult to understand:

(apply-template "template-filename" some-data) = output HTML

(:html
   (:body
     (:h1 "Data to follow")
     (loop <over data>
       (:p <some of the data>)))) = writing HTML in Lisp
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p9bo$j49$1@emma.aioe.org>
Stefan Ram wrote:
>   If HTML is written, then your designers will not be able to
>   change this, and they only can modify the design via CSS.
> 
>   If a PHP is written, then a designer with knowledge of PHP but
>   no knowledge of Lisp could still write the PHP-templates so as
>   to specify the HTML generated himself.
> 
>   Technically, both approaches are possible, so you just need to
>   make your mind up about which approach you want.

Your use of the passive voice makes it somewhat difficult to divine your 
precise meaning.  "Is written" by ... web designers, a Lisp function, or 
what?

I do think that this makes it a full circle.  I originally proposed a 
means of writing the user interface in PHP and making one call into Lisp 
to handle the application logic.

--
<?php $data_to_display = lisp_application_logic_call($form_variables, 
$cookies, ...); ?>
HTML here with appropriate PHP to access <?= $data_to_display ?>
--

But you're saying that the Lisp should generate PHP (or maybe you do 
simply mean "output" PHP, in which case we are still left with the 
problem of a template language that lets the designers write the 
template and allows me to pass it data from Lisp; either one is not what 
the above exemplifies).

I made my mind up about which approach I want a long time ago.  I may 
have failed to clearly state my mind on the topic, and for that I must 
apologize.

I want to write a user interface in PHP and let Lisp do the backend 
processing.  I do not want to run reverse proxying, and I do not want to 
have to edit my Apache configuration to add, remove, or modify which 
URIs involve a Lisp and which do not.  I also do not want to require my 
web designers to write anything other than HTML with occasional 
template-language constructs to access data where appropriate.

If any of these desires are unclear, please ask me to clarify them.  If 
all of them are clear, then one of two possibilities must hold: (1) you 
can point me to an existing solution; or (2) you cannot.  Please try to 
avoid using all-caps to tell me to check out a partial solution that 
cannot readily be turned into a complete one - partial solutions are 
easy to find.  Complete solutions are not.  Furthermore, if no complete 
solution exists, wouldn't it be nice to discuss how best to approach a 
complete solution instead of repeatedly pointing me to the same partial 
solutions.  I'll be glad to clarify myself if I've been unclear - just 
ask.  Thanks. :)
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0pbj8$teb$1@emma.aioe.org>
Stefan Ram wrote:
> Ari Johnson <·········@gmail.com> writes:
>>> If HTML is written, then your designers will not be able to
>>> change this, and they only can modify the design via CSS.
>>> If a PHP is written, then a designer with knowledge of PHP but
>>> no knowledge of Lisp could still write the PHP-templates so as
>>> to specify the HTML generated himself.
>>> Technically, both approaches are possible, so you just need to
>>> make your mind up about which approach you want.
>> Your use of the passive voice makes it somewhat difficult to
>> divine your precise meaning.  "Is written" by ... web
>> designers, a Lisp function, or what?
> 
>   By a Lisp function.
> 
>> I do think that this makes it a full circle.  I originally proposed a 
>> means of writing the user interface in PHP and making one call into Lisp 
>> to handle the application logic.
>> <?php $data_to_display = lisp_application_logic_call($form_variables,$cookies, ...); ?>
> 
>   PHP could access the application logic via SQL with the
>   additional trick that the SQL server acts as the facade
>   to an arbitrary program (e.g., Lisp), so to get the
>   factorial of 5:
> 
> SELECT RESULT FROM FACTORIAL WHERE ARGUMENT=5;
> 
>   or
> 
> SELECT DATA_TO_DISPLAY FROM LISP_APPLICATION_LOGIC_CALL
> WHERE FORM_VARIABLES=... AND COOKIES=...;
> 
>   Since PHP already can talk to an SQL server (I guess)
>   it will be able to talk to a Lisp server using 
>   SQL as its query language.
> 
>   I do not like to expose the implementation detail that some
>   application logic is written in Lisp to the client (here:
>   PHP).

The problem with this is that you then have a Lisp out there polling the 
SQL server, unless I've misunderstood you again (a good possibility).

>> But you're saying that the Lisp should generate PHP (or maybe you do 
>> simply mean "output" PHP, in which case we are still left with the 
>> problem of a template language that lets the designers write the 
>> template and allows me to pass it data from Lisp; either one is not what 
>> the above exemplifies).
> 
>   I have assumed that the designers know PHP and that PHP could
>   be used as a template language. But then, I do not know PHP well.

The assumption is that the designers will be able to figure out enough 
PHP to use it as a template language.

>> I also do not want to require my web designers to write
>> anything other than HTML with occasional template-language
>> constructs to access data where appropriate.
> 
>   So you assume that the web designers knows HTML and a
>   �template-language� to access data.

Yep, or can at least learn a template language.

>   So the web designer can write a template. A lisp process then
>   can read this template, interpret the template-language
>   constructs and eventually release an HTML document.

Yep.  My response to Mr. Baringer after he gave some examples in TAL, 
html-template, and cl-emb is along those lines.  For this, so far I 
think I'd have to accept a compromise on one of my previously-stated 
goals and allow Apache to decide whether something is going to use Lisp 
or not rather than having PHP or a filetype on disk make that decision. 
  (PHP would make the decision by calling into Lisp when necessary; the 
filetype would make the decision by having a .lsp or some such extension 
that Apache can recognize).

>   Sorry, I am used to write SQL in all-caps, but I managed to
>   avoid using any swear words (except �PHP�)!
> 

I also write SQL in all-caps.  And all SQL keywords are swear words, too! :P
From: Bill Atkins
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <873bgva3mm.fsf@rpi.edu>
···@zedat.fu-berlin.de (Stefan Ram) writes:

> Ari Johnson <·········@gmail.com> writes:
>>Stefan Ram wrote:
>>>SELECT RESULT FROM FACTORIAL WHERE ARGUMENT=5;
>>The problem with this is that you then have a Lisp out there
>>polling the SQL server, unless I've misunderstood you again (a
>>good possibility).
>
>   A process reads SQL queries, like »SELECT RESULT FROM
>   FACTORIAL WHERE ARGUMENT=5;« and maps them to Lisp calls, like
>   »(FACTORIAL 5)«.  It then returns the value of this expression
>   as a single-valued table.
>
>   There is no need for polling: A process controlled by
>   functions written in Lisp acts as the SQL server.
>
>   The client believes he is querying tables, while in fact he
>   is invoking lisp functions.
>
>   As another example, »( f( g 2 3 ))« is translated to:
>
> SELECT RESULT FROM f WHERE ARGUMENT =
> ( SELECT RESULT FROM g WHERE ARGUMENT=2 AND ARGUMENT1=3 );

Hideous!

>   Thus, a user or PHP-environment knowing how to query an SQL
>   server can invoke Lisp functions with no need to be aware of it.
From: Tim X
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87wte5ek7p.fsf@tiger.rapttech.com.au>
Bill Atkins <············@rpi.edu> writes:

> ···@zedat.fu-berlin.de (Stefan Ram) writes:
>
>> Ari Johnson <·········@gmail.com> writes:
>>>Stefan Ram wrote:
>>>>SELECT RESULT FROM FACTORIAL WHERE ARGUMENT=5;
>>>The problem with this is that you then have a Lisp out there
>>>polling the SQL server, unless I've misunderstood you again (a
>>>good possibility).
>>
>>   A process reads SQL queries, like »SELECT RESULT FROM
>>   FACTORIAL WHERE ARGUMENT=5;« and maps them to Lisp calls, like
>>   »(FACTORIAL 5)«.  It then returns the value of this expression
>>   as a single-valued table.
>>
>>   There is no need for polling: A process controlled by
>>   functions written in Lisp acts as the SQL server.
>>
>>   The client believes he is querying tables, while in fact he
>>   is invoking lisp functions.
>>
>>   As another example, »( f( g 2 3 ))« is translated to:
>>
>> SELECT RESULT FROM f WHERE ARGUMENT =
>> ( SELECT RESULT FROM g WHERE ARGUMENT=2 AND ARGUMENT1=3 );
>
> Hideous!
>
>>   Thus, a user or PHP-environment knowing how to query an SQL
>>   server can invoke Lisp functions with no need to be aware of it.

I'm afraid I'd have to agree this is hideous and I cannot see what it
would give you as you would essentially be limiting what you could do
in lisp to what you could express in SQL i.e. the question is limited
to SQL expressions and while you could process data in any way you
like to get the answer and return it as a psudo SQL table, the type of
question you can ask is still restricted to the questions you can
ask/express in SQL - in which case, what benefit would you get you
cannot already get with a decent database which supports stored
procedures?

Tim

-- 
tcross (at) rapttech dot com dot au
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <871wwdgz9u.fsf@thalassa.informatimago.com>
Tim X <····@nospam.dev.null> writes:

> Bill Atkins <············@rpi.edu> writes:
>
>> ···@zedat.fu-berlin.de (Stefan Ram) writes:
>>
>>> Ari Johnson <·········@gmail.com> writes:
>>>>Stefan Ram wrote:
>>>>>SELECT RESULT FROM FACTORIAL WHERE ARGUMENT=5;
>>>>The problem with this is that you then have a Lisp out there
>>>>polling the SQL server, unless I've misunderstood you again (a
>>>>good possibility).
>>>
>>>   A process reads SQL queries, like �SELECT RESULT FROM
>>>   FACTORIAL WHERE ARGUMENT=5;� and maps them to Lisp calls, like
>>>   �(FACTORIAL 5)�.  It then returns the value of this expression
>>>   as a single-valued table.
>>>
>>>   There is no need for polling: A process controlled by
>>>   functions written in Lisp acts as the SQL server.
>>>
>>>   The client believes he is querying tables, while in fact he
>>>   is invoking lisp functions.
>>>
>>>   As another example, �( f( g 2 3 ))� is translated to:
>>>
>>> SELECT RESULT FROM f WHERE ARGUMENT =
>>> ( SELECT RESULT FROM g WHERE ARGUMENT=2 AND ARGUMENT1=3 );
>>
>> Hideous!
>>
>>>   Thus, a user or PHP-environment knowing how to query an SQL
>>>   server can invoke Lisp functions with no need to be aware of it.
>
> I'm afraid I'd have to agree this is hideous and I cannot see what it
> would give you as you would essentially be limiting what you could do
> in lisp to what you could express in SQL i.e. the question is limited
> to SQL expressions and while you could process data in any way you
> like to get the answer and return it as a psudo SQL table, the type of
> question you can ask is still restricted to the questions you can
> ask/express in SQL - in which case, what benefit would you get you
> cannot already get with a decent database which supports stored
> procedures?

The benefits are obvious howerver.  Compare:

$server=mysql_select_db($base,mysql_connect($host,$login,$pwd));
$result=mysql_query("SELECT RESULT FROM f 
                      WHERE ARGUMENT = ( SELECT RESULT FROM g 
                                          WHERE ARGUMENT=2 
                                            AND ARGUMENT1=3 )",$server);
$row=mysql_fetch_assoc($result);
echo $row['RESULT'];

vs.:

$server=lisp_connect($host,$login,$pwd));
$result=lisp_query("SELECT RESULT FROM f 
                      WHERE ARGUMENT = ( SELECT RESULT FROM g 
                                          WHERE ARGUMENT=2 
                                            AND ARGUMENT1=3 )",$server);
$row=lisp_fetch_assoc($result);
echo $row['RESULT'];


See? It's exactly the same, there's nothing more to learn.  Just
s/mysql/lisp/ to use the lisp server instead of the mysql server.  You
could even call it LISQL and hide the fact that you've implemented
that server with a slow interpreted languages from beyond grave.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

This is a signature virus.  Add me to your signature and help me to live.
From: Tim X
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87mzf0e6no.fsf@tiger.rapttech.com.au>
Pascal Bourguignon <······@informatimago.com> writes:

> Tim X <····@nospam.dev.null> writes:
>
>> Bill Atkins <············@rpi.edu> writes:
>>
>>> ···@zedat.fu-berlin.de (Stefan Ram) writes:
>>>
>>>> Ari Johnson <·········@gmail.com> writes:
>>>>>Stefan Ram wrote:
>>>>>>SELECT RESULT FROM FACTORIAL WHERE ARGUMENT=5;
>>>>>The problem with this is that you then have a Lisp out there
>>>>>polling the SQL server, unless I've misunderstood you again (a
>>>>>good possibility).
>>>>
>>>>   A process reads SQL queries, like »SELECT RESULT FROM
>>>>   FACTORIAL WHERE ARGUMENT=5;« and maps them to Lisp calls, like
>>>>   »(FACTORIAL 5)«.  It then returns the value of this expression
>>>>   as a single-valued table.
>>>>
>>>>   There is no need for polling: A process controlled by
>>>>   functions written in Lisp acts as the SQL server.
>>>>
>>>>   The client believes he is querying tables, while in fact he
>>>>   is invoking lisp functions.
>>>>
>>>>   As another example, »( f( g 2 3 ))« is translated to:
>>>>
>>>> SELECT RESULT FROM f WHERE ARGUMENT =
>>>> ( SELECT RESULT FROM g WHERE ARGUMENT=2 AND ARGUMENT1=3 );
>>>
>>> Hideous!
>>>
>>>>   Thus, a user or PHP-environment knowing how to query an SQL
>>>>   server can invoke Lisp functions with no need to be aware of it.
>>
>> I'm afraid I'd have to agree this is hideous and I cannot see what it
>> would give you as you would essentially be limiting what you could do
>> in lisp to what you could express in SQL i.e. the question is limited
>> to SQL expressions and while you could process data in any way you
>> like to get the answer and return it as a psudo SQL table, the type of
>> question you can ask is still restricted to the questions you can
>> ask/express in SQL - in which case, what benefit would you get you
>> cannot already get with a decent database which supports stored
>> procedures?
>
> The benefits are obvious howerver.  Compare:
>
> $server=mysql_select_db($base,mysql_connect($host,$login,$pwd));
> $result=mysql_query("SELECT RESULT FROM f 
>                       WHERE ARGUMENT = ( SELECT RESULT FROM g 
>                                           WHERE ARGUMENT=2 
>                                             AND ARGUMENT1=3 )",$server);
> $row=mysql_fetch_assoc($result);
> echo $row['RESULT'];
>
> vs.:
>
> $server=lisp_connect($host,$login,$pwd));
> $result=lisp_query("SELECT RESULT FROM f 
>                       WHERE ARGUMENT = ( SELECT RESULT FROM g 
>                                           WHERE ARGUMENT=2 
>                                             AND ARGUMENT1=3 )",$server);
> $row=lisp_fetch_assoc($result);
> echo $row['RESULT'];
>
>
> See? It's exactly the same, there's nothing more to learn.  Just
> s/mysql/lisp/ to use the lisp server instead of the mysql server.  You
> could even call it LISQL and hide the fact that you've implemented
> that server with a slow interpreted languages from beyond grave.
>

thanks for proving my point.

Tim

-- 
tcross (at) rapttech dot com dot au
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87vetpfkem.fsf@thalassa.informatimago.com>
Tim X <····@nospam.dev.null> writes:
> I'm afraid I'd have to agree this is hideous and I cannot see what it
> would give you as you would essentially be limiting what you could do
> in lisp to what you could express in SQL i.e. the question is limited
> to SQL expressions and while you could process data in any way you
> like to get the answer and return it as a psudo SQL table, the type of
> question you can ask is still restricted to the questions you can
> ask/express in SQL - in which case, what benefit would you get you
> cannot already get with a decent database which supports stored
> procedures?

The benefits are obvious howerver.  Compare:

$server=mysql_select_db($base,mysql_connect($host,$login,$pwd));
$result=mysql_query("SELECT RESULT FROM f 
                      WHERE ARGUMENT = ( SELECT RESULT FROM g 
                                          WHERE ARGUMENT=2 
                                            AND ARGUMENT1=3 )",$server);
$row=mysql_fetch_assoc($result);
echo $row['RESULT'];

vs.

$server=lisql_connect($host,$login,$pwd));
$result=lisql_query("SELECT RESULT FROM f 
                      WHERE ARGUMENT = ( SELECT RESULT FROM g 
                                          WHERE ARGUMENT=2 
                                            AND ARGUMENT1=3 )",$server);
$row=lisql_fetch_assoc($result);
echo $row['RESULT'];

vs.

$server=lisp_connect($host,$login,$pwd));
$result=lisp_query("(f (g 2 3))",$server);
echo lisp_value($result,0);

See?  

With lisql_*, it's exactly the same as with mysql_*, there's nothing
more to learn.  You don't even know from the name  LISQL that you've
implemented that server with a slow interpreted languages from beyond
grave.

On the other hand, with lisp_*, you have to learn a whole new set of
syntax and concepts, you have to type all these useless parentheses
inside the double-quotes, you run a slow server with a prehistoric
query language, you reduce your LOC performance, etc.  I can see only
downsides with this solution.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
From: Tim X
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87irpoe69q.fsf@tiger.rapttech.com.au>
···@zedat.fu-berlin.de (Stefan Ram) writes:

> Tim X <····@nospam.dev.null> writes:
>>>>SELECT RESULT FROM f WHERE ARGUMENT =
>>>>( SELECT RESULT FROM g WHERE ARGUMENT=2 AND ARGUMENT1=3 );
>>I'm afraid I'd have to agree this is hideous and I cannot see
>>what it would give you as you would essentially be limiting
>>what you could do in lisp to what you could express in SQL 
>
>   It was intended for cases like when
>
>     - the designers already know HTML, PHP and SQL but refuse
>       to learn/use any other language, so you now can sell them 
>       any Lisp function as an SQL table, or when
>
>     - other local authorities forbid the use of other languages
>       than HTML, PHP and SQL.
>
>   I wanted to suggest how parts of Lisp could be »tunneled«
>   through SQL.
>
>   In the other direction, one could replace SQL by an
>   S-expression based language. But this surely has been done
>   before.
>
>

I get what the idea is, but feel the constraint of trying to still
work within the expressive limitations of SQL is overly limiting. As I
suggested in another post, you may as well just provide a RPC style
call using PHP sockets or whatever - avoid the SQL ugliness as it
doesn't actually buy you anything and even the most limited user
should be able to handle an RPC call just as easily as handling an SQL
statement. There isn't any need for the user to even know SQL and from
the development side, you don't need to implement an interpreter for
the SQL etc. 

Tim

-- 
tcross (at) rapttech dot com dot au
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87odzjk8fl.fsf@thalassa.informatimago.com>
Ari Johnson <·········@gmail.com> writes:
> [...]
> I made my mind up about which approach I want a long time ago.  I may
> have failed to clearly state my mind on the topic, and for that I must
> apologize.
>
> I want to write a user interface in PHP and let Lisp do the backend
> processing.  I do not want to run reverse proxying, and I do not want
> to have to edit my Apache configuration to add, remove, or modify
> which URIs involve a Lisp and which do not.  I also do not want to
> require my web designers to write anything other than HTML with
> occasional template-language constructs to access data where
> appropriate.

If that's all you want, I don't see where's the difficulty.

Just use PHP sockets http://es.php.net/sockets to connect to the lisp
process, send your application logic requests and read back the data.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"You cannot really appreciate Dilbert unless you read it in the
original Klingon"
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0pduu$977$1@emma.aioe.org>
Pascal Bourguignon wrote:
> Just use PHP sockets http://es.php.net/sockets to connect to the lisp
> process, send your application logic requests and read back the data.
> 

I posted originally with the hope that someone else had done something 
similar and that a discussion would ensue in which various ideas for the 
best way to do it would come to the surface.  So far, that's been a 
late-starting hope. :)
From: Tim X
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <871wwdfz9i.fsf@tiger.rapttech.com.au>
Ari Johnson <·········@gmail.com> writes:

> Stefan Ram wrote:
>>   If HTML is written, then your designers will not be able to
>>   change this, and they only can modify the design via CSS.
>>   If a PHP is written, then a designer with knowledge of PHP but
>>   no knowledge of Lisp could still write the PHP-templates so as
>>   to specify the HTML generated himself.
>>   Technically, both approaches are possible, so you just need to
>>   make your mind up about which approach you want.
>
> Your use of the passive voice makes it somewhat difficult to divine
> your precise meaning.  "Is written" by ... web designers, a Lisp
> function, or what?
>
> I do think that this makes it a full circle.  I originally proposed a
> means of writing the user interface in PHP and making one call into
> Lisp to handle the application logic.
>
> --
> <?php $data_to_display = lisp_application_logic_call($form_variables,
> $cookies, ...); ?>
> HTML here with appropriate PHP to access <?= $data_to_display ?>
> --
>
> But you're saying that the Lisp should generate PHP (or maybe you do
> simply mean "output" PHP, in which case we are still left with the
> problem of a template language that lets the designers write the
> template and allows me to pass it data from Lisp; either one is not
> what the above exemplifies).
>
> I made my mind up about which approach I want a long time ago.  I may
> have failed to clearly state my mind on the topic, and for that I must
> apologize.
>
> I want to write a user interface in PHP and let Lisp do the backend
> processing.  I do not want to run reverse proxying, and I do not want
> to have to edit my Apache configuration to add, remove, or modify
> which URIs involve a Lisp and which do not.  I also do not want to
> require my web designers to write anything other than HTML with
> occasional template-language constructs to access data where
> appropriate.
>
> If any of these desires are unclear, please ask me to clarify them.
> If all of them are clear, then one of two possibilities must hold: (1)
> you can point me to an existing solution; or (2) you cannot.  Please
> try to avoid using all-caps to tell me to check out a partial solution
> that cannot readily be turned into a complete one - partial solutions
> are easy to find.  Complete solutions are not.  Furthermore, if no
> complete solution exists, wouldn't it be nice to discuss how best to
> approach a complete solution instead of repeatedly pointing me to the
> same partial solutions.  I'll be glad to clarify myself if I've been
> unclear - just ask.  Thanks. :)

I think you may be approaching this from the wrong direction. The real
question seems to be "How can I get PHP to call a lisp function and
have the results returned in a PHP data structure so that it can be
accessed by PHP in later processing". This is going by your example
above. Following on from your example and statements, Lisp doesn't
need to know anything about PHP - it won't care who/what is calling
it. However, PHP will need to understand how the
list/hash/vector/sequence or whatever passed back from Lisp can be put
into a data structure PHP understands and can then access in a loop or
whatever later in the processing. What you seem to be after is a type
of remote procedure call (RPC). 

From the lisp side, this is nearly trivial. However, I don't know how
easily you can map the values returned from the call to lisp into
structures PHP can handle (because I don't know enough about PHP
rather than it being difficult). 

What I believe some of the other posters were trying to indicate is
that it would be possible to create a lisp interpreter for PHP and add
to it the ability to insert lisp function calls. You users wouldn't
even need to know lisp is being used - they continue to write HTML and
PHP, but instead of a native PHP interpreter, you just interpret it
via a lisp PHP interpreter and send the output directly back to the
server which sends it to the browser. Alternatively, you could ignore
the php and just have a lisp pre-processor which replaces calls to
lisp with output that is legal and interpretable PHP - but getting
this to work reliably and without too much overhead would be
difficult, especially for dynamic pages (I've noticed more and more
use of PHP for what could have been done quite adequately as static
pages). 

However, I suspect, for compatibility and maintenance reasons, the
sensible approach would be to generalise the probelm and work out a
robust and flexible mechanism within PHP which would allow PHP to call
a remote procedure and have its returned values mapped to a PHP data 
structure.

Of course, PHP may already have this facility, in which case, your
problem is already solved and nothing needs to be done. 

Tim

-- 
tcross (at) rapttech dot com dot au
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <4433f48b$0$15782$14726298@news.sunsite.dk>
Tim X wrote:
> However, I suspect, for compatibility and maintenance reasons, the
> sensible approach would be to generalise the probelm and work out a
> robust and flexible mechanism within PHP which would allow PHP to call
> a remote procedure and have its returned values mapped to a PHP data 
> structure.
> 
> Of course, PHP may already have this facility, in which case, your
> problem is already solved and nothing needs to be done. 


Right.  That's what I was after - an inter-process communication 
protocol that allows PHP or any other language (PHP was just the example 
used since it is popular in web applications and a perfect example of a 
language that is excellent for writing HTML templates but poor for 
writing complicated application logic) to call into Lisp.

The problem is merely one of communication.  What is the best way for 
Random-Language-A to talk to Common Lisp?  XML-RPC is high-latency. 
CORBA seems promising, but massive (compare the size of CLORB with the 
size of the Lisp side of the mod_lisp protocol, for instance).  The 
Lisp-call-as-SQL idea was novel, but unwieldy.

The major advantage of CORBA or XML-RPC is that they are both already 
there for many languages.  I still need to experiment a bit with CORBA 
to see how easy it is to set up - it seems to have a high labor cost in 
going from a clean installation to objects talking to each other.
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m3u0971zit.fsf@rigel.goldenthreadtech.com>
Ari Johnson <·········@gmail.com> writes:

> high-latency. CORBA seems promising, but massive (compare the size of
> CLORB with the size of the Lisp side of the mod_lisp protocol, for
> instance).

I'm sure it will typically be much larger.  OTOH, since it's part of a
server, a meg or so of FASL (for very several larger APIs) is not
likely a problem.  Certainly, it has never been a problem that I've
seen (often being less than 1% of the application _data_ space).
So, if you are using this in a server, I would be quite sure this size
thing will be a complete non-issue.


> The major advantage of CORBA or XML-RPC is that they are both
> already there for many languages.  I still need to experiment a bit
> with CORBA to see how easy it is to set up - it seems to have a high
> labor cost in going from a clean installation to objects talking to
> each other.

Really, this is not an issue, even though you keep fretting over it.
It is in fact basically trivial.  And the Lisp side (server or client)
is _really_ trivial.  Not knowing (or wanting to know) anything much
about PHP, I didn't know how CORBA fits in that picture.  But I just
Googled for a few minutes and found these bits:

;; CORBA for PHP
http://uk.php.net/manual/en/ref.satellite.php

;; Some talk on it's use
http://lists.xml.org/archives/xml-dev/200108/msg00852.html


Excerpt:
------------------
This is how complex CORBA is to use...  Assuming we have an IDL
interface file reading:

                        interface MyInterface {
                            void SetInfo (string info);
                            string GetInfo();

                            attribute int value;
                        }

When we've installed the Satellite CORBA client library, we can write PHP
to access something with that interface like this:

			<?php
                        $obj = new OrbitObject ($ior);

                        $obj->SetInfo ("A 2GooD object");

                        echo $obj->GetInfo();

                        $obj->value = 42;

                        echo $obj->value;
                        ?>

$ior is the object reference - think URL.

That will handle exceptions, too. SOAP is not unique in being able to
differentiate exceptions from valid method returns!

----------------------


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <44346c69$0$15794$14726298@news.sunsite.dk>
jayessay wrote:
> Ari Johnson <·········@gmail.com> writes:
> 
>> high-latency. CORBA seems promising, but massive (compare the size of
>> CLORB with the size of the Lisp side of the mod_lisp protocol, for
>> instance).
> 
> I'm sure it will typically be much larger.  OTOH, since it's part of a
> server, a meg or so of FASL (for very several larger APIs) is not
> likely a problem.  Certainly, it has never been a problem that I've
> seen (often being less than 1% of the application _data_ space).
> So, if you are using this in a server, I would be quite sure this size
> thing will be a complete non-issue.

I was mostly thinking larger in terms of programmer comprehension.  It 
is more for *me* to have to understand in order to bring it to bear. 
When I find time, I do intend to set up enough CORBA pieces to 
experiment. :)
From: Tim X
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87acazdmk0.fsf@tiger.rapttech.com.au>
Ari Johnson <·········@gmail.com> writes:

> Tim X wrote:
>> However, I suspect, for compatibility and maintenance reasons, the
>> sensible approach would be to generalise the probelm and work out a
>> robust and flexible mechanism within PHP which would allow PHP to call
>> a remote procedure and have its returned values mapped to a PHP data
>> structure.
>> Of course, PHP may already have this facility, in which case, your
>> problem is already solved and nothing needs to be done. 
>
>
> Right.  That's what I was after - an inter-process communication
> protocol that allows PHP or any other language (PHP was just the
> example used since it is popular in web applications and a perfect
> example of a language that is excellent for writing HTML templates but
> poor for writing complicated application logic) to call into Lisp.
>
> The problem is merely one of communication.  What is the best way for
> Random-Language-A to talk to Common Lisp?  XML-RPC is high-latency.
> CORBA seems promising, but massive (compare the size of CLORB with the
> size of the Lisp side of the mod_lisp protocol, for instance).  The
> Lisp-call-as-SQL idea was novel, but unwieldy.
>
> The major advantage of CORBA or XML-RPC is that they are both already
> there for many languages.  I still need to experiment a bit with CORBA
> to see how easy it is to set up - it seems to have a high labor cost
> in going from a clean installation to objects talking to each other.

All of the existing solutions have their pros and cons. There is no
one 'right' answer here because it depends on what yo want to do. More
specifically, you are limited by what the language doing the calling
is capable of (i.e. PHP, Java or whatever). 

Now that we seem to have established what the problem is, I don't see
there is a problem. All you need to do is become familiar with the
various techniques and choose the one which suits your needs or sit
down and implement CL xmlrpc, corba, soap, whatever. Just pick one of
the existing standard approaches and implement the 'remote' side in CL
and your done. 

Tim

-- 
tcross (at) rapttech dot com dot au
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87wte7kd9d.fsf@thalassa.informatimago.com>
Ari Johnson <·········@gmail.com> writes:

> Stefan Ram wrote:
>> Ari Johnson <·········@gmail.com> writes:
>>> I prefer that the data be exposed to the template in a way that the
>>> template can turn into HTML. I see no reason that this should be
>>> any more complicated than:
>>> <?php foreach($rows as $row): ?>
>>> HTML here with <?= $row['field'] ?>
>>> <?php endforeach; ?>
>>   OK. Lisp could output this as well.
>
> What could Lisp output?  PHP code?  That's patently ridiculous, so I'm
> assuming you don't mean that.  If you mean that I could write:
> <?lisp (loop for row in rows do ?>
> HTML here with <?= (field row) ?>
> <?lisp ) ?>
>
> ...then please tell me which existing template system allows that, and
> where its thorough and novice-friendly documentation is, preferably in
> ISBN form, so I can tell my web designers just what they need to read
> in order to write their templates.
>
>
> If you mean something else entirely, please let me know.  Thanks.

How many times do we have to tell you?!?!?!?

USE LISP!  USE UCW!  JUST DO IT!!!!

http://cliki.net/UCW 

UCW template example: range-view.tal
It's not <?lisp ... ?>, it's <tal:lisp> ... </tal:lisp> 
but you should get it anyways.
------------------------------------------------------------------------

<!-- -*- html -*- -->

<table xmlns:tal="http://common-lisp.net/project/bese/tal/core"
      xmlns:ucw="http://common-lisp.net/project/ucw/core"
      tal:in-package="it.bese.ucw">
  <tr tal:dolist="$items">
    <td valign="top" tal:content="$index">item index</td>
    <td><tal:lisp>(render-range-view-item (context.response *context*) $component $item)</tal:lisp></td>
  </tr>
  <tr>
    <td colspan="2">
      <table width="100%" cellpadding="2">
        <tr>
          <td>
            <a ucw:action="(scroll-start $component)"><tt>[&lt;&lt;]</tt></a>
          </td>
          <td>
            <a tal:when="$previousp"       ucw:action="(scroll-backward $component)"><tt>[&lt;]</tt></a>
            <a tal:when="(not $previousp)"                                          ><tt>[&lt;]</tt></a>
          </td>
          <td align="center">
            <tal:tal tal:dolist="$windows">
              [<a tal:when="(not $selected)"
                  ucw:action="(scroll-to-page $component (1- $num))"
                  tal:content="$num">i</a>
              <tal:tal tal:when="$selected" tal:content="$num">i</tal:tal>]
            </tal:tal>
          </td>
          <td>
            <a tal:when="$nextp"       ucw:action="(ucw::scroll-forward $component)"><tt>[&gt;]</tt></a>
            <a tal:when="(not $nextp)"                                              ><tt>[&gt;]</tt></a>
          </td>
          <td>
            <a ucw:action="(ucw::scroll-end $component)"><tt>[&gt;&gt;]</tt></a>
          </td>
        </tr>
      </table>
    </td>      
  </tr>
</table>

------------------------------------------------------------------------


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p7uo$b37$1@emma.aioe.org>
Pascal Bourguignon wrote:
> Ari Johnson <·········@gmail.com> writes:
> 
>> Stefan Ram wrote:
>>> Ari Johnson <·········@gmail.com> writes:
>>>> I prefer that the data be exposed to the template in a way that the
>>>> template can turn into HTML. I see no reason that this should be
>>>> any more complicated than:
>>>> <?php foreach($rows as $row): ?>
>>>> HTML here with <?= $row['field'] ?>
>>>> <?php endforeach; ?>
>>>   OK. Lisp could output this as well.
>> What could Lisp output?  PHP code?  That's patently ridiculous, so I'm
>> assuming you don't mean that.  If you mean that I could write:
>> <?lisp (loop for row in rows do ?>
>> HTML here with <?= (field row) ?>
>> <?lisp ) ?>
>>
>> ...then please tell me which existing template system allows that, and
>> where its thorough and novice-friendly documentation is, preferably in
>> ISBN form, so I can tell my web designers just what they need to read
>> in order to write their templates.
>>
>>
>> If you mean something else entirely, please let me know.  Thanks.
> 
> How many times do we have to tell you?!?!?!?
> 
> USE LISP!  USE UCW!  JUST DO IT!!!!
> 
> http://cliki.net/UCW 
> 
> UCW template example: range-view.tal
> It's not <?lisp ... ?>, it's <tal:lisp> ... </tal:lisp> 
> but you should get it anyways.

Until Mr. Baringer's quite polite post a moment ago, nobody mentioned 
TAL.  The UCW tutorial video, if I recall from having watched it in its 
entirety several months ago, does not make any mention of this.

Also, remember that the point is not whether I should get it or not.  I 
have no problem understanding code in any language.  The point is 
whether or not my *web developers* get it.  I apologize if that was unclear.

 > <helpful example TAL code snipped; thank you :)>
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87slovkcg7.fsf@thalassa.informatimago.com>
Ari Johnson <·········@gmail.com> writes:
> Until Mr. Baringer's quite polite post a moment ago, nobody mentioned
> TAL.  The UCW tutorial video, if I recall from having watched it in
> its entirety several months ago, does not make any mention of this.

TAL is part of UCW, but I don't see the point in naming the template
subsystem of UCW when you don't know the first word of it.

> Also, remember that the point is not whether I should get it or not.
> I have no problem understanding code in any language.  The point is
> whether or not my *web developers* get it.  I apologize if that was
> unclear.

Yes.  Of course, I'm not proposing to write a lot of lisp code in TAL
templates either.  The possibility is there, but usually you just have
simple function calls in the templates and all the code remains in the
lisp image.  Therefore, apart from the syntax of the TAL specific
tags, the documentation comes actually from your application, in the
form of a list of s-exrs the web designers can insert to call the
various functionalities.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

PLEASE NOTE: Some quantum physics theories suggest that when the
consumer is not directly observing this product, it may cease to
exist or will exist only in a vague and undetermined state.
From: Marco Baringer
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m2fykvajnw.fsf@bese.it>
Ari Johnson <·········@gmail.com> writes:

> I prefer that the data be exposed to the template in a way that the
> template can turn into HTML.  I see no reason that this should be any
> more complicated than:
>
> <?php foreach($rows as $row): ?>
> HTML here with <?= $row['field'] ?>
> <?php endforeach; ?>

i agree with you completely.

the corresponding tal (yaclml's templating system) would be:

<tr tal:dolist="$rows">
  <td>
    HTML here with <i tal:content="$field">dummy</i>
  </td>
</tr>

[i twisted the example a bit to make TAL not look as bad as it could be]

using html-template it would be:

<!-- TMPL_LOOP rows -->
  HTML here with <!-- TMPL_VAR field -->
<!-- /TMPL_LOOP -->

using cl-emb it would be:

<% @loop rows %>
  HTML here with <% @var field %>
<% @endloop %>

if all of these look too complicated for you then i don't know of a
lisp library which fits for your needs.

-- 
-Marco
Ring the bells that still can ring.
Forget the perfect offering.
There is a crack in everything.
That's how the light gets in.
	-Leonard Cohen
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p7k3$91d$1@emma.aioe.org>
Marco Baringer wrote:
> the corresponding tal (yaclml's templating system) would be:
> 
> <tr tal:dolist="$rows">
>   <td>
>     HTML here with <i tal:content="$field">dummy</i>
>   </td>
> </tr>
> 
> [i twisted the example a bit to make TAL not look as bad as it could be]
> 
> using html-template it would be:
> 
> <!-- TMPL_LOOP rows -->
>   HTML here with <!-- TMPL_VAR field -->
> <!-- /TMPL_LOOP -->
> 
> using cl-emb it would be:
> 
> <% @loop rows %>
>   HTML here with <% @var field %>
> <% @endloop %>
> 
> if all of these look too complicated for you then i don't know of a
> lisp library which fits for your needs.

Is there any web-designer-friendly documentation for any of these?  That 
is a serious concern of mine, although any of those three is potentially 
a good solution to the problem at hand.
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <3T31rs4hI718Nv8%stesch@parsec.no-spoon.de>
Ari Johnson <·········@gmail.com> wrote:
> I prefer that the data be exposed to the template in a way that the 
> template can turn into HTML.  I see no reason that this should be any 
> more complicated than:
> 
> <?php foreach($rows as $row): ?>
> HTML here with <?= $row['field'] ?>
> <?php endforeach; ?>

This kind of template tends to become more and more a jungle of
application logic, hidden in a template file.

Don't give designers too much power. They try to use it.

I have to maintain some projects that have some application logic
inside of Smarty templates. Real fun.
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p8in$drd$1@emma.aioe.org>
Stefan Scholl wrote:
> This kind of template tends to become more and more a jungle of
> application logic, hidden in a template file.

That tendency can be avoided.  I have found that it's easier to convince 
web designers not to write application logic than it is to spend half of 
your time adding features to your template language to make things your 
web designers want possible.  For instance, a template language I 
created before I learned this has a fatal flaw in that you cannot change 
anything in the <head> attribute (such as adding a <meta> tag with 
search engine hints, as was the particular desire that recently arose), 
and the templates that were in use by the time this came to the surface 
were sufficiently complex that it is impossible to make this work other 
than by adding the ridiculously kludgy "headtags" field to one of the 
database tables and then providing a {headtags} template call in the 
appropriate place.

> Don't give designers too much power. They try to use it.

Don't give them too little power - they tend to abuse you for it. :P

> I have to maintain some projects that have some application logic
> inside of Smarty templates. Real fun.

Been there, done that.  The solution is to not give the Smarty templates 
enough data to actually write application logic in them, and only give 
them enough data to perform any user interface functionality that can be 
invented.
From: Thomas Schilling
Subject: OT: Widen your horizon - deny paradigms, all of them
Date: 
Message-ID: <49atn6Fnfcr4U1@news.dfncis.de>
Stefan Ram wrote:

> http://my.opera.com/Vorlath/blog/show.dml/116808

That guy made me laugh. Not for his imaginary conversation with a
lisper, but for his other posts. I especially liked his post on
programming paradigms:

  http://my.opera.com/Vorlath/blog/show.dml/196029

Just some highlights:

  "One thing that stares you in the face is the return to sender
paradigm of function calls. Why can't I send a forwarding address of
where to continue processing the data?"

  Hm, let's call it, say, "continuations"...

  "Scoping is the next paradigm that stares you in the face. It's
impossible to come to the conclusion that scoping is good for anything
but disposable code. So if you want to create small throwaway code to
demonstrate an idea, then scoping will speed up the development. But in
large scale applications, it makes scalability and maintainability
impossible."

  Yeah! He wants LISP 1.2! Dynamic scope only, global access to all!

  "The next topic at hand is dependencies. If I type one statement, its
location in the software matters. I can't move it up one line in my code
without risk. Each statement has implicit dependencies depending on its
location in the code."

  And I especially like his conclusion:

  "I want multiple return values. I want less coupling. I want easier
maintenance. I want better scalability. I want better parallelism out of
my software. I want better extensibility. I want better portability. It
really isn't rocket science to have all this. So what gives? I read
conspiracy theories for a laugh. But I can't come to any other
conclusion than there's been an active cover-up in software technology.
The only other conclusion is that programmers who do language design for
general purpose languages are idiots. So which is it? There is a third
option, but I refuse to pronounce that I'm just too damn smart that it's
only obvious to me. That's would be ridiculous. Or is it?"

  This must be a fake.  This cannot be real, can it?
From: QCD Apprentice
Subject: Re: OT: Widen your horizon - deny paradigms, all of them
Date: 
Message-ID: <e0pk6g$83g$1@news.doit.wisc.edu>
Thomas Schilling wrote:
> Stefan Ram wrote:
> 
> 
>>http://my.opera.com/Vorlath/blog/show.dml/116808
> 
> 
> That guy made me laugh. Not for his imaginary conversation with a
> lisper, but for his other posts. I especially liked his post on
> programming paradigms:
> 
>   http://my.opera.com/Vorlath/blog/show.dml/196029
<snip oddness>
>   This must be a fake.  This cannot be real, can it?

I'm sorry man, but he might be quite real.
Yesterday he had a delightful post in which claims that he just doesn't 
believe in that whole "neutrino" thing.  It did not appear to be an 
April Fool's joke, given past posts with other bizzare misconceptions 
about physics.
Have you been on the sci.physics* hierarchy before?  Arrogant people 
with bizzare ideas are a dime a dozen.
From: Tim X
Subject: Re: OT: Widen your horizon - deny paradigms, all of them
Date: 
Message-ID: <87slotejl1.fsf@tiger.rapttech.com.au>
Thomas Schilling <······@yahoo.de> writes:

> Stefan Ram wrote:
>
>> http://my.opera.com/Vorlath/blog/show.dml/116808
>
> That guy made me laugh. Not for his imaginary conversation with a
> lisper, but for his other posts. I especially liked his post on
> programming paradigms:
>
>   http://my.opera.com/Vorlath/blog/show.dml/196029
>
> Just some highlights:
>
>   "One thing that stares you in the face is the return to sender
> paradigm of function calls. Why can't I send a forwarding address of
> where to continue processing the data?"
>
>   Hm, let's call it, say, "continuations"...
>
>   "Scoping is the next paradigm that stares you in the face. It's
> impossible to come to the conclusion that scoping is good for anything
> but disposable code. So if you want to create small throwaway code to
> demonstrate an idea, then scoping will speed up the development. But in
> large scale applications, it makes scalability and maintainability
> impossible."
>
>   Yeah! He wants LISP 1.2! Dynamic scope only, global access to all!
>
>   "The next topic at hand is dependencies. If I type one statement, its
> location in the software matters. I can't move it up one line in my code
> without risk. Each statement has implicit dependencies depending on its
> location in the code."
>
>   And I especially like his conclusion:
>
>   "I want multiple return values. I want less coupling. I want easier
> maintenance. I want better scalability. I want better parallelism out of
> my software. I want better extensibility. I want better portability. It
> really isn't rocket science to have all this. So what gives? I read
> conspiracy theories for a laugh. But I can't come to any other
> conclusion than there's been an active cover-up in software technology.
> The only other conclusion is that programmers who do language design for
> general purpose languages are idiots. So which is it? There is a third
> option, but I refuse to pronounce that I'm just too damn smart that it's
> only obvious to me. That's would be ridiculous. Or is it?"
>
>   This must be a fake.  This cannot be real, can it?

I hate to say it, but it actually might be. I recently visited my old
comp sci faculty and spent some time talking to some of the comp sci
staff - it was truely demoralising. All the courses that use to be
there on things like computability, compiler and language design,
machine architecture, logic, cryptography  etc have been replaced with courses on
java, writing web apps, windows system administration etc, There use
to be honours and post grad students doing some really interesting
research and there was a strong theme of computer science and theory.
Now they talk about being leaders in IT and the papers being written
are either rediculous untested and untestable science fiction ravings
or 'soft' research on how to do secure on-line transactions etc. As
far as they were concerned, the halting problem is when you tell
windows to shutdown and it just sits there!   
-- 
tcross (at) rapttech dot com dot au
From: Thomas Schilling
Subject: Re: OT: Widen your horizon - deny paradigms, all of them
Date: 
Message-ID: <49frrpFofqptU1@news.dfncis.de>
Tim X wrote:

> I hate to say it, but it actually might be. I recently visited my old
> comp sci faculty and spent some time talking to some of the comp sci
> staff - it was truely demoralising. All the courses that use to be
> there on things like computability, compiler and language design,
> machine architecture, logic, cryptography  etc have been replaced with courses on
> java, writing web apps, windows system administration etc, There use
> to be honours and post grad students doing some really interesting
> research and there was a strong theme of computer science and theory.
> Now they talk about being leaders in IT and the papers being written
> are either rediculous untested and untestable science fiction ravings
> or 'soft' research on how to do secure on-line transactions etc. As
> far as they were concerned, the halting problem is when you tell
> windows to shutdown and it just sits there!   

Ouch!
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <877j67lsd9.fsf@thalassa.informatimago.com>
Ari Johnson <·········@gmail.com> writes:
> That still requires me to do two things, neither of which I want to do
> again if I can help it (having done both more times than I like, and
> having always spent more time on them than on writing the actual
> application, and having furthermore found later on that doing so
> limited the capabilities of the application itself):
>
> 1. Generate HTML in my code rather than in the template.
> 2. Write my own template processor.
>
> My most recent response to Mr. Bradshaw indicates my current turn in
> thinking about how to avoid these pitfalls. :)

Then use UCW!  It already implements a template generator.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <2T31rrhfI718Nv8%stesch@parsec.no-spoon.de>
Ari Johnson <·········@gmail.com> wrote:
> That still requires me to do two things, neither of which I want to do 
> again if I can help it (having done both more times than I like, and 
 
> 1. Generate HTML in my code rather than in the template.

That was a problem in the last century when you couldn't use
markup and dividing layout and content was done with strange XML
hacks instead of HTML + CSS.

A list is a list. A headline is a headline.

We don't need to build lists with TABLE anymore.


> 2. Write my own template processor.

There are plenty of them available. But you seem to ignore all
the pointers in this thread.

NO, they don't consist of s-expressions.
From: Marco Baringer
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m2wte7ajvm.fsf@bese.it>
Pascal Bourguignon <······@informatimago.com> writes:

> If you don't like what UCW proposes, you can implement a simplistic
> HTML template in a few lines of Lisp.  It's rather trivial.

are you sure you meant ucw here?

-- 
-Marco
Ring the bells that still can ring.
Forget the perfect offering.
There is a crack in everything.
That's how the light gets in.
	-Leonard Cohen
From: Rainer Joswig
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <joswig-DB4AC0.18455702042006@news-europe.giganews.com>
In article <············@emma.aioe.org>,
 Ari Johnson <·········@gmail.com> wrote:

> Rainer Joswig wrote:
> > In article <············@emma.aioe.org>,
> >  Ari Johnson <·········@gmail.com> wrote:
> > 
> >> Wade Humeniuk wrote:
> >>> Ari Johnson wrote:
> >>>> I have no desire not to use Common Lisp to write application logic.
> >>>> In fact, quite to the contrary, I am on the verge of reimplementing
> >>>> a massive PHP application that I wrote years ago, this time in Lisp.
> >>>> This application received 47,000 hits yesterday, a fairly typical
> >>>> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
> >>>> not going to put my poor server through 50,000+ Lisp environment
> >>>> initializations in a day if I can help it.  I am also not going to
> >>>> tell my web designers "You must write your HTML in sexps and must
> >>>> do so perfectly or else no output at all will result."
> >>>>
> >>> What else is your server going to do all day??  As 60,000 is less
> >>> than 1/sec, your machine will be idle most of the day.  Its
> >>> interesting that you ascribe feelings to your machine, as if
> >>> it is being flogged to death.  Perhaps you have been around the
> >>> piggishness of PHP and the template approach too long.
> >> My server does many other things.  Why would I waste cycles, disk 
> >> accesses, and time handling each hit when they can be streamlined? 
> >> Given the choice between each hit involving 1ms of time and no disk 
> >> accesses or involving 50 times as long and digging through the disk to 
> >> load a Lisp image, why would you deliberately choose the latter?
> > 
> > If you start up a Lisp image more than once there is a good chance
> > that on reload you will hit some disk cache. Some Lisps
> > use also shared libraries. Anyway, look for Apache - Lisp
> > bridges which avoid this. See mod_lisp for example.
> > 
> 
> See my other replies regarding mod_lisp.  Maybe things have changed in 
> the past year or so, though, on that front. :)
> 
> However, as to disk caches and shared libraries - those are nice 
> OS-level speedups, but I do not like relying on the OS to do my 
> optimization for me.  Relying on disk caching to make your software run 
> quickly enough to be usable is not wise, IMHO.

Ah, so it is an religious issue.

-- 
http://lispm.dyndns.org/
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m3d5fz1ny8.fsf@rigel.goldenthreadtech.com>
Ari Johnson <·········@gmail.com> writes:

> See my other replies regarding mod_lisp.  Maybe things have changed in
> the past year or so, though, on that front. :)

Just curious - why don't you just use FastCGI???


> However, as to disk caches and shared libraries - those are nice
> OS-level speedups, but I do not like relying on the OS to do my
> optimization for me.  Relying on disk caching to make your software
> run quickly enough to be usable is not wise, IMHO.

???  That would tend to indicate you are not running or planning on
running any software at all.


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <0T31t370Iud1Nv8%stesch@parsec.no-spoon.de>
jayessay <······@foo.com> wrote:
> Ari Johnson <·········@gmail.com> writes:
>> See my other replies regarding mod_lisp.  Maybe things have changed in
>> the past year or so, though, on that front. :)
> 
> Just curious - why don't you just use FastCGI???

Only GNU/CLISP and CMUCL have support for FastCGI.
From: Frank Buss
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <nxy2ps5v9gw5$.c8q56nrs1zxg$.dlg@40tude.net>
Stefan Scholl wrote:

> Only GNU/CLISP and CMUCL have support for FastCGI.

I think every Lisp implementation which can open a server socket can
implement support for FastCGI.

-- 
Frank Buss, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1T31ta2bIud1Nv8%stesch@parsec.no-spoon.de>
Frank Buss <··@frank-buss.de> wrote:
> Stefan Scholl wrote:
>> Only GNU/CLISP and CMUCL have support for FastCGI.
> 
> I think every Lisp implementation which can open a server socket can
> implement support for FastCGI.

mod_lisp's protocol is much easier.
From: Marc Battyani
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <49qdnYslSd13Q63ZnZ2dnUVZ8qOdnZ2d@giganews.com>
"Stefan Scholl" <······@no-spoon.de> wrote
> Frank Buss <··@frank-buss.de> wrote:
>> Stefan Scholl wrote:
>>> Only GNU/CLISP and CMUCL have support for FastCGI.
>>
>> I think every Lisp implementation which can open a server socket can
>> implement support for FastCGI.
>
> mod_lisp's protocol is much easier.

Yes ;-) Not to mention cleaner faster understandable and already usable with 
all the Common Lisp implementations and several frameworks (TBNL, UCW, 
cl-modlisp, etc.)

BTW Ari Johnson has started almost exactly the same discussion 2 years ago:
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/1dd73a5d2083a386/b91b4f67d42bc489

Marc 
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0r2if$b60$1@emma.aioe.org>
Marc Battyani wrote:
> BTW Ari Johnson has started almost exactly the same discussion 2 years ago:
> http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/1dd73a5d2083a386/b91b4f67d42bc489

Thanks for the link.  I had called attention to that thread earlier in 
this one.  It actually was not at all the same discussion from the 
start, although this one immediately turned that direction. ;)
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <0T31tdrkIvhbNv8%stesch@parsec.no-spoon.de>
Marc Battyani <·············@fractalconcept.com> wrote:
> "Stefan Scholl" <······@no-spoon.de> wrote
>> Frank Buss <··@frank-buss.de> wrote:
>>> I think every Lisp implementation which can open a server socket can
>>> implement support for FastCGI.
>>
>> mod_lisp's protocol is much easier.
> 
> Yes ;-) Not to mention cleaner faster understandable and already usable with 
> all the Common Lisp implementations and several frameworks (TBNL, UCW, 
> cl-modlisp, etc.)

The bad thing is: There are only mod_lisp modules for Apache 1.3
and Apache 2.

I'd really like to use lighttpd
From: Tim Lavoie
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <6k68g3-l99.ln1@theasylum.dyndns.org>
On 2006-04-03, Stefan Scholl <······@no-spoon.de> wrote:
>
> The bad thing is: There are only mod_lisp modules for Apache 1.3
> and Apache 2.
>
> I'd really like to use lighttpd

Hi Stefan,

The proxy.server option in lighttpd works fine though, if you don't
mind the lisp process serving up its own part. 

     Cheers,
     Tim
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m38xqm1scv.fsf@rigel.goldenthreadtech.com>
Frank Buss <··@frank-buss.de> writes:

> Stefan Scholl wrote:
> 
> > Only GNU/CLISP and CMUCL have support for FastCGI.
> 
> I think every Lisp implementation which can open a server socket can
> implement support for FastCGI.

Exactly.  I've used it with ACL.  I don't know if mod_lisp differs
much from it in spirit, but it is something that is familiar (no
barrier) to most everyone in the web arena...


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Rob Warnock
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <caCdnajVJbXSIKzZRVn-vQ@speakeasy.net>
jayessay  <······@foo.com> wrote:
+---------------
| Frank Buss <··@frank-buss.de> writes:
| > Stefan Scholl wrote:
| > > Only GNU/CLISP and CMUCL have support for FastCGI.
| > 
| > I think every Lisp implementation which can open a server socket can
| > implement support for FastCGI.
| 
| Exactly.  I've used it with ACL.  I don't know if mod_lisp differs
| much from it in spirit, but it is something that is familiar
| (no barrier) to most everyone in the web arena...
+---------------

Excuse me, but the FastCGI protocol is *WAY* more complex than
the "mod_lisp" protocol, so much so that it *was* a "barrier"
to me when I was writing my first Lisp-based persistent web
application server. A large part of that complexity is the
packed-binary FastCGI "records" you have to parse and generate,
with a maximum data length of 65535 bytes per record, which
means that long requests/responses have to be broken up into
multiple records [which adds sub-request state management!].

By contrast, the plaintext, line-oriented nature of the "mod_lisp"
protocol is a "no-brainer" for anyone who has ever written a CGI
script. You can even test it by manually typing [well, probably
cut&pasting] into a socket with a "mod_lisp" server on the other
end and watching what comes out. Really.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m3u099z3l9.fsf@rigel.goldenthreadtech.com>
····@rpw3.org (Rob Warnock) writes:

> jayessay  <······@foo.com> wrote:
> +---------------
> | Frank Buss <··@frank-buss.de> writes:
> | > Stefan Scholl wrote:
> | > > Only GNU/CLISP and CMUCL have support for FastCGI.
> | > 
> | > I think every Lisp implementation which can open a server socket can
> | > implement support for FastCGI.
> | 
> | Exactly.  I've used it with ACL.  I don't know if mod_lisp differs
> | much from it in spirit, but it is something that is familiar
> | (no barrier) to most everyone in the web arena...
> +---------------
> 
> Excuse me, but the FastCGI protocol is *WAY* more complex than
> the "mod_lisp" protocol, so much so that it *was* a "barrier"
> to me when I was writing my first Lisp-based persistent web
> application server.

By "no barrier", I wasn't talking of the poor implementer, but rather
of the target customer.  This stuff is _very_ often already installed.


> A large part of that complexity is the packed-binary FastCGI
> "records" you have to parse and generate, with a maximum data length
> of 65535 bytes per record, which means that long requests/responses
> have to be broken up into multiple records [which adds sub-request
> state management!].

Been there and have done that.


> By contrast, the plaintext, line-oriented nature of the "mod_lisp"
> protocol is a "no-brainer" for anyone who has ever written a CGI
> script

So?  Once the protocol layer is handled (as admitted above,
messy/ugly) the _use_ of it is totally transparent.


>. You can even test it by manually typing [well, probably
>cut&pasting] into a socket with a "mod_lisp" server on the other end
>and watching what comes out. Really.

Sure, that sounds useful.


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1T31rgp7I6dtNv8%stesch@parsec.no-spoon.de>
Ari Johnson <···@theari.com> wrote:
> It would be even more helpful if I could provide a template system
> that my non-programmer interface designers can use on their own
> without my intervention.

And the problem is ...?

There different template systems for Common Lisp. Choose the
right one for your needs. Or write your own.
From: Marco Baringer
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m2fykwc5c2.fsf@bese.it>
Ari Johnson <···@theari.com> writes:

> I have a counter-example: myself.  I do not do web design, and I am not
> good at it.  I like to have experts at that sort of thing write the
> HTML and CSS for my web applications.  I am trying to figure out the
> best way to be able to write those applications in Lisp without
> compromising the ability to painlessly turn HTML templates handed
> to me with dummy data into active templates that get filled in with
> real data.  I need to be able to do so, like I said, painlessly.
> It would be even more helpful if I could provide a template system
> that my non-programmer interface designers can use on their own
> without my intervention.

there are _many_ lisp templating systems, html-template, cl-emb and
yaclml are the ones i've used (caveat lector: i wrote yaclml).

all of the designers i've ever worked with didn't know squat about
lisp (though they did have to learn a bit about the templating systems
we used).

> I have no desire not to use Common Lisp to write application logic.
> In fact, quite to the contrary, I am on the verge of reimplementing
> a massive PHP application that I wrote years ago, this time in Lisp.
> This application received 47,000 hits yesterday, a fairly typical
> sum that is occasionally eclipsed by numbers in the 60,000's.  I am
> not going to put my poor server through 50,000+ Lisp environment
> initializations in a day if I can help it.  I am also not going to
> tell my web designers "You must write your HTML in sexps and must
> do so perfectly or else no output at all will result."

then don't :)

1) don't. shutdown. the. lisp. ever. start it exactly once and have it
   serve connections over a socket (araneida and aserve a pure lisp
   solutions which do this, mod_lisp is another technique if you
   already have apache).

2) http://cliki.net/web has some links that you should look through, 

3) 'or else no output at all will result' just means your debugging
   code sucks.

i don't have a lisp web site which gets 47000 hits a day so i can't
say how well my apps scale (though a max hits/second number would be
more usefull to test against), i do know of lisp web sites which got
that kind of load, and they survived.

-- 
-Marco
Ring the bells that still can ring.
Forget the perfect offering.
There is a crack in everything.
That's how the light gets in.
	-Leonard Cohen
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0ouna$lid$1@emma.aioe.org>
Marco Baringer wrote:
> 1) don't. shutdown. the. lisp. ever. start it exactly once and have it
>    serve connections over a socket (araneida and aserve a pure lisp
>    solutions which do this, mod_lisp is another technique if you
>    already have apache).

I am definitely trying to avoid shutting down the lisp, ever, despite 
the common response on this thread that "It doesn't take that much time 
to start it up, so why are you worrying about it?"  I have experimented 
with each of the three things you listed, and none of them really 
convinced me that it was quite what I'm after.  I shouldn't have to use 
a reverse proxy to have a web application running on an Apache site, and 
I did not enjoy mod_lisp all that much.  Note also that I was 
discouraged from using mod_lisp about a year or so ago when I asked 
about it on cll sometime around Fall 2004.

> 2) http://cliki.net/web has some links that you should look through, 

Indeed, I am an active Cliki reader. :)

> 3) 'or else no output at all will result' just means your debugging
>    code sucks.

I shouldn't have to write a fancy debugger just to make HTML templates 
fail gracefully.  The key word here was "sexp," though.  My web 
designers shouldn't be asked to write in sexps, and the result of a 
syntax error in a sexp-based template cannot easily be programmatically 
translated into something a web designer will even begin to comprehend.

> i don't have a lisp web site which gets 47000 hits a day so i can't
> say how well my apps scale (though a max hits/second number would be
> more usefull to test against), i do know of lisp web sites which got
> that kind of load, and they survived.

I don't have peak figures.  Webalizer only breaks down by the hour and 
I'm Too Lazy (TM) to run a more detailed log analysis.  I rest assured 
that Lisp sites scale to that load level very gracefully - consider even 
the difference between (funcall #'compiled-page request-data) and 
parsing+interpreting a PHP file for each hit.  The problem is twisting 
the former into a system that you can spend more time actually coding 
(rather than dealing with dependencies and bizarre Lisp web server 
intricacies) without also losing the ease of collaboration with web 
designers.

I will take another look at the template packages that are available, 
although I don't suspect much has changed in the past year and a half. 
But it's worth a shot. :)
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <0T31rq8aI718Nv8%stesch@parsec.no-spoon.de>
Marco Baringer <··@bese.it> wrote:
> there are _many_ lisp templating systems, html-template, cl-emb and
> yaclml are the ones i've used (caveat lector: i wrote yaclml).
> 
> all of the designers i've ever worked with didn't know squat about
> lisp (though they did have to learn a bit about the templating systems
> we used).

One thing I have learned: Designers want to know how to write
simple web apps. So they learn PHP.

Giving them a template system can be as bad as letting them code
in PHP.

I prefer plain HTML (+CSS) or a picture. Then I (re)write the
HTML myself and fill it with life.
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p64d$vik$1@emma.aioe.org>
Stefan Scholl wrote:
> Marco Baringer <··@bese.it> wrote:
>> there are _many_ lisp templating systems, html-template, cl-emb and
>> yaclml are the ones i've used (caveat lector: i wrote yaclml).
>>
>> all of the designers i've ever worked with didn't know squat about
>> lisp (though they did have to learn a bit about the templating systems
>> we used).
> 
> One thing I have learned: Designers want to know how to write
> simple web apps. So they learn PHP.
> 
> Giving them a template system can be as bad as letting them code
> in PHP.

I have no objection to letting them code in PHP.  They won't use more 
than they know how to, and it lets them write every piece of HTML that 
ever gets sent to the client.  I trust my web designers.  I just know 
their technical limitations, and my own.

> I prefer plain HTML (+CSS) or a picture. Then I (re)write the
> HTML myself and fill it with life.

Web designers are notorious for wanting to change designs later on.  I 
prefer not to rewrite the same thing over and over to keep up with their 
changes, when I should be working on application logic.
From: Rainer Joswig
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <C0556E3A.33E3A%joswig@lisp.de>
Am 02.04.2006 6:58 Uhr schrieb "Ari Johnson" unter <···@theari.com> in
·························@nntp.aioe.org:

> In article <······················@speakeasy.net>,
>  ····@rpw3.org (Rob Warnock) wrote:
>> Note: I'm not saying having a persistent Lisp server running
>> to service calls from other languages is a *bad* thing -- it
>> indeed can be useful under some circumstances [although designing
>> the security protocols can be "interesting"]. But there are
>> much better ways of addressing the "startup time" boogieman.
> 
> The real bogeyman I was trying to kill is the one that keeps a lot of
> people from using Lisp for anything serious - they think it's a great
> language but that it's impractical to interface with people in.  This
> is a valid point and one that a shrink-wrapped user-friendly
> all-inclusive IPC would alleviate.

But you know that there are already lots CORBA, SOAP, RPC, HTTP,
SNMP, Midi, X11 interfaces for Lisp? Interfaces where you can
talk to a Lisp? It is not that things can't be improved,
but is it just not so that there is nothing and there aren't
any applications already that are doing this.

Many (commercial) applications have no problems communicating
to the outside or to be used from the outside.

It is not unusal for example that a Lisp system talks HTTP and the data
is described in XML. An example from the semantic web area is DIG,
where the Lisp system listens to HTTP POST requests containing
XML data (encoding logic queries). I also use an HTTP server
in Lisp as a communication interface.

There is also quite exotic stuff - Franz sells an NFS server
written in Lisp. Or my Lisp Machine at home provides server protocols
like SMTP, NFS, X11 and SNMP. OpenMCL on the Mac has a sophisticated
Midi interface - so that it can react to incoming Midi events
with a 'real-time' event scheduler. And so on...
From: Patrick May
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m2vetsicug.fsf@patrick.intamission.com>
Ari Johnson <···@theari.com> writes:
> I haven't reinvented IPC.  I've suggested a layer on top of it for
> the express purpose of making Lisp backend logic available to
> other-language frontends.  Does anything already exist for *that*?

     I'm developing Jini-based serviced-oriented architectures (SOAs)
for a couple of clients.  SOA offers an excellent means for
incorporating multiple languages into a single system.  For example, I
have one service that uses a Bayesian technique to determine the
deviation of a set of behaviors from what is historically normal.
That's not written in Java.  ;-)  Several services are rules-based;
implementing Rete in any language other than Lisp would have taken
much longer.

     The biggest difficulty is connecting Lisp services into the Jini
framework.  I have a socket interface that translates between the two,
but a Lisp RMI package would be ideal.  Given that CLOS is much more
expressive than the Java object model, it shouldn't be too hard in one
direction.  There would need to be some restrictions going from Lisp
to Java.

Regards,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc.    | The experts in large scale distributed OO
                         | systems design and implementation.
          ···@spe.com    | (C++, Java, Common Lisp, Jini, CORBA, UML)
From: Tim Bradshaw
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1143998029.409287.243990@j33g2000cwa.googlegroups.com>
Ari Johnson wrote:

> I haven't reinvented IPC.  I've suggested a layer on top of it for the
> express purpose of making Lisp backend logic available to other-language
> frontends.  Does anything already exist for *that*?

This is called CORBA.  There is a standardised Lisp binding to it.  At
least two implementations support this to my knowledge (LW and
Allegro). There are also standard bindings for C, java, etc.  I've
written applications using it, and indeed I used it as a mechanism for
being able to drive Lisp server processes from the Unix command line
(in fact in this case I wrote the command-line client in Lisp too, but
that was only because I didn't want the hassle of having to understand
the C bindings).

--tim
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0p3t4$jdd$1@emma.aioe.org>
Tim Bradshaw wrote:
> This is called CORBA.  There is a standardised Lisp binding to it.  At
> least two implementations support this to my knowledge (LW and
> Allegro). There are also standard bindings for C, java, etc.  I've
> written applications using it, and indeed I used it as a mechanism for
> being able to drive Lisp server processes from the Unix command line
> (in fact in this case I wrote the command-line client in Lisp too, but
> that was only because I didn't want the hassle of having to understand
> the C bindings).

CORBA seems to be the greatest common divisor of what I want.  How does 
it compare with XML-RPC (which I've used before, although not in Lisp), 
in terms of performance and ease-of-use in Lisp?  Is CLORB the 
standardized Lisp binding you refer to?

Thanks. :)
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m3hd5b1puw.fsf@rigel.goldenthreadtech.com>
Ari Johnson <·········@gmail.com> writes:

> Tim Bradshaw wrote:
> > This is called CORBA.  There is a standardised Lisp binding to it.  At
> > least two implementations support this to my knowledge (LW and
> > Allegro). There are also standard bindings for C, java, etc.  I've
> > written applications using it, and indeed I used it as a mechanism for
> > being able to drive Lisp server processes from the Unix command line
> > (in fact in this case I wrote the command-line client in Lisp too, but
> > that was only because I didn't want the hassle of having to understand
> > the C bindings).
> 
> CORBA seems to be the greatest common divisor of what I want.

Probably so.  I've used it to communicate pretty much seamlessly and
transparently between Lisp and '(Java C C++ python VB perl
shell-script, Ada).  There are Orbs and IDL compilers for just about
anything you can think of - though, I've not heard of anything for
PHP.  There are three big pluses for it:

1. Just about everything supports it.

2. If you're an X-monkey, you don't need to know _anything_ about the
   Y-languages used by other participants.  Indeed, you don't need to
   know much of anything about CORBA either.  The bindings are
   generated for you and you just hack away in whatever you like.

3. It just works.


Also, it works seamlessly between Windows and Uni*/Linux and anything
else for that matter, either in LAN or WAN settings.


> How does it compare with XML-RPC (which I've used before, although
> not in Lisp), in terms of performance and ease-of-use in Lisp?

It is _vastly_ easier to use and in Lisp it is vastly easier than in
anything else I've actually seen (see above list).


> Is CLORB the standardized Lisp binding you refer to?

No, the binding is an OMG specification.  Allegro and LispWorks both
have industrial strength implementations.  CLORB, is a "open source"
implementation, but I've never used it.


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <e0pvr2$lku$1@emma.aioe.org>
jayessay wrote:
> Probably so.  I've used it to communicate pretty much seamlessly and
> transparently between Lisp and '(Java C C++ python VB perl
> shell-script, Ada).  There are Orbs and IDL compilers for just about
> anything you can think of - though, I've not heard of anything for
> PHP.  There are three big pluses for it:
> 
> 1. Just about everything supports it.
> 
> 2. If you're an X-monkey, you don't need to know _anything_ about the
>    Y-languages used by other participants.  Indeed, you don't need to
>    know much of anything about CORBA either.  The bindings are
>    generated for you and you just hack away in whatever you like.
> 
> 3. It just works.
> 
> 
> Also, it works seamlessly between Windows and Uni*/Linux and anything
> else for that matter, either in LAN or WAN settings.
> 
> 
>> How does it compare with XML-RPC (which I've used before, although
>> not in Lisp), in terms of performance and ease-of-use in Lisp?
> 
> It is _vastly_ easier to use and in Lisp it is vastly easier than in
> anything else I've actually seen (see above list).

XML-RPC was pretty easy to use, in my experience.  The key thing that 
makes me think CORBA would be more difficult is the IDL thing - whereas 
with XML-RPC it was not necessary to formally specify an interface, with 
CORBA it seems to be more necessary.  That is not necessarily a bad 
thing, and in fact helps to reinforce good API design, but when you do 
the interface definition in IDL that's just one more language to learn 
and to deal with anytime you have to change an interface - which may be 
just infrequently enough to have to relearn IDL each time.

Then again, I've never actually written CORBA-enabled code, before, so I 
don't even know IDL well enough to fairly say whether it's unhealthy for 
me or not. :)

How is CORBA for latency, in your experience?

Thanks so much for the pointers. :)
From: jayessay
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m33bgu1rm0.fsf@rigel.goldenthreadtech.com>
Ari Johnson <·········@gmail.com> writes:

> jayessay wrote:
> > Probably so.  I've used it to communicate pretty much seamlessly and
> > transparently between Lisp and '(Java C C++ python VB perl
> > shell-script, Ada).  There are Orbs and IDL compilers for just about
> > anything you can think of - though, I've not heard of anything for
> > PHP.  There are three big pluses for it:
> > 1. Just about everything supports it.
> > 2. If you're an X-monkey, you don't need to know _anything_ about the
> >    Y-languages used by other participants.  Indeed, you don't need to
> >    know much of anything about CORBA either.  The bindings are
> >    generated for you and you just hack away in whatever you like.
> > 3. It just works.
> > Also, it works seamlessly between Windows and Uni*/Linux and anything
> > else for that matter, either in LAN or WAN settings.
> >
> >> How does it compare with XML-RPC (which I've used before, although
> >> not in Lisp), in terms of performance and ease-of-use in Lisp?
> > It is _vastly_ easier to use and in Lisp it is vastly easier than in
> > anything else I've actually seen (see above list).
> 
> The key thing that makes me think CORBA would be more difficult is
> the IDL thing

Nah, if you approach it with some sense it is all pretty trivial.
Mind you, if you don't (fine grained objects in the protocol and a lot
of other nonsense) you will have a glacially performing mess.

> necessary.  That is not necessarily a bad thing, and in fact helps
> to reinforce good API design, but when you do the interface
> definition in IDL that's just one more language to learn and to deal
> with anytime you have to change an interface - which may be just
> infrequently enough to have to relearn IDL each time.

Sure, that's an issue, but not one of any importance or difficulty in
my experience.  CORBA certainly isn't the greatest thing that ever was
(being based on C++ it sucks in multiple ways), but it does have the
virtues listed above.

> Then again, I've never actually written CORBA-enabled code, before, so
> I don't even know IDL well enough to fairly say whether it's unhealthy
> for me or not. :)

The thing about it is that it is very easy to get things up and
working between pretty much anything.


> How is CORBA for latency, in your experience?

Fine.  Even though IIOP has its warts, I've never seen performance as
an issue - even when used to server many simultaneous clients (GUI and
non GUI).  In general XML-RPC will be much worse in this respect.  But
again, with this sort of stuff (distributed applications) the
architecture and application protocol is the key thing; the underlying
plumbing should be mostly in the noise.


/Jon

-- 
'j' - a n t h o n y at romeo/charley/november com
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <ari-A8A9F1.22143001042006@nntp.aioe.org>
In article <··················@ram.dialup.fu-berlin.de>,
 ···@zedat.fu-berlin.de (Stefan Ram) wrote:

> ····@rpw3.org (Rob Warnock) writes:
> >Congratulations: You've just re-invented IPC and/or RPC.
> 
>   IPC, I guess from its name, assumes that there already is
>   another process running.
> 
>   COM would allow me to create an object from a central
>   repository of classes (including a name space) and then to use
>   this object. This object might even become a part of my own
>   process or might be executed as a separate process.  But it
>   does not have to be developed using the same programming
>   language as its client.
> 
>   I regret that Microsoft nowadays puts more emphasis on dotnet,
>   because COM allows even more diversity. On dotnet, they say,
>   all languages are just a skin of C#. If one could write and
>   register COM servers with Lisp, they could be used from, e.g.,
>   a VisualBasic client.

The reason I focus on backend long-running server processes is because 
Lisp is well-suited to them, whereas with COM it seems like loading up a 
Lisp image would be rather costly to do just to instantiate an object.  
I am not saying it wouldn't be incredibly useful, but it's just outside 
the scope of what I am thinking of here.

The impetus of the idea is simply that Lisp is to heavy-duty computing 
as SQL is to data storage - very very good at it and downright fun to do 
it in, but not on its own very much fun to provide an interface to that 
computing/data.
From: Pascal Bourguignon
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87fykwmy1x.fsf@thalassa.informatimago.com>
Ari Johnson <···@theari.com> writes:

> I was thinking the other day about why Lisp is so hard to use.  I came 
> to the conclusion that it is hard to use for different reasons in 
> different areas, and I came up with an idea.  Stop me if this has been 
> done. :)
>
> Lisp is hard to use for small scripts because of the cost of loading the 
> image 

Stop.  
Loading clisp or emacs --batch is as fast or faster than loading perl.
Well, the probability for me to have a copy of emacs or clisp in cache
is higher than a copy of perl...

In anycase, even if we observe these different times (with the data
in cache):

[···@thalassa pjb]$ time emacs -q -batch -eval "(print 'hello)"

hello

real	0m0.443s
user	0m0.368s
sys	0m0.068s
[···@thalassa pjb]$ time clisp -norc -ansi -q -x "(print 'hello)"
[1]> 
HELLO 
HELLO

real	0m0.060s
user	0m0.036s
sys	0m0.016s
[···@thalassa pjb]$ time perl -e 'print hello;'

real	0m0.005s
user	0m0.000s
sys	0m0.004s
[···@thalassa pjb]$ 

they're all well below the human interactive threshold.


> and the fact that it is a bit of work to load up all the things 
> you need.  For instance, compare the amount of code it would take in 
> Lisp versus in Perl to calculate some statistics from an Apache log 
> file.  In Lisp, you would have to do a bit of portability work to make 
> sure that you have the right things available to you, then load up 
> PPCRE, then initialize a bunch of stuff, and then finally scan the logs.  
> In Perl, you can probably accomplish this with a perl -e 'code here' 
> script.

In clisp too, you can write one-liners.  Just perhaps longer one-liners.

clisp -norc -ansi -q -x '
      (loop for line = (read-line t nil nil)
            with h = (make-hash-table :test (function equal))
            while line 
            do (multiple-value-bind (all url) 
                   (regexp:regexp-exec
                       #.(regexp:regexp-compile ".*\"GET \\([^\"]*\\)\"")
                       line)
                 (when url
                    (incf (gethash (regexp:match-string line url) h 0))))
            finally (maphash (lambda (k v) 
                       (when (< 2 v) (format t "~6D ~A~%" v k))) h))
    '< /tmp/access_log


> Lisp is hard to use for web applications mostly because PHP is so much 
> easier, but also because it is hard to interface Lisp with the web out 
> of the box.  You can use mod-lisp or reverse proxying to a Lisp web 
> server, but that just isn't easy.  Compare with PHP, which is a rather 
> crummy programming language but a great HTML template language when used 
> appropriately.  Add in the fact that PHP is just there - you can drop a 
> .php file into your web directory and expect it to work on the majority 
> of server installations out there.

If you really think that, you can do a great favor to the world!
Write a CL apache module to allow to write:

<ul>
<?cl?
 (loop for item in '(a b c d e)
       do (format t "<li>~A</li>~%" item))
?>
</ul>

in .html files.  ecl is embedable, you should be able to integrate it
easily with apache.


> [...]
> Why treat Lisp any differently than SQL?  Why not let people write their 
> application logic in Lisp and then connect to it from their user 
> interface language of choice?
>   $lisp = cl_connect($connstr);
>   $res = cl_call($lisp, $fun, $args);

About SQL: I'm still surprised that OO RDBMS haven't overcome SQL
yet.  After all, PHP is an OO programming language, isn't it?

There's no difficulty to connect or fork a lisp process.  Beside the
basic CLI or pipe/socket stuff, you can use protocols such as mod_lisp
or swank.  Why don't they already do it?  That is write all these PHP
pages as a simple call to a remote Lisp process.

> [...]
> Is there a precedent for any such thing?  Would it be a 
> best-of-both-worlds or worst-of-both-worlds feature?  Discuss. :)

Yes.
It's called Araneida, UCW, etc, (in the case of behind apache stuff).

Just use it!


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

PUBLIC NOTICE AS REQUIRED BY LAW: Any use of this product, in any
manner whatsoever, will increase the amount of disorder in the
universe. Although no liability is implied herein, the consumer is
warned that this process will ultimately lead to the heat death of
the universe.
From: Petter Gustad
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <878xqo8187.fsf@gustad.com>
Ari Johnson <···@theari.com> writes:

> Lisp is hard to use for small scripts because of the cost of loading the 
> image and the fact that it is a bit of work to load up all the things 

You can build a core which includes the regular expression library,
and even your parser functions. You could do something like:

lisp -core processlogfile.core -noinit -eval '(processlogfile)'
process-log-file done

> Lisp is hard to use for web applications mostly because PHP is so
> much easier, but also because it is hard to interface Lisp with the

I find Lisp much easier than PHP. However, I use a Lisp based web
server, more specifically portable allegroserve and webactions.

> Lisp is hard to use for heavy-lifting backend applications because

What is a havy-lifting backend application? Something to manipulate
the database in the bottom of your web application? I do that most of
the time through the web or through SLIME.

> Sufficiently complex web applications may actually code the V and C in 
> one language and split the M between two other languages - perhaps C and 
> SQL.  But doing so is painful, at best.

Webactions supports MVC style methodology. See:

http://opensource.franz.com/aserve/aserve-dist/webactions/doc/webactions.html

> Regardless, you use an SQL server because it does the data storage for 
> you for free.  In PHP, for instance, you just write:
>   $db = pg_connect($connstr);
>   $res = pg_query($db, $query);
>   $array = pg_fetch_array($res, null, PGSQL_ASSOC);

In Lisp I would would do something like:

(clsql:connect *db-spec*)

You can do queries and generate html based upon the query result by
defining a common lisp server pages (clp) function like this:

(def-clp-function clp_newslinks (req ent args body)
  (declare (ignore req ent args body))
  (html 
   ((:table)
    (loop for (e) being the records in 
          [select 'newslinks :order-by '(([regtime] :desc)) :limit 20 :refresh t]
          do (html 
              (:tr 
               (:td (:princ (date-to-string (regtime e)))
                    "   "
                    ((:a |href| (url e)) (:princ (title e))))))))))

In your html code you would call the clp to get the resulting table:

<clp_newslinks/>

> You simply write your database schema in SQL (mostly by creating tables) 
> and then connect to it from your implementation language.

I write my database schema in Lisp. The database used for the above
function could be written as (using cl-sql).

(clsql:def-view-class newslinks () 
  ((id :db-kind :key :db-constraints :not-null :db-type "SERIAL"
       :type integer :accessor id :initarg :id)
   (regtime :accessor regtime :type wall-time :initarg :regtime :initform (clsql:get-time))
   (url :accessor url :type (clsql:varchar 128) :initarg :url)
   (title :accessor title :type (clsql:varchar 128) :initarg :title)))

More information on:

http://clsql.b9.com/manual

> Why treat Lisp any differently than SQL?  Why not let people write their 
> application logic in Lisp and then connect to it from their user 

Why not write everything in Lisp...


Petter
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
From: Stefan Scholl
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <0T322cv7I7ajNv8%stesch@parsec.no-spoon.de>
Petter Gustad <·············@gustad.com> wrote:
> Ari Johnson <···@theari.com> writes:
>> Lisp is hard to use for web applications mostly because PHP is so
>> much easier, but also because it is hard to interface Lisp with the
> 
> I find Lisp much easier than PHP. However, I use a Lisp based web
> server, more specifically portable allegroserve and webactions.

Well, I spent some time yesterday to make it work with a
non-Debian CMUCL.

It's not so easy after all. :-)

Patch:
http://sourceforge.net/mailarchive/forum.php?thread_id=10101543&forum_id=882


I remember that I've tested portableaserve some time (months?
years?) ago with SBCL and it was not possible. Some said it was
bit-rot in acl-compat. :-)

According to the Changelog, some work was done regarding SBCL
now.
From: Petter Gustad
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <87veto8nud.fsf@filestore.home.gustad.com>
Stefan Scholl <······@no-spoon.de> writes:

> Petter Gustad <·············@gustad.com> wrote:
>> Ari Johnson <···@theari.com> writes:
>>> Lisp is hard to use for web applications mostly because PHP is so
>>> much easier, but also because it is hard to interface Lisp with the
>> 
>> I find Lisp much easier than PHP. However, I use a Lisp based web
>> server, more specifically portable allegroserve and webactions.
>
> Well, I spent some time yesterday to make it work with a
> non-Debian CMUCL.

Thanks for adding the patch. It will most likely help other users with
their portable allegroserve installation.

I don't run Debian (actually never did), but I've run portable
allegroserve on

$cat /etc/redhat-release 
Red Hat Linux release 7.1 (Seawolf)

$cat /etc/slackware-version 
Slackware 8.1

and more recently:

$ cat /etc/gentoo-release 
Gentoo Base System version 1.4.16

And several others which have been replaced. But I've probably
installed gray-streams-library.x86f on some of my systems where it's
been missing.

The issue here is related to installation. I'm running a web server
for the local basketball-team and they web-designer uses PHP. I've
spent some time building apache with PHP when they went up one PHP
revision number too...


Petter

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
From: ···············@gmail.com
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <1144311326.992701.21920@v46g2000cwv.googlegroups.com>
Hello,

Question.  I'm "curling" Lisp functions out of a running lisp (OpenMCL
connected to Apache through mod_lisp) from PHP using TBNL and URL
strings.  What's wrong with that?  It's about this simple:

<?php
 $ch =
curl_init("http://".$_SERVER['HTTP_HOST']."/tbnl/some-lisp-func?some-param=10");

       curl_setopt($ch, CURLOPT_HEADER, 0);

       curl_exec($ch);
       curl_close($ch);

       ?>

Lisp code:

(setq *dispatch-table*
      (nconc
       (mapcar (lambda (args)
                 (apply #'create-prefix-dispatcher args))
               '(("/tbnl/some-lisp-func" some-lisp-func)))
       (list #'default-dispatcher)))

(defun some-lisp-func ()
      (fmt "~A" (get-parameter "some-param")))

This is nice and dynamic and I can put it just about anywhere...I
haven't bothered but I'll probably put the PHP into a nice function
that I call all the time.  So all the designer (that needs some dynamic
info) would have to do is:

<?php
callLispFunc("lispFuncName",$arrayOfParams[]);
?>

This might a be a total nube thing not understanding the depth of the
topic so please slam me if I'm missing something...

-dustin
From: Ari Johnson
Subject: Re: Lisp backend protocol
Date: 
Message-ID: <m2y7y5fc7e.fsf@hermes.theari.com>
Time for a follow-up ... I spent a couple hours playing with this.
Here's what I've got right now - feel free to disregard, comment, or
adopt and expand as you see fit.  Obviously it's not done, since it
only handles strings, integers (PHP-sized; the Lisp side can handle
any size of integer), and lists.  But it was definitely instructive to
do this, regardless of the wisdom of doing it. :)


Sample usage:

Lisp side:
(defpackage :testpackage (:use :cl))
(in-package :testpackage)
(defun authenticate (address username password)
  (declare (ignore address)) ; Sadly, haven't figured it out in acl-compat yet
  (and (string= username "user") (string= password "pass")))
(defun sum (username &rest numbers)
  (declare (ignore username)) ; Could be useful in other functions
  (apply #'+ numbers))
(export 'authenticate 'sum) ; Only exported symbols are allowed by the server

PHP side:
<?php
require_once("lisp_bridge.inc.php");
$lisp = new Lisp_Connection;
$lisp->connect("localhost" 7777 "testpackage" "user" "pass")
  || die($lisp->error());
$args = array(1, 2, 3, 4, 5);
($result = $lisp->call("sum", $args)) || die($lisp->error());
// $result is an array of the multiple values returned by the Lisp function,
// which is only one item long here
?>
<p>Result was <?= $result[0] ?></p>
<?php
$lisp->close() || die($lisp->error());
?>


network.lisp (this is loaded as part of an ASDF system, but that isn't
relevant here; note also that it uses acl-compat although possibly
naively as it was originally done with sb-socket; acl-compat usage
only tested on OpenMCL):

(in-package #:cl-backend)

(defconstant +type-string+ 1)
(defconstant +type-integer+ 2)
(defconstant +type-list+ 3)

(defun network-server-start (&optional (handler 'network-connection-process)
			     &key (address "127.0.0.1") (port 7777))
  "Start the main network loop in a thread"
  (let (socket process)
    (unwind-protect
	 (progn
	   (setf socket (acl-socket:make-socket :connect :passive
						:local-port port
						:local-host address))
	   (setf process (acl-mp:process-run-function
			  "Network server"
			  #'network-server-loop socket handler)))
      (unless (acl-mp:process-active-p process)
	(close socket)))
    (if (acl-mp:process-active-p process)
	process
	nil)))

(defun network-server-loop (socket handler)
  "The main network loop - run it in a thread"
  (do ()
      (*network-server-loop-exit*)
    (handler-case
	(network-server-loop-once handler socket)
      (t nil)))
  (close socket))

(defun network-server-loop-once (handler socket)
  "Run the network loop one time"
  (let ((stream (acl-socket:accept-connection socket :wait t))
	process)
    (if stream
	(progn
	  (unwind-protect
	       (setf process (acl-mp:process-run-function
			      "Network client handler"
			      (if (symbolp handler)
				  (symbol-function handler)
				  handler)
			      stream
			      nil)) ; TODO: the darn address
	    (unless (acl-mp:process-active-p process)
	      (close stream)))
	  (if (acl-mp:process-active-p process)
	      process
	      nil))
	nil)))

(defun network-connection-process (stream address)
  "Handle a connection - parent function to handle multiprocessing"
  (unwind-protect
       (network-handle-connection stream address)
    (force-output stream)
    (close stream)
    (acl-mp:process-kill (acl-compat.mp:current-process))))

(defun network-handle-connection (stream address)
  "Handle a connection"
  (labels ((getbyte ()
             (read-byte stream nil 0))

	   (getitem ()
	     (let ((type (getbyte)))
	       (cond
		 ((= type +type-string+) (getstring))
		 ((= type +type-integer+) (getinteger))
		 ((= type +type-list+) (getlist))
		 (t nil))))

	   (putitem (item)
	     (etypecase item
	       (string (putstring item))
	       (integer (putinteger item))
	       (list (putlist item))))

           (getstring ()
             (let* ((length (getbyte))
                    (sequence (make-array length :element-type 'unsigned-byte
                                          :initial-element 0)))
               (read-sequence sequence stream :end length)
               (sequence-to-string sequence)))

	   (getinteger ()
	     (let* ((length (getbyte))
		    (sequence (make-array length :element-type 'unsigned-byte
					  :initial-element 0)))
	       (read-sequence sequence stream :end length)
	       (sequence-to-integer sequence)))

	   (getlist ()
	     (let* ((length (getbyte)))
	       (loop for i from 0 below length
		     collecting (getitem))))

           (putbyte (byte)
             (write-byte byte stream))

           (putstring (string)
             (let ((sequence (sequence-from-string string)))
	       (putbyte +type-string+)
               (putbyte (length sequence))
               (write-sequence sequence stream)))

	   (putinteger (integer)
	     (let ((sequence (sequence-from-integer integer)))
	       (putbyte +type-integer+)
	       (putbyte (length sequence))
	       (write-sequence sequence stream)))

	   (putlist (list)
	     (putbyte +type-list+)
	     (putbyte (length list))
	     (loop for item in list
		   do (putitem item))))
    (let* ((username (getitem))
           (password (getitem))
           (package-name (getitem))
           (package (find-package (string-upcase (or package-name "")))))
      (if (authenticate address username password package)
          (progn
            (putitem "OKAY")
            (force-output stream)
	    (do ((function (getitem) (getitem)))
                ((or (null function) (string= function "")))
	      (setf function (string-upcase function))
              (let* ((numargs (getbyte))
                     (args (loop for i from 1 to numargs
                                 collecting (getitem)))
                     (fun (lookup-function package function)))
                (if fun
                    (let ((result (multiple-value-list
                                   (apply fun username args))))
                      (putitem "OKAY")
                      (putbyte (length result))
                      (loop for i in result do (putitem i)))
                    (putitem "NO SUCH FUNCTION")))
              (force-output stream)))
          (putitem "INVALID AUTHENTICATION")))))

(defun sequence-to-string (sequence)
  (let* ((length (length sequence))
         (string (make-string length :initial-element #\Space)))
    (loop for i from 0 below length
          do (setf (char string i) (code-char (aref sequence i))))
    string))

(defun sequence-from-string (string)
  (let* ((length (length string))
         (sequence (make-array length :element-type 'unsigned-byte
                               :initial-element 0)))
    (loop for i from 0 below length
          do (setf (aref sequence i) (char-code (char string i))))
    sequence))

(defun sequence-to-integer (sequence)
  (let* ((length (length sequence))
	 (integer 0)
	 (negative (= (aref sequence 0) 1)))
    (loop for i from 1 below length
	  do (setf integer (logior (ash integer 8) (aref sequence i))))
    (when negative
      (setf integer (- integer)))
    integer))

(defun sequence-from-integer (integer)
  (let* ((negative (< integer 0))
	 (sequence (make-array 1 :element-type 'unsigned-byte
			       :initial-element 0
			       :adjustable t)))
    (when negative
      (setf (aref sequence 0) 1)
      (setf integer (- integer)))
    (loop until (zerop integer)
	  do (progn
	       (adjust-array sequence (1+ (length sequence)))
	       (setf (aref sequence (1- (length sequence)))
		     (logand integer #b11111111))
	       (setf integer (ash integer -8))))
    (values sequence negative)))

(defun authenticate (address username password package)
  "Determine whether the client is authorized"
  (if package
      (let* ((authsym (find-symbol "AUTHENTICATE" package))
             (authfun (symbol-function authsym)))
        (funcall authfun address username password))
      nil))

(defun lookup-function (package function-name)
  "Find the function"
  (multiple-value-bind (symbol status)
      (find-symbol function-name package)
    (and symbol
         (equalp status :external)
         (fboundp symbol)
         (symbol-function symbol))))


lisp_bridge.inc.php (PHP client to the above):

<?php

define(LISP_TYPE_STRING, 1);
define(LISP_TYPE_INTEGER, 2);
define(LISP_TYPE_LIST, 3);

class Lisp_Connection {
  var $_error;
  var $_socket;

  function Lisp_Connection() {
    $this->_error = null;
    $this->_socket = null;
  }

  function connect($hostname, $port, $package, $username, $password) {
    $this->_error = null;

    if($this->_socket)
      $this->close();

    $address = gethostbyname($hostname);

    if(!is_int($port))
      $port = getservbyname($port, "tcp");

    $this->_socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);

    if($this->_socket < 0) {
      $this->_error = "socket_create: " . socket_strerror($this->_socket);
      $this->_socket = null;
      return false;
    }

    $result = socket_connect($this->_socket, $address, $port);
    if($result <= 0) {
      $this->_error = "socket_connect: " . socket_strerror($result);
      socket_close($this->_socket);
      $this->_socket = null;
      return false;
    }

    $this->_send($username);
    $this->_send($password);
    $this->_send($package);
    $result = $this->_read();

    if($result != "OKAY") {
      socket_close($this->_socket);
      $this->_socket = null;
      $this->_error = $result;
      return false;
    }

    return true;
  }

  function call($function, $args) {
    $this->_error = null;

    if(!is_array($args)) {
      $this->_error = "Arguments must be given as an array";
      return null;
    }

    $this->_send($function);
    socket_write($this->_socket, chr(count($args)), 1);
    foreach($args as $arg)
      $this->_send($arg);

    $result = $this->_read();
    if($result != "OKAY") {
      $this->_error = $result;
      return null;
    }

    $count = ord(socket_read($this->_socket, 1));
    $results = array();
    for($i = 0; $i < $count; $i++)
      $results[$i] = $this->_read();

    return $results;
  }

  function close() {
    $this->_send("");
    socket_close($this->_socket);
    $this->_error = null;
    $this->_socket = null;
    return true;
  }

  function error() {
    $tmp = $this->_error;
    $this->_error = null;
    return $tmp;
  }

  function _send($item) {
// TODO: Other data types
    if(is_string($item))
      return $this->_send_string($item);
    elseif(is_integer($item))
      return $this->_send_integer($item);
    elseif(is_array($item))
      return $this->_send_list($item);
    else
      die("Invalid type in send");
  }

  function _send_string($string) {
    socket_write($this->_socket, chr(LISP_TYPE_STRING), 1);
    socket_write($this->_socket, chr(strlen($string)), 1);
    socket_write($this->_socket, $string);
    return true;
  }

  function _send_integer($integer) {
    if($integer < 0) {
      $negative = 1;
      $integer = -$integer;
    } else {
      $negative = 0;
    }

    $pieces = chr($negative);
    $length = 1;
    while($integer > 0) {
      $pieces .= chr($integer & 0xff);
      $integer >>= 8;
      $length++;
    }

    socket_write($this->_socket, chr(LISP_TYPE_INTEGER), 1);
    socket_write($this->_socket, chr($length), 1);
    socket_write($this->_socket, $pieces, $length);
    return true;
  }

  function _send_list($list) {
    socket_write($this->_socket, chr(LISP_TYPE_LIST), 1);
    socket_write($this->_socket, chr(length($list)), 1);
    foreach($list as $item)
      $this->_send($item);
    return true;
  }

  function _read() {
// TODO: Other data types
    $type = ord(socket_read($this->_socket, 1));
    if($type == LISP_TYPE_STRING)
      return $this->_read_string();
    elseif($type == LISP_TYPE_INTEGER)
      return $this->_read_integer();
    elseif($type == LISP_TYPE_LIST)
      return $this->_read_list();
    else
      die("Invalid type in read");
  }

  function _read_string() {
    $length = ord(socket_read($this->_socket, 1));
    $string = socket_read($this->_socket, $length);
    return $string;
  }

  function _read_integer() {
    $length = ord(socket_read($this->_socket, 1));
    $string = socket_read($this->_socket, $length);

    $negative = ord($string[0]);

    $integer = 0;
    for($i = 1; $i < $length; $i++)
      $integer = ($integer << 8) | ord($string[$i]);

    if($negative == 1)
      $integer = -$integer;

    return $integer;
  }

  function _read_list() {
    $length = ord(socket_read($this->_socket, 1));
    $list = array();
    for($i = 0; $i < $length; $i++)
      $list[] = $this->_read();
    return $list;
  }
}

?>