Hi,
Is it possible to write shared libraries in common lisp? If yes, how?
A url to any tutorial is also very much appreciated...
I am a GNU/Linux user and I am not very much interested doing this job
in Allegro CL.
Thanks.
--vst
(message (Hello 'Vehbi)
(you :wrote :on '(Sun, 10 Jul 2005 15:48:48 +0300))
(
VST> Is it possible to write shared libraries in common lisp? If yes, how?
VST> A url to any tutorial is also very much appreciated...
i think ECL (embeddable common lisp) can do it. actually, it always creates
them when compiles and loads file dynamically, but i'm not sure that it's
what you need..
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"People who lust for the Feel of keys on their fingertips (c) Inity")
From: Christopher C. Stacy
Subject: Re: Writing shared libraries using Lisp
Date:
Message-ID: <ud5pqobu2.fsf@news.dtpq.com>
Vehbi Sinan Tunalioglu <···@vsthost.com> writes:
> Is it possible to write shared libraries in common lisp? If yes, how?
> A url to any tutorial is also very much appreciated...
The commercial Common Lisp implementations can deliver
your program as a shared library ("so" or "DLL").
Consult your vendor's documentation.
Vehbi Sinan Tunalioglu <···@vsthost.com> writes:
> Hi,
>
> Is it possible to write shared libraries in common lisp? If yes, how?
> A url to any tutorial is also very much appreciated...
>
> I am a GNU/Linux user and I am not very much interested doing this job
> in Allegro CL.
Creating a .so shared function library (I guess for R Project?)...
Seems like the best support is by Allegro CL which you don't
want:
http://www.isi.edu/~cyl/acl/unix-shared-library.htm
On the other hand I have found such a page:
http://www.findinglisp.com/packages/
(Actually, it is said that it's just a wrapper
around libresolv.so but I'm a little bit confused
about that).
Maybe you should check out Allegro to see if
they have something special for academic work?
--
Emre Sevinc
eMBA Software Developer Actively engaged in:
http:www.bilgi.edu.tr http://ileriseviye.org
http://www.bilgi.edu.tr http://fazlamesai.net
Cognitive Science Student http://cazci.com
http://www.cogsci.boun.edu.tr
From: Vehbi Sinan Tunalioglu
Subject: Re: Writing shared libraries using Lisp
Date:
Message-ID: <42D36ED2.6000507@vsthost.com>
Emre Sevinc wrote:
> Creating a .so shared function library (I guess for R Project?)...
Yes, exactly... Preferably, I would write extensions in "lisp" rather
then "C" for http://www.r-project.org. It is a stastical computing
environment. Luke Tierney has a wonderful work: xlispstat[1]. I think
for statistical computation, especially for time series analysis, lisp
is a wonderful opportunity.
>
> http://www.isi.edu/~cyl/acl/unix-shared-library.htm
>
I saw that page before. Nice tutorial.
> http://www.findinglisp.com/packages/
>
> (Actually, it is said that it's just a wrapper
> around libresolv.so but I'm a little bit confused
> about that).
This is the opposite way of what I want.
>
> Maybe you should check out Allegro to see if
> they have something special for academic work?
>
You are right, Allegro seems to be the only alternative at this moment...
Thanks...
--vst
ECL (http://ecls.sf.net) is a Common-Lisp environment which, apart from
having a bytecodes interpreter and compiler, can compile lisp to C and
produce native standalone executables, shared libraries and loadable
modules. This functionality is supported in Windows, Linux, FreeBSD,
NetBSD and Mac OSX.
Juanjo
Juanjo <····@arrakis.es> wrote:
+---------------
| ECL (http://ecls.sf.net) is a Common-Lisp environment which, apart from
| having a bytecodes interpreter and compiler, can compile lisp to C and
| produce native standalone executables, shared libraries and loadable
| modules. This functionality is supported in Windows, Linux, FreeBSD,
| NetBSD and Mac OSX.
+---------------
I've just started playing with ECL, and some parts of it are certainly
very interesting, especially how small simple compiled CL programs are.
But I've discovered that it has one major downside for the kind of "Lisp
scripting" I often do, namely, even with a very small executable and only
a single sharable library for the bulk of the system, it seems to take a
*loooooonnngg* time to start up. Example:
$ cat hello.lisp
(princ "Hello world!")
(terpri)
(quit)
$ ls -lt hello.c hello.o hello # intermediate & final compiled outputs
-rw-r--r-- 1 rpw3 rpw3 1036 Jun 29 20:49 hello.c
-rw-r--r-- 1 rpw3 rpw3 18984 Jun 29 20:49 hello.o
-rwxr-xr-x 1 rpw3 rpw3 23546 Jun 29 20:49 hello*
$ ldd hello # libs it depends on
libecl.so => /usr/local/lib/ecl/libecl.so (0x0000002a9566d000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000002a9591e000)
libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95a22000)
libgmp.so.3 => /usr/lib64/libgmp.so.3 (0x0000002a95baa000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a95ce2000)
/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x0000002a95556000)
$ ls -l /usr/local/lib/ecl/libecl.so
-rwxr-xr-x 1 root root 3421899 Jun 29 20:35 /usr/local/lib/ecl/libecl.so*
$
O.k. so far: a 24 KB ELF executable and a 3.5 MB runtime library, plus
the other usual libc suspects. Looks very lightweight, yes? But here's
the issue [and, yes, this is the lowest of several runs, so "libecl.so"
will have been cached in memory by the operating system]:
$ time ./hello
Hello world!
real 0m0.773s
user 0m0.760s
sys 0m0.010s
$
Whereas a CMUCL "script", with a 260 KB ELF executable and a 23 MB core
file runs *much* faster!! [And, yes, this is the lowest of several runs,
so the operating system has cached the core image, since it's mmap()'d.]
$ cat ./hello2
#!/usr/local/bin/cmucl -script
(princ "Hello world!")
(terpri)
(quit)
$ time ./hello2
Hello world!
real 0m0.016s
user 0m0.010s
sys 0m0.000s
$
I can only imagine that the additional 3/4 second ECL is consuming is
coming from recreating the base core image from scratch every time,
*NOT* something you want to be doing in CGI scripts (say)...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> I can only imagine that the additional 3/4 second ECL is consuming is
> coming from recreating the base core image from scratch every time,
> *NOT* something you want to be doing in CGI scripts (say)...
Sounds like some kind of server is the best solution for an
image-based language like Lisp. In the CGI case that means
FastCGI, so you don't startup/shutdown that much anymore; in the
shell script case it means using something like the emacs server
that runs in the background and executes stuff. With a system
like Plan9 you could even integrate in into the file system, by
making it emulate a /bin directory (if you copy a script file into
it, it gets loaded and registered).
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
From: Edi Weitz
Subject: Re: Writing shared libraries using Lisp
Date:
Message-ID: <ubr57p3kr.fsf@agharta.de>
On Wed, 13 Jul 2005 09:21:36 +0200, Ulrich Hobelmann <···········@web.de> wrote:
> Sounds like some kind of server is the best solution for an
> image-based language like Lisp.
Did you actually read the article you're replying to? Did you see the
CMUCL startup time in Rob's posting? Why do you think you can
generalize from one data point of one implementation (ECL) in beta
stage to all "image-based language[s] like Lisp?"
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Edi Weitz wrote:
> On Wed, 13 Jul 2005 09:21:36 +0200, Ulrich Hobelmann <···········@web.de> wrote:
>
>
>>Sounds like some kind of server is the best solution for an
>>image-based language like Lisp.
>
>
> Did you actually read the article you're replying to?
Yes.
> Did you see the
> CMUCL startup time in Rob's posting?
Yes, way shorter than the ECL time. But assuming that he wanted
to use ECL for some reason, not CMUCL, a server is the solution to
the startup problem.
In this thread (or some other thread the last couple of days)
people were also discussing Lisp image sizes, and that some don't
like the idea of having a Lisp image around for every single shell
script. Small ECL executable files are a solution to this at
least, but with the startup drawback.
If you have a virtual binary directory, which is just a Lisp
server that loads your scripts, you don't have startup, and you
only need one image.
> Why do you think you can
> generalize from one data point of one implementation (ECL) in beta
> stage to all "image-based language[s] like Lisp?"
Well, there's that one implementation that manages to compile to
smallish executables, and there's all those non-beta
implementations that use images. Maybe the *language* Lisp isn't
image-based, but the development style sure is, with you hacking
forms into a running image, and since *almost every single
implementation* uses images too, I don't quite see the reason for
your excitement here.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
Ulrich Hobelmann <···········@web.de> wrote:
+---------------
| Edi Weitz wrote:
| > Did you see the CMUCL startup time in Rob's posting?
|
| Yes, way shorter than the ECL time. But assuming that he wanted
| to use ECL for some reason, not CMUCL, a server is the solution to
| the startup problem.
+---------------
Actually, I *do* use CMUCL in "server mode" for all my heavy web
lifting [via a mod_lisp-like C-based CGI, which could trivially
be replaced with the real mod_lisp].
But I also like to use CL as a generic scripting language, whether
for CGI or miscellaneous "shell scripting". And in both those
applications, one wants simple scripts to run in negligible human
time, preferably under 100ms [which to my taste is a "just-noticable
difference" (1 JND) in response time]. Perl does that; /bin/sh does
that; and so do both CLISP & CMUCL.
Background note: For some time, I was using both CLISP & CMUCL for
"scripting", since while I wanted the performance of a native-compiled
implementation I was concerned about the startup time with CMUCL's
"large image". But to my surprise, I found that for small scripts
CMUCL's startup time was slightly *less* that CLISP's on all of the
platforms I use [various Linuxes and FreeBSDs on P3s, Athlons, and
Opterons (in 32-bit mode)]. So for any operating system which caches
mmap'd files in the page pool [which includes all the ones I use],
the "large image" issue is now a non-issue for me.
+---------------
| In this thread (or some other thread the last couple of days)
| people were also discussing Lisp image sizes, and that some don't
| like the idea of having a Lisp image around for every single shell
| script. Small ECL executable files are a solution to this at
| least, but with the startup drawback.
+---------------
Hunh?!? As I said a couple of weeks before this, you don't *need*
a separate "Lisp image around for every single shell script"!!
Just pre-load all of your heavily-used libraries, packages, modules,
and whatever into *one* Lisp image, installed as the default, which
gets shared by all the "scripts" [interpreted Lisp] that reference it!
The one common image gets shared/cached, and all your scripts use the
same one.
In my experience, you can throw a *lot* of additional packages
into a CMUCL image before it gets much bigger than the base image. ;-}
E.g., all of the CGI, SQL, HTML, and HTTP forms stuff I use adds
less than 10% to the size of a base CMUCL image.
Again, it doesn't matter whether it's a "libecl.so" or a "lisp.core",
if it is mmap'd into memory [MAP_PRIVATE, of course], then the file,
whether DSO, DLL, or "image", will get cached *and* shared, so the
size of the library or the image in the filesystem is pretty much a
non-problem.
No, the problem I was talking about is that one implementation takes
*way* longer than a human JND time (770%) to start up, while two others
take *way* less than a human JND time to start up (15-20%). So which
ones are people likely to want to use for interactive "shell scripting"
on a daily basis? The one that is clearly making you wait or the ones
that are "instantaneous", like most of the other shell commands you use?
+---------------
| If you have a virtual binary directory, which is just a Lisp
| server that loads your scripts, you don't have startup, and you
| only need one image.
+---------------
Uh... How do you *securely* give firewalled access to dozens of users
on a system who want to use Lisp for their *own* "shell scripts"?
Run a server per user? No thanks...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> But I also like to use CL as a generic scripting language, whether
> for CGI or miscellaneous "shell scripting". And in both those
> applications, one wants simple scripts to run in negligible human
> time, preferably under 100ms [which to my taste is a "just-noticable
> difference" (1 JND) in response time]. Perl does that; /bin/sh does
> that; and so do both CLISP & CMUCL.
OpenMCL is also very responsive in my experience.
Pascal
--
2nd European Lisp and Scheme Workshop
July 26 - Glasgow, Scotland - co-located with ECOOP 2005
http://lisp-ecoop05.bknr.net/
Rob Warnock wrote:
> Uh... How do you *securely* give firewalled access to dozens of users
> on a system who want to use Lisp for their *own* "shell scripts"?
> Run a server per user? No thanks...
Well, it's just like every user has their own shell or their own
window manager running. There's no alternative, unless you want
to check that users only use a tiny subset of Lisp (not feasible,
IMHO).
The good thing about your solution is that the Lisp only runs
while it's processing some script. A server would eat more memory
there...
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
From: Edi Weitz
Subject: Re: Writing shared libraries using Lisp
Date:
Message-ID: <u7jfvp2qr.fsf@agharta.de>
On Wed, 13 Jul 2005 09:42:10 +0200, Ulrich Hobelmann <···········@web.de> wrote:
> I don't quite see the reason for your excitement here.
What makes you think I'm excited?
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Edi Weitz wrote:
> On Wed, 13 Jul 2005 09:42:10 +0200, Ulrich Hobelmann <···········@web.de> wrote:
>
>
>>I don't quite see the reason for your excitement here.
>
>
> What makes you think I'm excited?
Well, I was just adding to Rob's comment according to my view.
Your comment sounded like you questioned if I've read anything, if
I understand anything etc., without your post having any other
point than just criticizing me, and without any constructive
criticism in it.
I hope my reasons for that post are clear now. If they aren't,
ok. You just sounded kindof pissed to me.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
Hi,
first of all I apologize for the lack of quoting -- google is horrible
for this. Second, in Spanish we have a saying, there's nothing
unsurmountable except death. I read your email and was playing only for
one night with the CVS tree and got much better results:
···@mpq3p46:~/src/ecls-new/build $ time ecl -eval '(quit)'
real 0m0.154s
user 0m0.150s
sys 0m0.003s
···@mpq3p46:~/src/ecls-new/build $ time lisp -eval '(quit)'
real 0m0.017s
user 0m0.010s
sys 0m0.007s
The first one is ECL as of my local CVS tree (to be committed by the
end of the week), the second one is CMUCL.
Now, the question is, what did I do? I just tweaked the SUBTYPEP
routine, which is quite often used throughout the core of ECL, in
particular when defining new classes. See below for more details.
The message now is the following. Why did ECL have "long" start times?
Because nobody cared much about the opposite. 770 ms was ok for me.
Nobody complained in the mailing list and hence the reduced group of
developers did nothing about it.
But there is plenty of room for improvement, as I have already shown.
So if instead of writing this message to comp.lang.lisp you had sent
either a bug report to the mailing list, maybe we would have done
something to that respect before. Incidentally, I do not read
comp.lang.lisp that often.
Regards,
Juanjo
----
The short story long: ECL has a very precise type system. The SUBTYPEP
routine is based on Hery Baker's paper about the implementation of type
lattices in Common-Lisp by using bitmaps to express type relations. I
would say it is one of the most predictable implementations, and also
terribly accurate (*), and it is a pity that I do not have more time
for improving the compiler to take advantage of the type inference
possibilites of this routine.
As for the improvement on speed, up to now we precomputed several type
tags at compilation time, in the hope that SUBTYPEP on frequently used
built in types would speed up. But it turns out that this leads most
type tags becoming bignums. A more clever implementation only computes
type tags (or bitmaps) for the types which are really required by
SUBTYPEP. As a result, all resulting bitmaps are typically fixnums and
everything goes faster.
(*) The only shortcomings are SATISFIES and compound CONS types. But
there are workarounds to both problems.
Juanjo <····@arrakis.es> wrote:
+---------------
| I read your email and was playing only for one night with the
| CVS tree and got much better results:
|
| ···@mpq3p46:~/src/ecls-new/build $ time ecl -eval '(quit)'
|
| real 0m0.154s
| user 0m0.150s
| sys 0m0.003s
| ···@mpq3p46:~/src/ecls-new/build $ time lisp -eval '(quit)'
|
| real 0m0.017s
| user 0m0.010s
| sys 0m0.007s
|
| The first one is ECL as of my local CVS tree (to be committed by the
| end of the week), the second one is CMUCL.
+---------------
*Much* better indeed! Still not as snappy as either CMUCL or CLISP,
obviously, but already enough better than it's usable for some kinds
of scripting [though maybe still not for heavily-used CGI scripts].
I'm looking forward to the improved one being available, thanks!!
+---------------
| The message now is the following. Why did ECL have "long" start times?
| Because nobody cared much about the opposite. 770 ms was ok for me.
| Nobody complained in the mailing list and hence the reduced group of
| developers did nothing about it.
|
| But there is plenty of room for improvement, as I have already shown.
| So if instead of writing this message to comp.lang.lisp you had sent
| either a bug report to the mailing list, maybe we would have done
| something to that respect before. Incidentally, I do not read
| comp.lang.lisp that often.
+---------------
I understand. You have not been specifically targeting the "scripting"
area before, and therefore have not optimized for it. On the other hand,
please consider the circumstances under which I tried ECL for the very
first time:
- I saw your comp.lang.lisp posting of May 21st, announcing the
release of ECL 0.9f.
- A few weeks later, having a spare moment, I grabbed the source
and compiled it for an x86_64 platform, mostly uneventfully
[except for some permissions problems with the "make install" --
by default Linux doesn't let root write on non-root-writable
files -- which was hacked through with only minor effort], and
tried out the generation of native executables. It worked just
fine, though I did notice the slow startup time.
- Then I saw your comp.lang.lisp posting of July 12th in this thread,
again pointing out that ECL can "produce native standalone executables,
shared libraries and loadable modules", so I replied to this, noting
that there was a issue with using it for general "scripting" due to
the startup time.
Note that so far *everything* I know about ECL I've gotten from
comp.lang.lisp, a newsgroup, a medium which I *much* prefer to
mailing lists!!
I imagine that many others are probably in the same situation,
not just with ECL, but with many implementations: one hears about
something on c.l.lisp, goes off & tries it, perhaps comments on
the experience back to the group, and leaves it at that, especially
if one already has existing alternatives that work "well enough".
It's unlikely that a casual user will join an implementation-specific
mailing list in that case, *especially* if one is browsing through a
large number of implementations.
Note that this tension between a general newsgroup (comp.lang.lisp
in this case) and implementation-specific mailing lists applies to
many other Common Lisp implementations besides ECL, and even to many
other things besides Common Lisp! ;-} ;-} I understand that detailed
discussions of specific implementation details among active developers
and serious users might be better done in a mailing list, but at
the same time there should be some openness to input on a general
newsgroup from "casual" users, who often stumble across things in
the weirdest corners!
Again, thanks for the factor-of-5 speedup in starting time of ECL
native executables! It's certainly brought it into the range of
reasonable times for "scripting" applications, especially if the
speedup also applies to "#!/usr/local/bin/ecl -shell" scripts
[which it should]. I look forward to seeing the new version soon...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: Robert Uhl
Subject: Re: Writing shared libraries using Lisp
Date:
Message-ID: <m3oe8mlvn4.fsf@4dv.net>
····@rpw3.org (Rob Warnock) writes:
>
> Note that so far *everything* I know about ECL I've gotten from
> comp.lang.lisp, a newsgroup, a medium which I *much* prefer to mailing
> lists!!
You may wish to take a look at gmane.org; they convert mailing lists
into newsgroups. A decent newsreader should offer some form of foreign
server interface and thus you can read the mailing lists in newsgroup
form, which is Very Nice Indeed.
--
Robert Uhl <http://public.xdi.org/=ruhl>
Who does not love wine, women, and song,
Remains a fool his whole life long.
--Johann Heinrich Voss
Vehbi Sinan Tunalioglu <···@vsthost.com> writes:
> Hi,
>
> Is it possible to write shared libraries in common lisp? If yes, how?
> A url to any tutorial is also very much appreciated...
http://www.sf.net/projects/xchatlisp/
is a shared library using ECL. It does not use the latest features of
ECL in this regard (i.e. not having to revert to hackery *g*), though.
Regards,
--
Julian Stecklina
LISP has survived for 21 years because it is an approximate local
optimum in the space of programming languages. - John McCarthy (1980)
On Sun, 10 Jul 2005 15:48:48 +0300, Vehbi Sinan Tunalioglu
<···@vsthost.com> wrote:
> Hi,
>
> Is it possible to write shared libraries in common lisp? If yes, how?
> A url to any tutorial is also very much appreciated...
>
> I am a GNU/Linux user and I am not very much interested doing this job
> in Allegro CL.
>
> Thanks.
> --vst
If you want to use a shared library to be used from R, have a look at
www.omegahat.org under RXlisp. It allows you to embed Xlisp-Stat in R.