I just finished reading an artical in the Dec. 2000 issue of Computer
Magazine called "Making Pointer-based Data Structures Cache Conscious", by
Chilimbi, et.al. In it, the authors suggest that because of increasing
memory-CPU speed mismatches, modifying data structures to use less space and
be more cache coherant, even at the cost of raw instruction count, might
well be a win for today's machine architectures.
Although the article was based on structures in the current language of the
day (i.e., Java), the suggestions that the authors make would probably be
even more appropriate for a language like Lisp, due to the smaller size of
objects and greater frequency of pointer traversals. One of their main
points was that data structures should be compacted by using offsets instead
of full pointers and via data compaction so that more of them would stay in
cache. In addition, they believe that objects should be allocated together
so that objects likely to be traversed to would be brought into cache, as
well.
Oddly enough, this seems suspiciously close to some of the optimizations
that were made in Lisp machines. CDR-coding reduced the size of cons cells
and the garbage collectors of the day tried to place items thought to be
used together on the same memory page. Back then, the reason was to try to
reduce disk to memory paging, but a mismatch is a mismatch, eh?
If the data in this article (as well as the corresponding articles from
SIGPAN '99) is correct, spending a few cycles on decompressing a CDR might
make sense again today. That last analysis of CDR-coding I saw was from
about 10 years ago, when memory-CPU speeds were much more even. The
analysis then was that CDR-coding took more cycles to execute and caused
branch stalls, so it wasn't as good as directly coded addresses. Thinking
about (a) superscalar execution, (b) 6-to-10 cycle cache memory latencies,
and (c) larger instruction caches and branch tables, maybe CDR coding might
make sense again.
Even more importantly, perhaps one CDR-codes only the nursery of a
generational scavenger. This allows more objects to be created before GC is
necessary and, while aging objects are pushed out to older tranches,
translation into full-offset objects can be made on those objects that are
going to be around for a while. If enough cache is resident, the entire
nursery might be cached in, leading to very substantial speed increases.
So, again we see that the world simply re-invents Lisp but, more
importantly, one of the neatest hacks from the olden days might be
reasonable again. Opinions?
faa
In article <·····················@news.uswest.net>, ·······@uswest.net says...
>
>I just finished reading an artical in the Dec. 2000 issue of Computer
>Magazine called "Making Pointer-based Data Structures Cache Conscious", by
>Chilimbi, et.al. In it, the authors suggest that because of increasing
>memory-CPU speed mismatches, modifying data structures to use less space and
>be more cache coherant, even at the cost of raw instruction count, might
>well be a win for today's machine architectures.
A less labour intensive rewrite is to make your code smaller (if you end up
with fewer cache misses, do you really care whether they're due to data or
instructions?). The latest major rewrite of Jaws (well, a few years old now)
was undertaken with just this aim in mind, and did show a noticeable
performance increase, despite having fewer unrolled loops etc.
Of course, writing compact code might mean having to eschew fashionable
techniques like ubiquitous object-orientation:-)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Ian Kemmish 18 Durham Close, Biggleswade, Beds SG18 8HZ, UK
···@jawssytems.com Tel: +44 1767 601 361
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Behind every successful organisation stands one person who knows the secret
of how to keep the managers away from anything truly important.
In article <···················@news.dircon.co.uk>,
Ian Kemmish <···@jawssystems.com> wrote:
>A less labour intensive rewrite is to make your code smaller (if you end up
>with fewer cache misses, do you really care whether they're due to data or
>instructions?).
But since instructions are mostly sequential, so the hardware can prefetch
the next block into the cache, so even larger code may have few cache
misses. With data, even if the hardware does prefetching, it will only be
helpful if the data exhibits high locality. This is true for array
structures, but when there's lots of pointers and indirection, as there is
in Lisp, it's only true if the GC keeps pointers together with the things
they point to, or if the hardware has a pointer-aware prefetcher (as
someone mentioned existed on LMI Lisp Machines).
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Robert Virding
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3a40bdcd$1@news.pi.se>
Does anyone have good reference to implementations of CDR-coding,
especially in the collector? Thanks in advance,
Robert
--
Robert Virding Tel: +46 (0)8 545 55 017
Alteon Web Systems Email: ··@bluetail.com
S:t Eriksgatan 44 WWW: http://www.bluetail.com/~rv
SE-112 34 Stockholm, SWEDEN
"Folk s�ger att jag inte bryr mig om n�gonting, men det skiter jag i".
··@bluetail.com (Robert Virding) writes:
> Does anyone have good reference to implementations of CDR-coding,
> especially in the collector? Thanks in advance,
There's a string of papers on cdr-coding on the Lisp Machines,
starting from 1969. I think your best bet is the latest ones, David
Moon's Symbolics papers, but I don't have most of them to hand. Try
this:
@inbook{moon90,
author = "David A. Moon",
title = {Symbolics Architecture},
chapter = 3,
booktitle = "Computers for Artificial Intelligence Processing",
publisher = Wiley,
year = 1990
}
There's a lucid user-level explanation of the technique in the Genera
manuals (see the Symbolics Common Lisp Language Concepts book), if you
can lay your hands on a set, but it doesn't discuss collecting at all.
The latest word on the topic, that I know of, is:
@techreport{sra92,
author = "Zhong Shao and John H. Reppy and Andrew W. Appel"
title = "Unrolling Lists"
institution = "Princeton University",
year = 1992,
number = "CS-TR-453-94"
}
This paper is more about automatically deciding the representation,
but it's got a good bibliography that includes lots of the old
cdr-coding papers. Happy hunting!
--
Pekka P. Pirinen
Adaptive Memory Management Group, Harlequin Limited
"If you don't look after knowledge, it goes away."
- Terry Pratchett, The Carpet People
···@jawssystems.com (Ian Kemmish) wrote on 07.12.00 in <···················@news.dircon.co.uk>:
> now) was undertaken with just this aim in mind, and did show a noticeable
> performance increase, despite having fewer unrolled loops etc.
Maybe not despite, but because of fewer unrolled loops. For exatly the
cache footprint reasons mentioned, I've heard people argue that these
days, unrolled loops are actually counter productive.
Kai
--
http://www.westfalen.de/private/khms/
"... by God I *KNOW* what this network is for, and you can't have it."
- Russ Allbery (···@stanford.edu)
"Frank A. Adrian" <·······@uswest.net> writes:
> I just finished reading an artical in the Dec. 2000 issue of Computer
> Magazine called "Making Pointer-based Data Structures Cache Conscious", by
> Chilimbi, et.al.
[snip]
> Oddly enough, this seems suspiciously close to some of the optimizations
> that were made in Lisp machines. CDR-coding [...]
What does CDR-coding mean in this context? - To me, CDR is Clock Data
Recovery, and that's sure wrong here.
Kai
In article <··············@orthanc.exbit-technology.com>, Kai
Harrekilde-Petersen <···@exbit.dk> wrote:
> "Frank A. Adrian" <·······@uswest.net> writes:
>
> > I just finished reading an artical in the Dec. 2000 issue of Computer
> > Magazine called "Making Pointer-based Data Structures Cache Conscious",
> > by
> > Chilimbi, et.al.
>
> [snip]
>
> > Oddly enough, this seems suspiciously close to some of the
> > optimizations
> > that were made in Lisp machines. CDR-coding [...]
>
> What does CDR-coding mean in this context? - To me, CDR is Clock Data
> Recovery, and that's sure wrong here.
A Lisp "cons cell" is a data record that contains two pointers called
(for wierd historical reasons) CAR and CDR. The most common use of them
is to build linked lists in which the CDR of each cell points to the
next element in the list and the CAR points to the data item at that
position in the list.
In fact for efficiency the "pointers" aren't always actually pointers.
In particular it's useful to be able to store an integer directly in the
CAR and/or CDR positions, rather than storing a pointer to some record
containing an integer. Therefore most Lisps arrange to reserve e.g. the
even values for actual pointers and the odd values for integers with
integers represented as e.g. 2N+1. On machines with compulsory
word-alignment for pointers this doesn't affect pointers at all, and
give you half the number of integer values you'd otherwise have (i.e. 31
bit on a 32 bit machine).
It's perhaps only slightly less common to want to store characters,
booleans, the empty list, empty array, zero length string etc directly
as well. This requires stealing another bit from the pointer
representation, forcing all pointers to be 4-byte aligned (on a byte
addressed machine).
Cons cells can be used to build any data structure, but simple
single-linked lists are very common, and it's fairly common to allocate
them all at once, which means that each cons cell ends up pointing at a
cell right next to it in memory.
"CDR-coding" consists of stealing *another* two bits from the pointer
representation. There are three possible values:
- the next word of memory is the CDR, as usual
- the next word of memory would be a pointer to NULL (the
empty list), if it was there. i.e. the current element is
the last in the list.
- the next word of memory is the CAR of the next cons cell in
the current list
You now have a total of four bits stolen from the pointer
representation, but as a result you might be able to save quite a lot of
memory for contiguously-allocated lists. The cost in extra instructions
to check for special-cases is quite high, especially when such a list is
modified by inserting or deleting elements in the middle.
-- Bruce
>>>>> "Bruce" == Bruce Hoult <·····@hoult.org> writes:
> It's perhaps only slightly less common to want to store characters,
> booleans, the empty list, empty array, zero length string etc directly
> as well. This requires stealing another bit from the pointer
> representation, forcing all pointers to be 4-byte aligned (on a byte
> addressed machine).
You can also encode those as small values and make sure that pointers
are always larger than those values, so you don't need extra bits.
> "CDR-coding" consists of stealing *another* two bits from the pointer
> representation. There are three possible values:
> - the next word of memory is the CDR, as usual
> - the next word of memory would be a pointer to NULL (the
> empty list), if it was there. i.e. the current element is
> the last in the list.
> - the next word of memory is the CAR of the next cons cell in
> the current list
The "next word is NULL" case is not strictly necessary so you
can reduce the number of required bits to just one.
> memory for contiguously-allocated lists. The cost in extra instructions
> to check for special-cases is quite high, especially when such a list is
> modified by inserting or deleting elements in the middle.
But such `setcdr' operations are not very frequent, or might
even not be allowed (in languages like ML).
Stefan
Bruce Hoult <·····@hoult.org> writes:
> You now have a total of four bits stolen from the pointer
> representation, but as a result you might be able to save quite a lot of
> memory for contiguously-allocated lists. The cost in extra instructions
> to check for special-cases is quite high, especially when such a list is
> modified by inserting or deleting elements in the middle.
Note that one such test that becomes nontrivial on General Purpose
hardware (i.e. not lisp machines) is EQ. Instead of one instruction,
those extra bits must be masked off before the comparison. I don't
know if anyone has ever considered placing those extra bits "elsewhere",
e.g. in a parallel location to the actual pointer/tag word.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Like Frank, I've been wondering about this for a couple of years now. I
am starting to think current superscalar and/or VLIW technology might
encourage clever programmers to reintroduce the concept of
'microcoding', but under a New Millennium-style buzzword (e.g. VLIW!).
Shades of the B5000....
And with 64 bits to play with, who's to say we can't spare a couple for
CDR-coding?
In article <·············@beta.franz.com>,
Duane Rettig <·····@franz.com> wrote:
>Bruce Hoult <·····@hoult.org> writes:
>> You now have a total of four bits stolen from the pointer
>> representation, but as a result you might be able to save quite a lot of
>> memory for contiguously-allocated lists. The cost in extra instructions
>> to check for special-cases is quite high, especially when such a list is
>> modified by inserting or deleting elements in the middle.
>
>Note that one such test that becomes nontrivial on General Purpose
>hardware (i.e. not lisp machines) is EQ. Instead of one instruction,
>those extra bits must be masked off before the comparison. I don't
>know if anyone has ever considered placing those extra bits "elsewhere",
>e.g. in a parallel location to the actual pointer/tag word.
Getting back to the original point of this thread, given superscalar
CPUs running in the GHz region, register-to-register or
immediate-to-register instructions are cheap (as long as they're in the
I-cache), and dereferencing pointers is *extremely* expensive (if the
targets are not in the D-cache).
Duane, I figure you would know this as well as anyone, so I have to ask:
What is the true cost of doing this masking compared to that of chasing
pointers into uncached RAM on a couple of well-known architectures,
e.g. x86, SPARC, or PowerPC? If it's done all the time I imagine the
instruction fetches would be damned near free.
-- Chuck, used-to-wannabe computer architect
--
Chuck Fry -- Jack of all trades, master of none
······@chucko.com (text only please) ······@tsoft.com (MIME enabled)
Lisp bigot, car nut, photographer, sometime guitarist and mountain biker
The addresses above are real. All spammers will be reported to their ISPs.
······@best.com (Chuck Fry) writes:
> And with 64 bits to play with, who's to say we can't spare a couple for
> CDR-coding?
Let's not make this mistake again. When IBM (and Amdahl) created their XA
architectures for the 370 compatibles (extending the address range from 24
bits to 32 bits), several programs suddenly broke. These programs, which
used the upper 8 bits of address words for tags and flags, assumed (correctly,
but without foresight) that these bits were available for use. However,
whether these programs were justified in the original designs or not, it
would be safe for me to venture a guess that _none_ of them exist in their
previous design form, and that all have been either rewritten or scrapped.
But now, we're dealing with a different issue - one that makes it even harder
to justify using more address bits: 64-bit architectures tend to define their
entire virtual space. For example, the PA-RISC with the W bit set defines
text segments starting at 0x4000000000000000, and data segments starting at
0x8000000000000000. Which bits would you steal? Some bits in the middle?
Of course, lisps on GP hardware already use bits within the address space for
tags, but they tend to be only the least significant bits. There is indeed
a cost to this usage of bits, and that is the granularity of lisp object
addressing, which becomes increased to a two-word "allocation-unit" boundary
in the major lisps, which is the minimum size needed for a cons cell. This
increased granularity presents no problem for lisp implementations, and it
implies 3 bits of tag for 32-bit lisps, and 4 bits of tag for 64-bit lisps.
This means, all other things being equal, that there is only one extra bit
left for a decision to add cdr-coding. I don't think that one bit is enough;
I believe that you need two bits.
> In article <·············@beta.franz.com>,
> Duane Rettig <·····@franz.com> wrote:
> >Bruce Hoult <·····@hoult.org> writes:
> >> You now have a total of four bits stolen from the pointer
> >> representation, but as a result you might be able to save quite a lot of
> >> memory for contiguously-allocated lists. The cost in extra instructions
> >> to check for special-cases is quite high, especially when such a list is
> >> modified by inserting or deleting elements in the middle.
> >
> >Note that one such test that becomes nontrivial on General Purpose
> >hardware (i.e. not lisp machines) is EQ. Instead of one instruction,
> >those extra bits must be masked off before the comparison. I don't
> >know if anyone has ever considered placing those extra bits "elsewhere",
> >e.g. in a parallel location to the actual pointer/tag word.
>
> Getting back to the original point of this thread, given superscalar
> CPUs running in the GHz region, register-to-register or
> immediate-to-register instructions are cheap (as long as they're in the
> I-cache), and dereferencing pointers is *extremely* expensive (if the
> targets are not in the D-cache).
I believe that EQ testing does speak directly to the point of cdr-coding;
if you measure the number of times CDR is taken vs the number of times
EQ is performed in a typical program, you find that EQ usage overwhelms
CDR usage. Thus, there is a strong impetus to optimize EQ testing
in preference to CDR operations.
Remember that any boolean test, e.g. "(when x ...)" is equivalent to
"(unless (eq x nil) ...)", in a similar manner that in C "if (x) {...}"
is equivalent to "if (x!=0) {...}", so there are many more EQ tests than
meets the eye in lisp code. If you use a bit which changes the value
of a pointer without changing its object's identity, then tests for
identity can no longer be performed in one instruction, and lisp then
loses to other languages...
> Duane, I figure you would know this as well as anyone, so I have to ask:
> What is the true cost of doing this masking compared to that of chasing
> pointers into uncached RAM on a couple of well-known architectures,
> e.g. x86, SPARC, or PowerPC? If it's done all the time I imagine the
> instruction fetches would be damned near free.
Well, obviously bit manipulation is faster than memory access, but why
are you asking this question? What do you have in mind?
And why uncached RAM, as opposed to cached data? If you're talking
about normal cons-cell lists vs cdr coding, shouldn't we assume a garbage
collector that can/does tend to keep list structures local with respect
to each other?
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig <·····@franz.com> writes:
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke. These programs, which
> used the upper 8 bits of address words for tags and flags, assumed (correctly,
> but without foresight) that these bits were available for use. However,
> whether these programs were justified in the original designs or not, it
> would be safe for me to venture a guess that _none_ of them exist in their
> previous design form, and that all have been either rewritten or scrapped.
Thanks for reminding me. Most of my stuff broke. But the mess was even
worse: they had to put in mode bits to allow older programs to run
unchanged. In a system like VM you ended up with two mode bits (EC/BC
and XA or something like that? I've forgotten most of this stuff and I
can't find my yellow booklet) and programs that only ran with one
combination of them. It was very nice to have eight bits to work with
with each pointer. Perhaps architecture designers should reconsider
giving user programs some bits in the address space.
--
Lieven Marchand <···@bewoner.dma.be>
Lambda calculus - Call us a mad club
Lieven Marchand <···@bewoner.dma.be> writes:
> It was very nice to have eight bits to work with
> with each pointer. Perhaps architecture designers should reconsider
> giving user programs some bits in the address space.
OK, let's play this request out: suppose that general purpose computer
manufacturer A decides to put in two extra bits besides the address, so
that instead of 64 bits of address, there are now 64 bits plus two extra
for tags/flags (two bits, eight bits, the story is still the same). Lisp
vendor L loves it, and designs some cdr-coding (or whatever else it wants
to do).
Now, in order to use those bits, they must be available per word, so
they must be included in the data path and memory. Thus, each data
word now has 64 bits for regular data plus these two extra bits for
L to use.
Now, C vendor (C, of course) and Java vendor J are getting on the
bandwagon of the newly popular A machines, but they notice that they
are not getting all they could be getting out of the architecture;
they notice that there are 66 bits of address width, 66 bits in the
data path, and a disconnect between the data and address in two of
the bits, and thus the address space is 1/4 the size that it could
be if those bits could be made use of. So C and L complain to A.
The A developers then set to work on the new A' (or should we call
it XA? :-) architecture which includes full functionality for the 65th
and 66th bit, it comes out and leaves L to reimplement its lisp in more
general terms or else be lost in the dust.
The moral of this is that whatever we ask of hardware vendors had better
be general-purpose enough to be useful in any language, or else it will
eventually fail.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig <·····@franz.com> writes:
>
> OK, let's play this request out: suppose that general purpose computer
> manufacturer A decides to put in two extra bits besides the address....
[long description elided]
> The moral of this is that whatever we ask of hardware vendors had better
> be general-purpose enough to be useful in any language, or else it will
> eventually fail.
I don't buy this line of reasoning. There are far too many `unusual'
machines that are commercially viable for this to be the case (think
about some of the IBM architectures).
Even `standard' computer architectures have their quirks: In many
processors there are constraints on the use of the topmost address
bits, and in many standard architectures there are extra bits on each
word for ECC (all that wasted RAM).
Of course, as it stands right now, new hardware that doesn't
faithfully mimic the Pentium instruction set at better than about
500MHz would be at an extreme disadvantage.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Joe Marshall <···@content-integrity.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> >
> > OK, let's play this request out: suppose that general purpose computer
> > manufacturer A decides to put in two extra bits besides the address....
>
> [long description elided]
>
> > The moral of this is that whatever we ask of hardware vendors had better
> > be general-purpose enough to be useful in any language, or else it will
> > eventually fail.
>
> I don't buy this line of reasoning. There are far too many `unusual'
> machines that are commercially viable for this to be the case (think
> about some of the IBM architectures).
Which machines, and how so `unusual'? Are you really still talking
about general-purpose machines?
> Even `standard' computer architectures have their quirks: In many
> processors there are constraints on the use of the topmost address
> bits, and in many standard architectures there are extra bits on each
> word for ECC (all that wasted RAM).
True enough, but ECC and parity tend to be transparent to the user
level programmer. They aren't "seen".
As for segmented architectures, even HP's PA-RISC, which forced the
assembler programmer to load segment registers and perform external
branches in their 32-bit architecture - they fixed their instruction
set to allow continuous addressing in their 64-bit architecture.
Now it is also transparent to the user-level programmer.
> Of course, as it stands right now, new hardware that doesn't
> faithfully mimic the Pentium instruction set at better than about
> 500MHz would be at an extreme disadvantage.
Well, as a software vendor to many different architectures, I disagree,
but that is really another discussion.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig <·····@franz.com> writes:
> Lieven Marchand <···@bewoner.dma.be> writes:
>
> > It was very nice to have eight bits to work with
> > with each pointer. Perhaps architecture designers should reconsider
> > giving user programs some bits in the address space.
>
> Now, in order to use those bits, they must be available per word, so
> they must be included in the data path and memory. Thus, each data
> word now has 64 bits for regular data plus these two extra bits for
> L to use.
>
> Now, C vendor (C, of course) and Java vendor J are getting on the
> bandwagon of the newly popular A machines, but they notice that they
> are not getting all they could be getting out of the architecture;
> they notice that there are 66 bits of address width, 66 bits in the
> data path, and a disconnect between the data and address in two of
> the bits, and thus the address space is 1/4 the size that it could
> be if those bits could be made use of. So C and L complain to A.
>
> The A developers then set to work on the new A' (or should we call
> it XA? :-) architecture which includes full functionality for the 65th
> and 66th bit, it comes out and leaves L to reimplement its lisp in more
> general terms or else be lost in the dust.
>
> The moral of this is that whatever we ask of hardware vendors had better
> be general-purpose enough to be useful in any language, or else it will
> eventually fail.
Maybe. Would A really do a new architecture to add 2 bits? They would
probably go for 128 data bits and 4 extra bits. A lot of architectures
have some hidden bits associated with each page entry in the paging
table for operating systems to use. Those haven't been subjected to
the kind of pressure from vendors like you describe either, although
some OS'es use them and some don't. Likewise for security rings. I
don't know offhand of an OS in popular use on Intel that uses all four
rings, but they're not going away either.
--
Lieven Marchand <···@bewoner.dma.be>
Lambda calculus - Call us a mad club
Lieven Marchand <···@bewoner.dma.be> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > The moral of this is that whatever we ask of hardware vendors had better
> > be general-purpose enough to be useful in any language, or else it will
> > eventually fail.
>
> Maybe. Would A really do a new architecture to add 2 bits?
Of course not. My whole straw proposal was an illustration to show
how it is dangerous to ask general-purpose hardware manufactureres
for those features that are not likely to be used by other langauges,
but it does not speak to the whole problem of convincing them to do
the job in the first place...
> They would
> probably go for 128 data bits and 4 extra bits. A lot of architectures
> have some hidden bits associated with each page entry in the paging
> table for operating systems to use. Those haven't been subjected to
> the kind of pressure from vendors like you describe either, although
> some OS'es use them and some don't. Likewise for security rings. I
> don't know offhand of an OS in popular use on Intel that uses all four
> rings, but they're not going away either.
Correct. Hardware designers will put in things that they feel are
important.
However, sometimes something gets in that benefits lisp and similar
languages: take the taddcc instruction on the Sparc, for example
(and this is a perfect example of how something like this eventually
fails) - The taddcc instruction is perfect for those software
architectures which view a small integer (e.g., "fixnum" in lisp) as
a signed integer shifted left two bits where the least significant bits
are 0. The taddcc/tsubcc instructions make addition and subtraction
of these numbers very efficient on a 32-bit lisp, and allow overflows
or non-fixnums to be software-trapped to a more generic add/subtract
function.
However, take a look at what has happened:
1. A more recent Sparc architecture release (either V9 or V10, I
forget which) has deprecated taddcc/tsubcc. This is presumably
because there was not enough use for it (though the lisp community,
at least, use it).
2. When we go to the 64-bit architecture, two changes occur:
a. The best representation for a fixnum is a shifted-by-three-bits
number, with three ls-bits 0.
b. Overflow occurs into the sign bit of a 64-bit machine integer,
rather than into the sign bit of a 32-bit machine integer.
The taddcc/tsubcc instructions look at two ls-bits, and check
overflow into the 32nd bit. There are no equivalent instructions
which could be considered "64-bit txxxcc", which would check the
3 ls-bits and a 64-bit signed overflow.
So we have a case where a used-(mostly/only)-by-lisp feature on GP
hardware will eventually go away.
I still believe that the wisest thing that the lisp community can ask
of hardware and operating system vendors is fast user-level trap
capability; with fewer registers saved and fewer cycles between the
trap and the return from the trap. Such a feature would be much more
likely to be widely used than cdr-coding, even in lisp systems.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In article <·············@beta.franz.com>,
Duane Rettig <·····@franz.com> wrote:
>I still believe that the wisest thing that the lisp community can ask
>of hardware and operating system vendors is fast user-level trap
>capability; with fewer registers saved and fewer cycles between the
>trap and the return from the trap. Such a feature would be much more
>likely to be widely used than cdr-coding, even in lisp systems.
I've wished for that feature several times over the years,
and I rarely use lisp.
Linux microbenchmarks have at least made people pay a little
attention to trap performance. ("My cheap Linux system does
something useless 10 times faster than your expensive commercial
Unix system.") So far I think the effects have been all in
software.
--
John Carr (···@mit.edu)
In article <·············@beta.franz.com>, Duane Rettig says...
>I still believe that the wisest thing that the lisp community can ask
>of hardware and operating system vendors is fast user-level trap
>capability; with fewer registers saved and fewer cycles between the
>trap and the return from the trap. Such a feature would be much more
>likely to be widely used than cdr-coding, even in lisp systems.
>
I will admit I haven't thought this through, but it seems like you
are almost asking for the ValueCall (VALC) operation on the Unisys
A-series (aka Clearpath NX). The ValueCall opcode will load the value from a
memory location, if the memory location is a value, your done, if it is a
pointer to data, it dereferences (loads the data pointed to) and if it is a
pointer to code, it calls it, expecting the code to return the relevant data
on the stack (A-series is a stack machine).
And who said Call-by-Name was useless? :)
- Tim McCaffrey
·················@spam2filter.unisys.com.takethisoff (Tim McCaffrey) writes:
> In article <·············@beta.franz.com>, Duane Rettig says...
>
> >I still believe that the wisest thing that the lisp community can ask
> >of hardware and operating system vendors is fast user-level trap
> >capability; with fewer registers saved and fewer cycles between the
> >trap and the return from the trap. Such a feature would be much more
> >likely to be widely used than cdr-coding, even in lisp systems.
> >
> I will admit I haven't thought this through, but it seems like you
> are almost asking for the ValueCall (VALC) operation on the Unisys
> A-series (aka Clearpath NX). The ValueCall opcode will load the value from a
> memory location, if the memory location is a value, your done, if it is a
> pointer to data, it dereferences (loads the data pointed to) and if it is a
> pointer to code, it calls it, expecting the code to return the relevant data
> on the stack (A-series is a stack machine).
>
> And who said Call-by-Name was useless? :)
Touche'. However, I would call it Call-by-deReference :-)
I don't know that architecture, but you have the right idea. Some
of the features that would have to be present in order for such an
operation to be useful to us are:
- The memory address would have to be passed into the code when
called, so that new locations for the forwarded data could be
calculated.
- The code would need to also be called in a store operation.
Whether the opration is read or write would have to be passed to
the code or else calculable.
- Access to simple data would have to be as fast as a simple load
instruction.
This implementation seems to require a lot of hardware to implement
(again, I don't know this architecture, but am guessing), and I'd also
guess that there are a couple of extra bits involved, there, to
determine whether the other bits are data, indirect, or code.
A more general purpose and portable read/write barrier would involve
setting memory page attributes in the MMU to "no access" and when the
location is read from or written to, the appropriate actions are
taken by the garbage collector's trap handler to perform the proper
forwarding and accessing of the forwarded object. This would require
a very fast trap handler, at the same order as a first level page
fault handler, perhaps partially implemented in hardware.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: John Ahlstrom
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3A379CEF.50F90032@cisco.com>
Duane wrote:
> Touche'. However, I would call it Call-by-deReference :-)
>
>
But why coin a new term when there is a perfectly
good, old one that exactly describes what is going
on? VALC was expressly designed to implement
call-by-name.
--
There is probable cause to suspect foul play,
but it is too late for us to find out.
The Supremes
Duane Rettig <·····@franz.com> wrote:
+---------------
| I still believe that the wisest thing that the lisp community can ask
| of hardware and operating system vendors is fast user-level trap
| capability; with fewer registers saved and fewer cycles between the
| trap and the return from the trap. Such a feature would be much more
| likely to be widely used than cdr-coding, even in lisp systems.
+---------------
Anybody remember the DEC PDP-10 (and DECsystem-20)?!? A certain number
of instructions were left unspecified, or rather, were specified as
"traps" in the architecture. Half of them were used for kernel calls,
but the other half of these "Unimplemented User Operations", or UUOs
as they were know, trapped to user-mode instead. ISTR you could get
the ill-mem-ref & arithmetic overflows to trap to user too, through
a kernel trampoline, but I could be mistaken about that part.
Or consider the AMD Am29000 family architecture, which used *extremely*
light-weight trampolines on some traps to handle them in user mode --
specifically, the ASSERTs in the register-window code that triggered
"spills & fills". If the address of the user-mode trap handler was
kept in a kernel register (which is was, for the spill-fill handler),
the trampoline only cost you a couple of cycles. E.g., the entire
kernel code for the "spill" trap trampoline:
uspilltrap:
mfsr tpc, PC1 ; save return address
mtsr PC1, uspill ; "branch"
add tav, uspill, 4 ; (sequential fetch)
mtsr PC0, tav ; "branch" completely
iret
Now the instructions which caused these traps were a set of "assert"
instructions that said one register was {EQ,NE,GT,LT,GE,LE} than another,
and the third arg was the trap vector number. These could very easily
be adapted to use for Lisp...
-Rob
-----
Rob Warnock, 31-2-510 ····@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043
>>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
Duane> However, take a look at what has happened:
Duane> 1. A more recent Sparc architecture release (either V9 or V10, I
Duane> forget which) has deprecated taddcc/tsubcc. This is presumably
Duane> because there was not enough use for it (though the lisp community,
Duane> at least, use it).
V9 is the latest. Note that according to my manual, only
taddcctv/tsubcctv are deprecated. They left taddcc/tsubcc in with a
note to follow these instructions with an appropriate branch or trap.
However, I have no doubt that taddcc/tsubcc will go away in some
future version.
Note that lots of other instructions were also deprecated in V9:
o all branches are replaced with branch-with-prediction
o The 32-bit multiply and divide instructions in favor of the 64-bit
operations.
Duane> 2. When we go to the 64-bit architecture, two changes occur:
Duane> a. The best representation for a fixnum is a shifted-by-three-bits
Duane> number, with three ls-bits 0.
What's wrong with having using 4 of the 16 tag bits for fixnums? Then
fixnums still have 00 as the 2 least significant bits. (Or does ACL
use all 16 tag values already?)
Duane> b. Overflow occurs into the sign bit of a 64-bit machine integer,
Duane> rather than into the sign bit of a 32-bit machine integer.
Duane> The taddcc/tsubcc instructions look at two ls-bits, and check
Duane> overflow into the 32nd bit. There are no equivalent instructions
Duane> which could be considered "64-bit txxxcc", which would check the
Duane> 3 ls-bits and a 64-bit signed overflow.
Not quite. txxxcc will set the 64-bit overflow condition on 64-bit
overflow. However, it doesn't set the 64-bit overflow condition if
the 2 ls-bits aren't zero. This is only indicated in the 32-bit
overflow condition. I don't understand why they didn't make the
64-bit overflow work the same way.
Another annoyance is that the add with carry instruction only adds in
the carry produced by a previous 32-bit operation. There's no
instruction to add the carry produced by a 64-bit operation.
Ray
Raymond Toy <···@rtp.ericsson.se> writes:
> >>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
>
> Duane> 2. When we go to the 64-bit architecture, two changes occur:
> Duane> a. The best representation for a fixnum is a shifted-by-three-bits
> Duane> number, with three ls-bits 0.
>
> What's wrong with having using 4 of the 16 tag bits for fixnums? Then
> fixnums still have 00 as the 2 least significant bits. (Or does ACL
> use all 16 tag values already?)
I assume you mean 4 of the 16 tag _values_. Peter Vaneynde asked a more
inclusive question in another article, so I'll answer this at the same
time I answer his question. It turns out that the number of bits used
for tag makes no difference to the following:
> Duane> b. Overflow occurs into the sign bit of a 64-bit machine integer,
> Duane> rather than into the sign bit of a 32-bit machine integer.
> Duane> The taddcc/tsubcc instructions look at two ls-bits, and check
> Duane> overflow into the 32nd bit. There are no equivalent instructions
> Duane> which could be considered "64-bit txxxcc", which would check the
> Duane> 3 ls-bits and a 64-bit signed overflow.
>
> Not quite. txxxcc will set the 64-bit overflow condition on 64-bit
> overflow. However, it doesn't set the 64-bit overflow condition if
> the 2 ls-bits aren't zero. This is only indicated in the 32-bit
> overflow condition. I don't understand why they didn't make the
> 64-bit overflow work the same way.
Thanks for the reminder; I looked at this over a year ago as part of
feasibility studies to port our 64-bit lisp on the Alpha to other
major architectures. My statement is still correct in its conclusion,
if not in the details; both operations of the instruction must have
64-bit effect if the instruction is to be useful to us. When I saw
the description in the architecture manual, I apparently made a mental
note: "not a 64-bit instruction", even though apparently it does have
_some_ 64-bit effects...
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: Peter Vaneynde
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <6xlmtk7fse.fsf@lant.be>
Duane Rettig <·····@franz.com> writes:
> 2. When we go to the 64-bit architecture, two changes occur:
> a. The best representation for a fixnum is a shifted-by-three-bits
> number, with three ls-bits 0.
Why only 3 bits? I would have tought to use at least 8 bits for tag
information, that or keep with the current 2 bits (if that is
sufficient). Why 3?
> I still believe that the wisest thing that the lisp community can ask
> of hardware and operating system vendors is fast user-level trap
> capability; with fewer registers saved and fewer cycles between the
> trap and the return from the trap. Such a feature would be much more
> likely to be widely used than cdr-coding, even in lisp systems.
Correct. That and considering that not all programs "die" after a
SIGILL or SIGFPU. Sometimes signals are considered "slow" and not
worthy of either fast or correct code. (see the lazy saving of the
fpu flag and the bad interaction of that with signals on Linux for
instance).
Groetjes, Peter
--
LANT nv/sa, Research Park Haasrode, Interleuvenlaan 15H, B-3001 Leuven
·····················@lant.be Phone: ++32 16 405140
http://www.lant.be/ Fax: ++32 16 404961
Peter Vaneynde <··············@lant.be> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > 2. When we go to the 64-bit architecture, two changes occur:
> > a. The best representation for a fixnum is a shifted-by-three-bits
> > number, with three ls-bits 0.
>
> Why only 3 bits? I would have tought to use at least 8 bits for tag
> information, that or keep with the current 2 bits (if that is
> sufficient). Why 3?
OK, deep question. I'll try to answer it thoroughly. [comp.arch,
sorry if this long post seems inappropriate to continue as a
cross-post; My reasoning is that although it has less to do with
computer architecture, it does have to do with Lisp architecture on
general-purpose computers, and may help make sense of some of the
other issues on this thread]
Also, before I start, I must say that I make no claims that this is the
_right_ way to set up the lisp tagging system, and other lisp vendors
may make other design decisions. However, I have a feeling that they
will agree that this is an optimal arrangement. Also, as a historical
note, our Allegro CL was a late-comer to the two-bit-tagged fixnums design;
we moved from a three-bit fixnum tag 7 or 8 years ago and joined both
Lucid(Liquid) and Harlequin(Xanalys) Lispworks in their two-tag-bit
fixnum design. The performance advantage was very specific, and is
described below. I am not sure what other CL vendors have done.
=====
I now want to answer your first question "why not 8 bits for tag?".
Consider that in a byte addressed machine, every bit you "steal"
from the address halves the number of bytes directly addressible
by the rest of the bits. So the size and placement of the tag is
important to consider.
There are two places you could put such a tag: either in the most
significant bits or the least significant bits:
- If you place the tag in the most significant bits, then you can
retain byte-addressibility, but you lose half of the address space
for each bit stolen.
Now, even if you are of the opinion that we will never need more
than 56 bits (i.e. 64 - 8) you must still deal with the fact that
programming conventions place data spaces into different ranges
of memory; e.g. Alpha allows memory to be mapped at any location,
but HP sets up address 0x4000000000000000 and higher for text,
and 0x8000000000000000 and higher for data. If you place tags in
the top bits, how are you going to distinguish between these two
spaces?
- If you place the tag in the least significant bits, you lose
granularity in the addressing for each bit used in the tag. This
is not a problem as long as there are enough bits left in the rest
of the word to address the smallest lisp object that will be
allocated, assuming that it is always aligned the same way.
If therefore you take up 8 bits of tag in the least significant
bits, the smallest lisp object you can allocate is 256 bytes.
Seems like quite a large cons cell to me!
In fact, the ideal number of bits to allocate is determined by the
minimum size of a cons cell, which is 8 bytes (i.e. 3 bits of tag)
on a 32 bit lisp, and 16 bytes (i.e. 4 bits of tag) on a 64-bit
lisp.
=====
Finally, it will make the current design easier to describe if I set
down some terms. These are my definitions, and other vendors may use
different terms:
Natural Word: This is a lisp-centric term; in a 32-bit lisp it is a
32-bit value, and in a 64-bit lisp it is a 64-bit value. Various
architectures have different ways to generate 32 or 64 bit
applications. Many set a mode bit which transforms registers. Alpha,
on the other hand, has no 32 bit mode, but provides a software-only
solution called taso (truncated address-space-option). The natural
word is the word size set up by using one of these options.
LispVal: A single natural word that describes any lisp object. It
contains a tag (in this design it is some number of ls-bits) and a
data portion. Depending on the value of the tag, the data portion
is either interpreted as a pointer to storage in the lisp heap, or
is itself the immediate data.
AU (Allocation Unit): A unit of measurement that is two LispVals in
size. All objects in the lisp heap are allocated on an AU aligned
boundary, and are each an even multiple of AUs in size. This is
also the minimum size that a non-cdr-coded cons cell can be, since
it must at least contain a CAR and CDR LispVal within it.
LispObject: A lisp object which is allocated in the lisp heap. It
is always aligned on even AU bounaries.
lisp type: Common Lisp defines many types of objects, many more than
can be distinguished by less than 6 or 7 bits. Thus, for this
kind of tagging situation, one tag value is always given the
meaning "other", which means that one must consult the first byte
of the LispObject itself for the actual type. Tags that are not
the "other" tag describe the lisp type directly.
The number of tag bits in a LispVal is optimally set to 3 bits for
32-bit ports, which allows for 8-byte granularity, sufficient to
address a cons cell of two 4-byte LispVals, and to 4 bits for 64-bit
ports, which allows for 16-byte granularity, sufficient to address
a cons cell of two 8-byte LispVals.
Now, we narrow the discussion to fixnums:
A fixnum is an immediate LispVal, because the portion of it that
is not tag bits is immediate data which represents the value of
the small integer it describes. A fixnum always includes the 0
tag, because it allows fixnums to be manufactured from machine
integers by simply shifting left. The number of bits that provide
the actual value determine the range of fixnums, beyond which
other lisp integers called bignums are allocated to provide a
smooth continuum over a large integer range.
Because fixnums are important for small integer calculations, it
may be desirable to have more than the 32-3 bits or 64-4 bits
for the fixnum value. To do this, we "steal back" some of the
tag bits by actually allocating multiple tag values for fixnums.
To steal back one bit in a 32-bit lisp, we use tags 0 and 4, and
thus the ms-bit of the tag becomes the ls-bit of the fixnum, i.e.
tag 0 is for even fixnums and tag 4 is for odd fixnums. To steal
one bit from a 64-bit lisp, we would use tag 0 for even, and tag
8 for odd fixnums, or two steal 2 bits we would use tags 0, 4, 8,
and 12.
Now the actual design decision becomes "how many tag bits should we
steal for fixnums?"
One criterion for determining this is to realize that every tag bit
which is stolen essentially halves the number of tags which can be
used for direct type determination. I don't remember where I heard
this, or about what language, but as I recall, one typed language
thought so highly of fixnums that they allocated all but one bit of
the tag for fixnums, thus leaving only tags 1, 3, 5, and 7 for three
direct-type tags and an other tag in a 32-bit port.
Raymond Toy asked asked a question along these lines when he
suggested that we steal 2 bits out of 4 for the 64-bit port, and
asked if we had already used up the tag space. The answer, Ray, is
"No", we have not yet used any more tags; there are many tags open
still in the 64-bit port, so stealing two bits would be possible.
Finally, we come to the design criterion which set our choice; it
is array access efficiency. The majority of simple arrays in any
lisp are either strings or else they contain elements of the same
size as a Natural Word. Now, all array accesses (e.g., AREF, SVREF,
and their inverses) contain an offset component, to compensate for the
tag and array header, and a shift component to compensate for the
difference between the width of the array element and the "width"
of the fixnum index, as established by the number of bits in its tag.
If the width of the array element matches the "width" of the fixnum,
then no shift is necessary, and the access is faster. In fact, tight
loops of array access operations are within 20% of C speed, and in
special cases where atomic array loops (i.e. with no chance of a gc
moving the array) where the tag offset is precalculated, the access
can be 20% faster than C. These loops tend to only differ by only
one instruction.
To put some understandability to the above paragraph, assume the
choice we made of two fixnum tag bits for 32-bit ports, and three
tag bits for 64-bit ports: A (simple-array t (*)), which is a vector
of LispVals, differs by the size of the LispVal from element to
element. Thus, the first element of an array is at offset+0,
and the second element of the array is at offset+4 for 32-bit ports
and offset+8 for 64-bit ports. Then, if we consider what the bit
pattern is for finxums as we count, we note that on 32-bit ports,
fixnums are counted as 0 (for fixnum 0), 4 (for fixnum 1), 8, 12, 16,
etc., or a difference of 4 for each fixnum increment. And in a 64-bit
port, the counting is 0 (fixnum 0), 8 (fixnum 1), 16, 24, 32 ...
or a difference of 8 per fixnum increment. Thus, fixnum increments
track exactly the address increments of array elements, and thus
fixnums are able to directly access array elements after the offset
is taken care of.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
"Duane Rettig" <·····@franz.com> wrote in message
··················@beta.franz.com...
> Finally, we come to the design criterion which set our choice; it
> is array access efficiency. The majority of simple arrays in any
> lisp are either strings or else they contain elements of the same
> size as a Natural Word. Now, all array accesses (e.g., AREF, SVREF,
> and their inverses) contain an offset component, to compensate for the
> tag and array header, and a shift component to compensate for the
> difference between the width of the array element and the "width"
> of the fixnum index, as established by the number of bits in its tag.
> If the width of the array element matches the "width" of the fixnum,
> then no shift is necessary, and the access is faster. In fact, tight
> loops of array access operations are within 20% of C speed, and in
> special cases where atomic array loops (i.e. with no chance of a gc
> moving the array) where the tag offset is precalculated, the access
> can be 20% faster than C. These loops tend to only differ by only
> one instruction.
Interesting.
You mean 20% faster than - A:
for (i = 0; i < ARRAYSIZE; i++) {
array[i]++;
}
or 20% faster than - B:
for (i = 0; i < ARRAYSIZE * sizeof(typeOfArray); i += sizeof(typeOfArray)) {
array[i]++;
}
or 20% faster than - C:
for (p = array; p < array + ARRAYSIZE; p++) {
(*p)++;
}
In practice, the relative timing of each of the above differs from
architecture to architecture - possibly from compiler to compiler.
For some architectures and compilers, it makes a big difference
if "array" is bound to a static address. A good C compiler could
convert A to B or C, if it notes that "i" is only used to index
into "array". Does gcc?
There is value in making the simple syntax for common tasks
run efficiently. I've actually heard C programmers complain
about the infinite oportunities that the language presents to
the programmer for optimization. They are a distraction.
When I learned Lisp, the system required some 35K words
(about 150 KB, since a word was 36 bits) - thus a resource
hog. And, it was considered "slow" - since it was "interpreted".
These days, I'd never use it - except that my text editor has it
built in, and I need to customize it from time to time. And,
instead of sharing a 0.2 MIPS & 700 KB RAM system with
35 other users, I have multiple 100+ MIPS & 128 MB RAM
systems to myself. Of course, even this is entirely consumed
by my screen saver - which requires over 13 MB RAM
and must periodically connect to the internet to get more data.
Golly.
I used to think that Lisp was hard to read. But now that I use
Perl, Lisp is no big deal.
"Stephen Uitti" <······@ford.com> writes:
> > In fact, tight
> > loops of array access operations are within 20% of C speed, and in
> > special cases where atomic array loops (i.e. with no chance of a gc
> > moving the array) where the tag offset is precalculated, the access
> > can be 20% faster than C.
> You mean 20% faster than - A:
> for (i = 0; i < ARRAYSIZE; i++) {
> array[i]++;}
> or 20% faster than - B:
> for (i = 0; i < ARRAYSIZE * sizeof(typeOfArray); i += sizeof(typeOfArray)) {
> array[i]++;}
He means as fast as
array = array + 1
with the addition that the above is trivially parallelizable by currently
available compilers without more intervention by the user than specifying
an appropriate compiler flag - sadly enough, after all these decades of
development, compilers _still_ can't read my mind 8-|.
You get to say what language the one-liner is in.
> There is value in making the simple syntax for common tasks
> run efficiently. I've actually heard C programmers complain
> about the infinite oportunities that the language presents to
> the programmer for optimization. They are a distraction.
Indeed.
Jan
Jan Vorbrueggen <···@mailhost.neuroinformatik.ruhr-uni-bochum.de> writes:
> "Stephen Uitti" <······@ford.com> writes:
>
> > > In fact, tight
> > > loops of array access operations are within 20% of C speed, and in
> > > special cases where atomic array loops (i.e. with no chance of a gc
> > > moving the array) where the tag offset is precalculated, the access
> > > can be 20% faster than C.
> > You mean 20% faster than - A:
> > for (i = 0; i < ARRAYSIZE; i++) {
> > array[i]++;}
> > or 20% faster than - B:
> > for (i = 0; i < ARRAYSIZE * sizeof(typeOfArray); i += sizeof(typeOfArray)) {
> > array[i]++;}
>
> He means as fast as
>
> array = array + 1
>
> with the addition that the above is trivially parallelizable by currently
> available compilers without more intervention by the user than specifying
> an appropriate compiler flag - sadly enough, after all these decades of
> development, compilers _still_ can't read my mind 8-|.
>
> You get to say what language the one-liner is in.
Which one?
BTW, in Common Lisp you can do
(map-into myarr #'1+ myarr)
Regards,
--
Nils Goesche
"Don't ask for whom the <CTRL-G> tolls."
|> Lieven Marchand <···@bewoner.dma.be> writes:
|> I still believe that the wisest thing that the lisp community can ask
|> of hardware and operating system vendors is fast user-level trap
|> capability; with fewer registers saved and fewer cycles between the
|> trap and the return from the trap. Such a feature would be much more
|> likely to be widely used than cdr-coding, even in lisp systems.
Oh Boy, would I like to second that one.
Not just the Lisp community, but other (less well known)
language areas too. I do a lot of work in persistent
systems, and we love to trap pointer deferences for all
sorts of reasons. Most systems do this in software, but
some try using traps, either through VM level protection
or, on some, systems illegal addresses (in particlar systems
like the Alpha where stuffing a 1 into the LSB guarantees
a trap.)
The big killer in any trap based system is the enourmous
cost of making the trap traverse the OS. Some of the
earlier Sparc designs were quite brutally expensive.
The MIPS designs could take user level traps, but no OS
allowed access to the the feature.
Lastly I think CDR coding, cute as it is misses a more useful
use for a tag bit. Differentiating pointers from scalar
data. If you are designing any object based language system
you need to collect garbage. Being able to find pointers
really quickly is very useful. Those languages that do not
provide for a cannonical object format (Java being the most
recent offender) would really benefit. Of course changes
to the laguage spec can eliminate this problem. Won't happen
of course. If Java becomes the force some pundits predict
maybe we will see some real hardware support - but since
the language support system is mostly unable to make use
of such support in its current spec, we have a chicken
and egg problem.
However, once you have one tag bit, might as well have them all.
Francis Vaughan
·······@cs.adelaide.edu.au (Francis Vaughan) writes:
> |> I still believe that the wisest thing that the lisp community can ask
> |> of hardware and operating system vendors is fast user-level trap
> |> capability; with fewer registers saved and fewer cycles between the
> |> trap and the return from the trap. Such a feature would be much more
> |> likely to be widely used than cdr-coding, even in lisp systems.
> The big killer in any trap based system is the enourmous
> cost of making the trap traverse the OS.
So would you consider adding a PAL call to your Alpha for the purpose?
Jan
In article <··············@localhost.localdomain>, ···@bewoner.dma.be
says...
> Thanks for reminding me. Most of my stuff broke. But the mess was even
> worse: they had to put in mode bits to allow older programs to run
> unchanged. In a system like VM you ended up with two mode bits (EC/BC
> and XA or something like that? I've forgotten most of this stuff and I
> can't find my yellow booklet) and programs that only ran with one
> combination of them. It was very nice to have eight bits to work with
> with each pointer. Perhaps architecture designers should reconsider
> giving user programs some bits in the address space.
This is not an issue for architecture designers anymore, because
you can use paging to remap the addresses the way you wish. This
will only work with mid to top bits of course.
You can take linux and do the remapping the way you wish if it's
really that important.
Miha
Can't resist a discussion about old IBM 'frames and lisp. I did my lisp
homework assignments on line-at-a-time TSO :-)
If I remember correctly, back around 1980 or so, if you asked an IBM
engineer why they "needed" to move past 24-bit addressing to 31-bit, the
reason given was unrelated to application programming. That is, noone
needed to write programs which required that much memory, except for one
very special group of programmers.....the programmers who wrote IBM's
database management systems and operating systems.
Along with the big, new address spaces (actually just before them) came
additions to the instruction-set which permitted and protected the ability
to switch virtual storage context from one virtual address-space to
another (note that this is orthogonal to the ability to switch from
"kernel-mode" to "user-mode"). In practical terms, this permitted the OS
itself to spread ITS OWN data-structures across multiple address spaces in
order to circumvent the 16MB architectural limit. DBMS's did the same
thing.
The design and performance "win" in both cases was to permit the OS's
paging routines and the firmware's virtual-storage enhancements, which
were highly-optimized, to manage the buffering of data and data-structures
in real-memory. The "applications", ie. the bulk of the OS and DBMS, were
thereby relieved of these duties: coding and design were simplified.
I should mention that due to previous architecture/design choices, not all
of the older 16MB address space was available to the OS and/or DBMS
(and/or application) for exclusive, private use. Thus the effective
available virtual storage was less than 16MB, sometimes considerably less,
dependent on local OS changes as well. Even more reason to enlarge it and
to permit access to multiple virtual address spaces.
Any IBM folks want to chime-in here? Are all of the System/370 hands
extinct already? I haven't touched a 'frame in 9 years (don't miss
them either :-))
* Nick Geovanis The data were not anything that would lead anybody
| IT Computing Svcs to conclude the Fed would do anything different
| Northwestern Univ than what they already believed they were likely
| ··········@nwu.edu to do. - Chief Stock Market Analyst, 12/14/2000
+-------------------> Cantor Fitzgerald and Co.
From: Anne & Lynn Wheeler
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <upuiu4ky1.fsf@earthlink.net>
Nicholas Geovanis <·······@merle.acns.nwu.edu> writes:
> I should mention that due to previous architecture/design choices, not all
> of the older 16MB address space was available to the OS and/or DBMS
> (and/or application) for exclusive, private use. Thus the effective
> available virtual storage was less than 16MB, sometimes considerably less,
> dependent on local OS changes as well. Even more reason to enlarge it and
> to permit access to multiple virtual address spaces.
>
> Any IBM folks want to chime-in here? Are all of the System/370 hands
> extinct already? I haven't touched a 'frame in 9 years (don't miss
> them either :-))
a major MVS issue (in 24bit addressing land) was that it had inherited
from SVS, MVT, PCP (back to early '60s), etc a paradigm/design where
the supervisor occupied the same address space as the application
programs. This was slightly mitigated in the SVS->MVS (in the early
'70s) where the restriction that all applications programs and all
supervisor functions occupy the same 24-bit address space was slightly
lifted (single virtual storage ... became multiple virtual storage ...
where there was a 16mbyte address space for each application ... with
the MVS supervisor and various subsystem components residing in each
one).
However, by the late-70s having all supervisor functions in the same
address space as the application along with various supervisor
subsystem requirements were again starting to serverely strees the
24bit address limit.
while some of the MVS gurus might have believed that they were the
biggest address space hogs in the world, some MVS installations were
having difficulty leaving even 6mbytes (of the 16mbytes) available to
application program. There were actual applications in the '70s that
demonstrated large address space appetites. Some of these were large
database transaction subsystems that had to exist in the tiny space
left in the 16mbytes space after the MVS supervisor and subsystem
requirements were met.
In the initial port of apl/360 to cms/apl ... the apl workspace
limited was opened up from typically 32k-64k bytes to just under
16mbytes. There were a number of applications that actually took
advanage of the increased workspace size.
One of those were the HONE service. This was the service in the '70s
and '80s that supported world-wide sales, marketing, hdqtrs, and field
support operations. One example, starting sometime in the mid '70s,
IBM mainframe orders became so complex that it couldn't be done
manually, a HONE application was needed to fill-in the order. Another
big use of HONE was for economic planning & modeling ... much of the
"what-if" processes done today on PC spreadsheets were performed in
APL.
In '77, the US HONE operations were consolidated in a single location
in california with, what was at the time believed to be the largest
single-system image operation in the world (cluster of SMP processors
sharing large disk farm). In '78/'79, the single-system image was
replicated in dallas and boulder providing disaster survivability
support (in case of national disaster, like an earthquake in
cal.). This was in addition to various HONE clones that resided in
numerous countries around the world.
Almost the entire HONE "experience" was delivered first on cms/apl and
then later on apl/cms.
random refs:
http://www.garlic.com/~lynn/2000f.html#62
http://www.garlic.com/~lynn/2000f.html#30
http://www.garlic.com/~lynn/2000.html#75
http://www.garlic.com/~lynn/99.html#112
--
Anne & Lynn Wheeler | ····@garlic.com - http://www.garlic.com/~lynn/
From: Anne & Lynn Wheeler
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <ulmti4iqh.fsf@earthlink.net>
... oh yes, and a somewhat separate issue for the 370 ... in addition
to virtual 24bit address was the issue of real 24bit address. With
sufficient concurrent applications it was possible to start to stress
the 16mbyte real storage limits (and possibly precipitate page
thrashing).
many 3033 shops in late '70s (basically a souped up 370/168) started
seeing these real-storage constraint problems.
the issue was how to address the problem.
CCW (i.e. i/o transfer) already supported 31bit real address with
IDALs.
To get some slight relief, the 3033 introduced a hack for getting more
than 16mbyte real storage. The 370 pagetable entry is 16bits, 12bits
for specifying real page number (when combined with 12bit, 4k pages,
yields 24bit addressing), an invalid bit, a protection bit, and two
unused bits ... something like
NNNNNNNNNNNNPxxI
where "xx" are the unused/undefined bits. The 3033 hack was to use the
two "xx" bits and prefix them to the 12bit page number to allow
addressing up to 2**14 real pages ... or 2**26, 64mbytes of real
storage. Executable code was still limited to 24bit virtual addresses
but it was possible to allocate virtual pages in real storage above
the 24bit line by setting the appropriate bits in the page table
entry. And of course, the standard 370 CCW IDALs already had 31bits
available for addressing real storage in i/o operations.
cross-memory services was also introduced with the 3033. in an attempt
to help get some of the supervisor subsystem functions out of the same
address space as the application (at least get things to the point
where maybe a whole virtual 8mbytes was available to applications)
... and not to have a significant downside impact on the MVS
"address-pointer" parameter paradigm, these supervisor subsystem
functions had to reside in their own address space while still
directly supporting services requiring addressing of the application
virtual address space. cross-memory services introduced new addressing
modes allowing instructions to address virtual storage different than
the virtual address space that they were executing in.
random refs:
http://www.garlic.com/~lynn/99.html#99
http://www.garlic.com/~lynn/99.html#7
http://www.garlic.com/~lynn/2000e.html#57
http://www.garlic.com/~lynn/2000g.html#11
--
Anne & Lynn Wheeler | ····@garlic.com - http://www.garlic.com/~lynn/
From: Anne & Lynn Wheeler
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <uhf464hxz.fsf@earthlink.net>
Anne & Lynn Wheeler <····@garlic.com> writes:
> http://www.garlic.com/~lynn/99.html#99
oops finger slip, that should have been:
http://www.garlic.com/~lynn/99.html#190
giving the 3033 announce & ship dates
--
Anne & Lynn Wheeler | ····@garlic.com - http://www.garlic.com/~lynn/
In article <·············@beta.franz.com>,
Duane Rettig <·····@franz.com> wrote:
>······@best.com (Chuck Fry) writes:
>
>> And with 64 bits to play with, who's to say we can't spare a couple for
>> CDR-coding?
>
>Let's not make this mistake again. When IBM (and Amdahl) created their XA
>architectures for the 370 compatibles (extending the address range from 24
>bits to 32 bits), several programs suddenly broke. These programs, which
>used the upper 8 bits of address words for tags and flags, assumed (correctly,
>but without foresight) that these bits were available for use. However,
>whether these programs were justified in the original designs or not, it
>would be safe for me to venture a guess that _none_ of them exist in their
>previous design form, and that all have been either rewritten or scrapped.
>[snip]
>--
>Duane Rettig Franz Inc. http://www.franz.com/ (www)
>1995 University Ave Suite 275 Berkeley, CA 94704
>Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
A few of those 24-bit programs still exist, and they still run! Only those
programs that wanted to exploit the wonders of 31-bit mode were exposed, and
indeed those who coded without foresight were doomed to continue to play in
their little 24-bit corner. But they didn't break -- IBM takes backwards
compatibility seriously (for user-level programs, that is). They simply
couldn't automatically exploit new features.
There was the following problem, however. For the most part 31-bit mode
was not very different, so some programmers were lulled into a false
sense of security and thought they could simply run in 31-bit mode.
Those are the programs that broke when subtle differences became fatal.
Technically, the architecture supports both 24-bit (almost 360-compatible)
and 31-bit mode, with user-level instructions to switch mode. The default
settings for loaders are such that programs which don't declare any special
properties will run in 24-bit mode, so those dusty old object decks will
still run. Modern compilers (and assembler programmers) flag their code
as 31-bit, of course.
The newly-announced z-series (64-bit S/390) adds a third mode (24-31-64),
and old 24-bit programs are *still* supported. Current 31-bit programs
will run just fine on z-machines. Those that know better can use new
instructions to access the full 64-bit registers, or enter 64-bit mode
and explore as large an address space as the control program will give
them; those that don't will not gain from the new capabilities, but they
will not lose anything either. THAT's compatibility.
Michel.
····@watson.ibm.com (hack) writes:
> In article <·············@beta.franz.com>,
> Duane Rettig <·····@franz.com> wrote:
> >······@best.com (Chuck Fry) writes:
> >
> >> And with 64 bits to play with, who's to say we can't spare a couple for
> >> CDR-coding?
> >
> >Let's not make this mistake again. When IBM (and Amdahl) created their XA
> >architectures for the 370 compatibles (extending the address range from 24
> >bits to 32 bits), several programs suddenly broke. These programs, which
> >used the upper 8 bits of address words for tags and flags, assumed (correctly,
> >but without foresight) that these bits were available for use. However,
> >whether these programs were justified in the original designs or not, it
> >would be safe for me to venture a guess that _none_ of them exist in their
> >previous design form, and that all have been either rewritten or scrapped.
> >[snip]
>
> A few of those 24-bit programs still exist, and they still run! Only those
> programs that wanted to exploit the wonders of 31-bit mode were exposed, and
> indeed those who coded without foresight were doomed to continue to play in
> their little 24-bit corner. But they didn't break -- IBM takes backwards
> compatibility seriously (for user-level programs, that is). They simply
> couldn't automatically exploit new features.
I stand corrected, if there are 24-bit programs out there. (and I also
stand corrected for talking about 32 bits instead of 31 bits - forgot
about that little difference :-) I am also all for backward
compatibility. However, I can say with certainty (almost by definition)
that the only reason why those 24-bit programs could stay 24-bit was
because they were not _forced_ to go to 31 bits. This may seem like
circular reasoning, but it provides the preparation to point out
something you missed below:
> There was the following problem, however. For the most part 31-bit mode
> was not very different, so some programmers were lulled into a false
> sense of security and thought they could simply run in 31-bit mode.
> Those are the programs that broke when subtle differences became fatal.
What you call a subtle difference is certainly not so subtle. One
criticism that Lisp as a language has had to bear is its tendency to
be large (though nowadays it tends to be one of the smaller of
languages, comparatively). However, the very fact that lisp is able
to handle large problem spaces well has always allowed it to be used
for increasingly larger projects. Currently, we have customers,
especially in the CAD realm, whose data spaces push beyond the normal
2 to 3 Gb address spaces allowed by 32 bit systems, thus causing a
need to move toward usage of 64-bit addressing. Certainly it was
long ago that these problem sets broke through the now seemingly
pitiful virtual address range of 16 Mb, which is what 24 bits
of addressing provides.
So it was not necessarily a false sense of security in an optional
move, but instead a non-scalable design in an ever-growing need for
space, which caused the need for at least some programs to move to
31-bit mode, and which thus broke. I suspect that there were at
least some programs besides Lisps which broke in this way, but I
don't myself know that.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: ······@nospam-cs.wisc.edu
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <916etp$ore@spool.cs.wisc.edu>
In comp.arch Duane Rettig <·····@franz.com> wrote:
> languages, comparatively). However, the very fact that lisp is able
> to handle large problem spaces well has always allowed it to be used
> for increasingly larger projects. Currently, we have customers,
> especially in the CAD realm, whose data spaces push beyond the normal
> 2 to 3 Gb address spaces allowed by 32 bit systems, thus causing a
> need to move toward usage of 64-bit addressing. Certainly it was
Out of curiosity: what kinds of large commercial
applications use LISP? Do they use it as an embedded
scripting engine, or to write the application proper?
I guess you could ask the same question about functional
languages in general. All I am aware about is Ericcson's
use of Erlang. Phil Wadler has a web page talking
about some others:
http://www.cs.bell-labs.com/who/wadler/realworld/
Manoj
······@nospam-cs.wisc.edu writes:
> In comp.arch Duane Rettig <·····@franz.com> wrote:
> > languages, comparatively). However, the very fact that lisp is able
> > to handle large problem spaces well has always allowed it to be used
> > for increasingly larger projects. Currently, we have customers,
> > especially in the CAD realm, whose data spaces push beyond the normal
> > 2 to 3 Gb address spaces allowed by 32 bit systems, thus causing a
> > need to move toward usage of 64-bit addressing. Certainly it was
>
> Out of curiosity: what kinds of large commercial
> applications use LISP? Do they use it as an embedded
> scripting engine, or to write the application proper?
Both are possible, as well as using lisp as glue to hold
together systems with subsystems written in different languages.
However, the applications I refer to, which _must_ use lisp for
its ability to handle large problem sets, tend to be largely
written in lisp.
> I guess you could ask the same question about functional
> languages in general. All I am aware about is Ericcson's
> use of Erlang. Phil Wadler has a web page talking
> about some others:
> http://www.cs.bell-labs.com/who/wadler/realworld/
Looks like a good list, and I notice that Lisp is a major
contributor in that list. However, lisp is not a purely
functional language, per se; I would characterize it as very
general purpose, with good support for functional, object-oriented,
dynamic, etc. programming.
You can also check out our website (below) under "Success Stories",
and also http://www.lisp.org, for more info.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: ······@nospam-cs.wisc.edu
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91886h$skj@spool.cs.wisc.edu>
In comp.arch Duane Rettig <·····@franz.com> wrote:
> ······@nospam-cs.wisc.edu writes:
>> In comp.arch Duane Rettig <·····@franz.com> wrote:
>> > languages, comparatively). However, the very fact that lisp is able
>> > to handle large problem spaces well has always allowed it to be used
>> > for increasingly larger projects. Currently, we have customers,
>> > especially in the CAD realm, whose data spaces push beyond the normal
>> > 2 to 3 Gb address spaces allowed by 32 bit systems, thus causing a
>> > need to move toward usage of 64-bit addressing. Certainly it was
>>
>> Out of curiosity: what kinds of large commercial
>> applications use LISP? Do they use it as an embedded
>> scripting engine, or to write the application proper?
> Both are possible, as well as using lisp as glue to hold
> together systems with subsystems written in different languages.
> However, the applications I refer to, which _must_ use lisp for
> its ability to handle large problem sets, tend to be largely
> written in lisp.
I guess I'm trying to figure out why LISP is unique
in its ability to handle these large problem sets.
What is it about these data sets that prevents use
of C or C++, and *requires* LISP?
I would have thought LISP would be more in demand
because of the dynamic nature of the language,
quick prototyping capability and so on ...
Manoj
PS: Note that I'm not a big fan of C or C++, I believe
that the world should switch to SML and Python as
its primary languages :)
······@nospam-cs.wisc.edu writes:
> In comp.arch Duane Rettig <·····@franz.com> wrote:
> > ······@nospam-cs.wisc.edu writes:
> >> In comp.arch Duane Rettig <·····@franz.com> wrote:
> >> > languages, comparatively). However, the very fact that lisp is able
> >> > to handle large problem spaces well has always allowed it to be used
> >> > for increasingly larger projects. Currently, we have customers,
> >> > especially in the CAD realm, whose data spaces push beyond the normal
> >> > 2 to 3 Gb address spaces allowed by 32 bit systems, thus causing a
> >> > need to move toward usage of 64-bit addressing. Certainly it was
> >>
> >> Out of curiosity: what kinds of large commercial
> >> applications use LISP? Do they use it as an embedded
> >> scripting engine, or to write the application proper?
>
> > Both are possible, as well as using lisp as glue to hold
> > together systems with subsystems written in different languages.
> > However, the applications I refer to, which _must_ use lisp for
> > its ability to handle large problem sets, tend to be largely
> > written in lisp.
>
> I guess I'm trying to figure out why LISP is unique
> in its ability to handle these large problem sets.
> What is it about these data sets that prevents use
> of C or C++, and *requires* LISP?
nothing *requires* Lisp, per se, as you can always write a common-lisp
in, for example, C and C++.
> I would have thought LISP would be more in demand
> because of the dynamic nature of the language,
> quick prototyping capability and so on ...
Lisp offers many things that help with a large project. i think the
most useful is that Lisp helps you retain information.
*) you can dump your entire state in a core and easily pick up right
where you left off.
*) you can dump Lisp structures for later recall simply by
readably-printing them. you can even include the functions for
processing the date in the data structure which you store to disk (an
object orientation move that C++ can only dream of).
*) no more lost pointers.
*) no tedious and error prone manual memory management. Lisp usually
has generational garbage collection which can avoid memory
fragmentation issues that C++ usually suffers from.
*) all values have types associated with them, therefore, the meanings
associated with values tend to be retained better.
*) first class functions allow a high level programming style.
*) macros let you warp the language toward your problem set. when the
language makes thing you want to do more natural, you do them without
having to concentrate on (and possibly get confused by) gratuitous
detail.
information lossage is probably my biggest problem in using computers
today. i make a file and then forget what it was supposed to do.
> Manoj
>
> PS: Note that I'm not a big fan of C or C++, I believe
> that the world should switch to SML and Python as
> its primary languages :)
>
--
J o h a n K u l l s t a m
[········@ne.mediaone.net]
sysengr
From: ······@nospam-cs.wisc.edu
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91bova$dfo@spool.cs.wisc.edu>
In comp.arch Johan Kullstam <········@ne.mediaone.net> wrote:
> ······@nospam-cs.wisc.edu writes:
>> In comp.arch Duane Rettig <·····@franz.com> wrote:
>> > ······@nospam-cs.wisc.edu writes:
>> >> Out of curiosity: what kinds of large commercial
>> >> applications use LISP? Do they use it as an embedded
>> >> scripting engine, or to write the application proper?
>>
>> > Both are possible, as well as using lisp as glue to hold
>> > together systems with subsystems written in different languages.
>> > However, the applications I refer to, which _must_ use lisp for
>> > its ability to handle large problem sets, tend to be largely
>> > written in lisp.
>>
>> I guess I'm trying to figure out why LISP is unique
>> in its ability to handle these large problem sets.
>> What is it about these data sets that prevents use
>> of C or C++, and *requires* LISP?
> nothing *requires* Lisp, per se, as you can always write a common-lisp
> in, for example, C and C++.
[ snipped lots of good stuff about LISP ]
Oh sure, I agree. But I'm curious as to what exactly
the original poster meant while writing "applications [...]
which _must_ use lisp for its ability to handle large
problem sets".
What is it about LISP that enables handling
large data sets? Is it just the fact that objects can be
easily serialized to storage and back (as you described
earlier)? Or the garbage collection? Or the uniformity
and simplicity of the data model (everything is a list)?
I guess it would be good if the original poster
described the applications in question and explained
how LISP helped in their implementation.
Manoj
······@nospam-cs.wisc.edu writes:
> In comp.arch Johan Kullstam <········@ne.mediaone.net> wrote:
> > ······@nospam-cs.wisc.edu writes:
> >> In comp.arch Duane Rettig <·····@franz.com> wrote:
> >> > ······@nospam-cs.wisc.edu writes:
> >> >> Out of curiosity: what kinds of large commercial
> >> >> applications use LISP? Do they use it as an embedded
> >> >> scripting engine, or to write the application proper?
> >>
> >> > Both are possible, as well as using lisp as glue to hold
> >> > together systems with subsystems written in different languages.
> >> > However, the applications I refer to, which _must_ use lisp for
> >> > its ability to handle large problem sets, tend to be largely
> >> > written in lisp.
> >>
> >> I guess I'm trying to figure out why LISP is unique
> >> in its ability to handle these large problem sets.
> >> What is it about these data sets that prevents use
> >> of C or C++, and *requires* LISP?
> > nothing *requires* Lisp, per se, as you can always write a common-lisp
> > in, for example, C and C++.
>
>
> [ snipped lots of good stuff about LISP ]
>
>
> Oh sure, I agree. But I'm curious as to what exactly
> the original poster meant while writing "applications [...]
> which _must_ use lisp for its ability to handle large
> problem sets".
I guess that would be me. Please recall that in that same article I
said:
You can also check out our website (below) under "Success Stories",
and also http://www.lisp.org, for more info.
I suppose that due to fear of entering into "advertising" mode,
I was not as clear as I should be. So I'll be more direct: Go
to http://www.franz.com, click on "Success Stories", and find out
why some of our customers themselves say that they could not have
built their apps without Lisp.
> What is it about LISP that enables handling
> large data sets? Is it just the fact that objects can be
> easily serialized to storage and back (as you described
> earlier)? Or the garbage collection? Or the uniformity
> and simplicity of the data model (everything is a list)?
Many of these are true, though everything is not a list, and the
data model is not any simpler than other languages. However, I
would characterize Lisp, and especially Common Lisp, as a language
that scales well because it is both general purpose _and_ is not
brittle.
That is, you can make changes dynamically to functions and classes
(yes, you can redefine classes on the fly, and the CLOS system
lazily updates the new class usage!). When errors in your program
inevitably occur, debug interactions allow you to find and fix
your bug _and_ retry portions of what has been done already (no
need to start from the beginning). You can compile and load files,
but there is no "link edit" phase, because of the dynamicity of lisp
functions, so you can define a function on the fly and run it
interpreted if you wish, or you can compile the function and it is
immediately available for efficient running, while you are in the
middle of running your application. Lisp also has introspection;
you can see as much or as little of the internal structure of your
program as you wish, depending on how you set it up; while debugging
your app, you are not left wondering what some string of bits really
represent, because the inspector tells you what they are.
> I guess it would be good if the original poster
> described the applications in question and explained
> how LISP helped in their implementation.
Once again, it is our customers that describe these things, some of
whose descriptions you can read for yourself.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In article <·············@beta.franz.com>, Duane Rettig <·····@franz.com> wrote:
> I suppose that due to fear of entering into "advertising" mode,
> I was not as clear as I should be. So I'll be more direct: Go
> to http://www.franz.com, click on "Success Stories", and find out
> why some of our customers themselves say that they could not have
> built their apps without Lisp.
>
> > What is it about LISP that enables handling
> > large data sets? Is it just the fact that objects can be
> > easily serialized to storage and back (as you described
> > earlier)? Or the garbage collection? Or the uniformity
> > and simplicity of the data model (everything is a list)?
>
> Many of these are true, though everything is not a list, and the
> data model is not any simpler than other languages. However, I
> would characterize Lisp, and especially Common Lisp, as a language
> that scales well because it is both general purpose _and_ is not
> brittle.
>
> That is, you can make changes dynamically to functions and classes
> (yes, you can redefine classes on the fly, and the CLOS system
> lazily updates the new class usage!). When errors in your program
> inevitably occur, debug interactions allow you to find and fix
> your bug _and_ retry portions of what has been done already (no
> need to start from the beginning). You can compile and load files,
> but there is no "link edit" phase, because of the dynamicity of lisp
> functions, so you can define a function on the fly and run it
> interpreted if you wish, or you can compile the function and it is
> immediately available for efficient running, while you are in the
> middle of running your application. Lisp also has introspection;
> you can see as much or as little of the internal structure of your
> program as you wish, depending on how you set it up; while debugging
> your app, you are not left wondering what some string of bits really
> represent, because the inspector tells you what they are.
>
> > I guess it would be good if the original poster
> > described the applications in question and explained
> > how LISP helped in their implementation.
>
> Once again, it is our customers that describe these things, some of
> whose descriptions you can read for yourself.
I don't want to mean about this, and I know pretty much nothing about
Lisp, so have no opinion, but the thing such lists never mention is the
customers who used Lisp and found it to be a terrible mistake for their
project.
From a world with which I am more familiar, I have spoken to various
friends (not C bigots, people who've used many languages) involved in
projects with Java where they believe the use of Java was a terrible
mistake, that the resultant apps were horrendously slow memory hogs---but
I strongly suspect you want see any mention of them at a Sun web site.
Maynard
········@ricochet.net (Maynard Handley) writes:
> In article <·············@beta.franz.com>, Duane Rettig <·····@franz.com> wrote:
> > > I guess it would be good if the original poster
> > > described the applications in question and explained
> > > how LISP helped in their implementation.
> >
> > Once again, it is our customers that describe these things, some of
> > whose descriptions you can read for yourself.
>
> I don't want to mean about this, and I know pretty much nothing about
> Lisp, so have no opinion, but the thing such lists never mention is the
> customers who used Lisp and found it to be a terrible mistake for their
> project.
Of course not. Why would we want to do that? :-)
Actually, the reason why I directed the original poster to our website
was because the original question was why people felt they _must_
use lisp. Because the question was a directed question, and not of the
form "what are the pros and cons ...", I felt justified in pointing
there, even though it is one-sided...
> From a world with which I am more familiar, I have spoken to various
> friends (not C bigots, people who've used many languages) involved in
> projects with Java where they believe the use of Java was a terrible
> mistake, that the resultant apps were horrendously slow memory hogs---but
> I strongly suspect you want see any mention of them at a Sun web site.
Yes, we get customers who come from both C++ and Java shops to
replace, either all or in part, their applications that either
took too long to finish or were not very maintainable.
The major reason why people moved away from Lisp is historical:
Lisp was always very general-purpose, but was especially good at
Artificial Intelligence, and was for a long time very closely
associated with AI. The Lisp companies rode the great AI wave
in the early 80's, when large corporations poured billions of
dollars into the AI hype that promised thinking machines in 10
years. When the promises turned out to be harder than originally
thought, the AI wave crashed, and lisp crashed with it because of
its association with AI. We refer to it as the AI Winter. Some
high-level managers who were burned by the AI hype actually kill
any project with either the name AI or Lisp in it (that's why you
don't even see or hear the term AI much anymore; most of the more
successful branches of AI (e.g. speech recognition, expert systems)
have distanced themselves from the name. And many distanced
themselves from Lisp as well, trying to make a go of their projects
in C, C++, or Java. Some have succeeded. Others, not. Some have
come back to Lisp, and not told anyone, for the twofold reason that
it might be a negative in their marketing strategy to mention the
L word, and also because it became a competitive advantage to not
mention how their time-to-market is so quick.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
>>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
> L word, and also because it became a competitive advantage to not
> mention how their time-to-market is so quick.
I'd be surprised if it took their competitors more than a few minutes
to figure out what compiler was used to build the application.
Stefan
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjd7esej28.fsf@tfeb.org>
[Note I've set followups to comp.lang.lisp: I don't think this article
is really on topic for comp.arch any more]
Duane Rettig <·····@franz.com> writes:
> The major reason why people moved away from Lisp is historical:
>
> [ and the rest which I mostly agree with elided ]
I think there are some other reasons, one major one which I've come
across being that it's possible to write *amazingly* bad code in Lisp
and still have it kind of work. If you wrote C as badly you'd just
die of memory leaks &c before you got anywhere (in other words, before
you spent a lot of money), but in Lisp you can stumble on much
further, until you end up with a huge, mostly-working but entirely
unmaintainable system which ends up costing a huge amount to maintain
and/or reimplement. As well as the boring memory management stuff
(which should be mostly true of Java as well), Lisp has such unmatched
linguistic flexibility that it's possible to shoot yourself in the
foot in ways many other languages can't even approach. And lisp
programmers tend to come from highly academic backgrounds where
programming skills are despised, thus further encouraging these kinds
of disasters.
The solution to these problems is to run projects properly, but of
course many weren't. Sometimes, unfortunately, the followup projects
*were* run properly thus making Lisp look bad.
--tim (working on such a project...)
On 16 Dec 2000 10:50:39 +0000, Tim Bradshaw <···@tfeb.org> wrote:
>I think there are some other reasons, one major one which I've come
>across being that it's possible to write *amazingly* bad code in Lisp
>and still have it kind of work.
C++ allows the newbie to easily write buggy code while Lisp, just like
visual basic, let's the newbie write safe but extremely slow code. I guess
this isn't a reason for CL's lack of popularity (if compared to VB), and Duane
is right: Lisp is still associated with the failure of most of the AI
companies of the 80s.
Or perhaps you're right, and the *real* problem is that we don't have
any MS Visual Lisp... ;-)
//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------
Fernando Rodr�guez <·······@must.die> writes:
> On 16 Dec 2000 10:50:39 +0000, Tim Bradshaw <···@tfeb.org> wrote:
>
>
> >I think there are some other reasons, one major one which I've come
> >across being that it's possible to write *amazingly* bad code in Lisp
> >and still have it kind of work.
>
>
> C++ allows the newbie to easily write buggy code while Lisp, just like
> visual basic, let's the newbie write safe but extremely slow code.
And we all know that performance is far more important than
correctness.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjelzahtcq.fsf@tfeb.org>
······@nospam-cs.wisc.edu writes:
>
> What is it about LISP that enables handling
> large data sets? Is it just the fact that objects can be
> easily serialized to storage and back (as you described
> earlier)? Or the garbage collection? Or the uniformity
> and simplicity of the data model (everything is a list)?
>
Everything is *not* a list. It was in 1956 but we've moved on since
then.
--tim
* ······@nospam-cs.wisc.edu
| Or the uniformity and simplicity of the data model (everything is a list)?
It would be helpful if you could explain why you believe this "everything
is a list" part to be true and how you came to believe it. (Hint: It has
actually _never_ been true, and it mystifies Lisp people that so many
people who come to Lisp hold this myth. If we could only track it down
and find ways to replace it with something less false...)
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
"Erik Naggum" <····@naggum.net> wrote in message
·····················@naggum.net...
> * ······@nospam-cs.wisc.edu
> | Or the uniformity and simplicity of the data model (everything is a
list)?
>
> It would be helpful if you could explain why you believe this
"everything
> is a list" part to be true and how you came to believe it. (Hint: It
has
> actually _never_ been true, and it mystifies Lisp people that so many
> people who come to Lisp hold this myth. If we could only track it down
> and find ways to replace it with something less false...)
It's because, to a level zero - or 10,000 foot approximation, everything
in Lisp *is* a list. That's what they taught in the first week of my
lisp/Algol
class. Arrays weren't even mentioned for several weeks. Naturally, when
my friends asked me what was so strange about the language, I'd say
"Lisp stands for LISt Processor. Everything is a list". Since my friends
were only interested in the summary, this answer was correct.
Mechanical engineers aren't taught Einstienian mechanics - they're taught
Newtonian mechanics. Newtonian mechanics isn't wrong - it's just
less accurate. When the book says to over design by a factor of six,
what difference does it make?
Perhaps Lisp needs a better executive summary.
Why do people use Perl and Java instead of Lisp and Forth?
"Stephen Uitti" <······@ford.com> writes:
> "Erik Naggum" <····@naggum.net> wrote in message
> ·····················@naggum.net...
> > * ······@nospam-cs.wisc.edu
> > | Or the uniformity and simplicity of the data model (everything is a
> list)?
> >
> > It would be helpful if you could explain why you believe this
> "everything
> > is a list" part to be true and how you came to believe it. (Hint: It
> has
> > actually _never_ been true, and it mystifies Lisp people that so many
> > people who come to Lisp hold this myth. If we could only track it down
> > and find ways to replace it with something less false...)
>
> It's because, to a level zero - or 10,000 foot approximation, everything
> in Lisp *is* a list. That's what they taught in the first week of my
> lisp/Algol class. Arrays weren't even mentioned for several weeks.
==^^^^^^^^^^
This should have given you a clue as to how bad the class would be.
Your professor either didn't know lisp, or didn't care to really
teach it. And spouting such inaccuracies in classes propagates
misinformation throuout the industry.
In fact, if we look at a lisp object count, there tends to be more
bytes of arrays than conses (list elements), although there are more
conses in number - Note the table in the middle of this output:
cl-user(1): (room t)
area area address(bytes) cons other bytes
# type 8 bytes each
(free:used) (free:used)
Top #x3063a000
New #x30534000(1073152) 95:3981 714976:273680
New #x3042e000(1073152) ----- -----
0 Old #x30000cc8(4379448) 369:76056 2100512:1659240
Root pages: 30
Lisp heap: #x30000000 pos: #x3063a000 resrve: 17842176
C heap: #x54000000 pos: #x54006000 resrve: 1024000
code type items bytes
96: (simple-array t) 5372 679160 26.8%
1: cons 78283 626264 24.8%
8: function 8142 492832 19.5%
7: symbol 15339 368136 14.6%
18: bignum 2651 153856 6.1%
101: (simple-array character) 650 63760 2.5%
12: standard-instance 3333 53328 2.1%
9: closure 2126 36152 1.4%
98: (simple-array (unsigned-byte 8)) 11 21064 0.8%
15: structure 422 13616 0.5%
100: (simple-array (unsigned-byte 32)) 8 9752 0.4%
108: (simple-array code) 20 4088 0.2%
10: hash-table 95 3040 0.1%
17: double-float 103 1648 0.1%
111: (simple-array foreign) 46 1096 0.0%
16: single-float 116 928 0.0%
97: (simple-array bit) 10 224 0.0%
64: (array t) 9 216 0.0%
20: complex 11 176 0.0%
11: readtable 6 96 0.0%
107: (simple-array (signed-byte 32)) 1 88 0.0%
19: ratio 4 64 0.0%
13: sysvector 2 32 0.0%
69: (array character) 1 24 0.0%
total bytes = 2529640
aclmalloc arena:
max size free bytes used bytes total
112 3472 112 3584
496 3968 0 3968
1008 4032 0 4032
2032 4064 0 4064
4080 0 8160 8160
total bytes: 15536 8272 23808
cl-user(2):
> Naturally, when
> my friends asked me what was so strange about the language, I'd say
> "Lisp stands for LISt Processor. Everything is a list". Since my friends
> were only interested in the summary, this answer was correct.
I disagree; the second part of your answer was incorrect, and you then
succeeded in propagating the inaccuracy to your friends.
Another "executive summary" (as you say) that is totally incorrect but
which often gets propagated is "lisp is slow because it is interpreted".
Here's my theory on why these inaccurate "executive summaries" get
taught and propagated:
All languages have common aspects, and they have aspects that are unique.
A Venn diagram of all languages would be fairly messy, and there would be
very few one-color areas in such a diagram, because most languages must
have at least enough functionality to be useful, either in teaching or in
the industry, or both. Lisp is a hard language to describe, because it is
so general-purpose, and has so many features that other languages have.
So to teach Lisp, many will teach other languages first, and then teach
only the differences between Lisp and those first languages. This
inevitably leads to the kind of teaching you received, especially if your
professors were themselves taught this way. For example, Why teach arrays
in lisp; they are effectively the same as in the other languages we taught!
Or, Lisp is good at recursion, so let's teach the recursion concept in lisp
(this tends to lead one to the assumption that lisp is only good at recusrion,
and can't iterate). Or, Other languages are compiled; let's show the
students a Lisp interpreter (the fact that Lisp code can be interpreted
does not preclude extremely efficient compilation, but teaching about
interpreters in lisp can lead one to incorrectly conclude that lisp is
only interpreted; incorrect for over 20 years).
> Mechanical engineers aren't taught Einstienian mechanics - they're taught
> Newtonian mechanics. Newtonian mechanics isn't wrong - it's just
> less accurate. When the book says to over design by a factor of six,
> what difference does it make?
Don't know mechanics, but perhaps better design accuracy would lead to
higher profits. It would seem like a useful difference to me.
> Perhaps Lisp needs a better executive summary.
Yes, it does. You shouldn't use the one you are currently using.
> Why do people use Perl and Java instead of Lisp and Forth?
At the face of it, your question is incorrect, because people do
use Lisp (Forth, too, presumably, but I won't comment on that).
I assume, though, that your question was really meant to be
qualitative, as to why _more_ people use the former languages
than the latter. If that is really the question, then I really
have no idea. However, you should put some perspective on the
question - the question used to be "Why do so many more people
use C++ than Lisp?", but we hardly ever hear this question
anymore. Any guesses as to the name of the next fad language
to be replaced in the beginning of this eternal question?
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
"Duane Rettig" <·····@franz.com> wrote in message
··················@beta.franz.com...
> "Stephen Uitti" <······@ford.com> writes:
> > Naturally, when
> > my friends asked me what was so strange about the language, I'd say
> > "Lisp stands for LISt Processor. Everything is a list". Since my friends
> > were only interested in the summary, this answer was correct.
>
> I disagree; the second part of your answer was incorrect, and you then
> succeeded in propagating the inaccuracy to your friends.
It's a summary. It's no more inaccurate than saying "You need to
breathe to live." Sure it's more complicated than that. And some
microbes don't breathe.
For what it's worth, the subject of Lisp hasn't really come up much
in the last, uhm, 23 years. I can't imagine that I've commited any real
crime against the language. Except maybe by silence. I did not, for
example, bitch that EMACS changed from TECO (to some extent
a Forth like stack language) to Lisp as the extension language.
And at least I didn't say "The manuals suck", based on the MacLisp
manuals of the 70s. (I hope this has been fixed.)
> All languages have common aspects, and they have aspects that are unique.
Part of the executive summary is related to differential marketing.
What incremental value does product X have compared to its competitors?
Perhaps the level one description of Lisp would include the words
"general purpose", "automatic typed", "garbage collecting"
(or "automatic memory management", or "implicit memory management").
Should Lisp be considered "Object Oriented"? In the world
of differential marketing, even if the product doesn't change,
the competition does.
Though Forte' and Java have list objects, I wouldn't say that
the languages are very good at lists. So, a powerful feature
of Lisp remains list processing. Honorable mention could go
to other features.
> Another "executive summary" (as you say) that is totally incorrect but
> which often gets propagated is "lisp is slow because it is interpreted".
In 1977, the Lisp environment I used was not compiled.
I was not aware of a compiler for Lisp until 1987, when I bought
my Mac - not that I was looking for one. But I never considered it
slow due to interpretation - it was a resource hog (150 KB!), pretty
much a moot point these days.
Lisp also suffers from being twenty or thirty years ahead of its time.
Not that Lisp environments couldn't be made small. In 1980, a
friend had written a Lisp entirely in 8080 assembly language. It
was quite a performer on hardware available to an individual at
the time (I owned such a machine). I don't know if it was ever
marketed as a product, so I doubt it had much impact.
> Here's my theory on why these inaccurate "executive summaries" get
> taught and propagated:
>
> For example, Why teach arrays
> in lisp; they are effectively the same as in the other languages we
taught!
The Lisp course I took in '77 concentrated on how the language
was implemented. It was never intended to show how to write
production code. Correct. No arrays.
> Or, Lisp is good at recursion, so let's teach the recursion concept in
lisp
I recall that one of the assignments concentrated on building
meaningful data that could be used to build other meaningful
data. Something a little more sophisticated than "Guess the
Animal", but not attempting AI.
>Or, Other languages are compiled; let's show the students a Lisp
>interpreter (the fact that Lisp code can be interpreted does not
>preclude extremely efficient compilation, but teaching about
>interpreters in lisp can lead one to incorrectly conclude that lisp is
>only interpreted; incorrect for over 20 years).
Executive summaries, like prejudice, do not change quickly.
This is likely what irritates so much. Lisp has been around
forever (it predates Fortran). Lisp has changed in that time.
Computers have changed. The problems that are considered
hard have changed. In particular, dynamic memory use
has become considerably more important. Explicit
memory management (C) is generally faster, but this is
increasingly less important (and for many programmers,
increasingly less true).
> > Perhaps Lisp needs a better executive summary.
>
> Yes, it does. You shouldn't use the one you are currently using.
Since I'm not the Lisp expert, perhaps someone closer to the industry
should generate one.
> > Mechanical engineers aren't taught Einstienian mechanics - they're
taught
> > Newtonian mechanics. Newtonian mechanics isn't wrong - it's just
> > less accurate. When the book says to over design by a factor of six,
> > what difference does it make?
>
> Don't know mechanics, but perhaps better design accuracy would lead to
> higher profits. It would seem like a useful difference to me.
Actually, I misspoke. MEs are taught Einstienian mechanics. However,
the level one summary is that you don't need it unless you are dealing
with the very small (sub atomic), the very fast (near light speed), or
down a deep gravity well (perhaps near the Sun). I imagine (without
proof) that these are topics of concern for at least some MEs. It isn't
clear for example that Pioneer needed it for navigation to Mercury.
> Any guesses as to the name of the next fad language
> to be replaced in the beginning of this eternal question?
One of the old quotes was "I don't know what language
we'll be using in 2000, but it will be called Fortran."
Of the thirty odd languages I've used, the ones I continue
using are the ones that are pretty easy to read, are
available, and are in demand for some task. Annoyingly,
programmers don't choose their own tools these days.
The winners at the moment are C and uhm...
r = read
a = available
d = demand
x = no
r a d C
x a d C++
x a d Perl
r x x BASIC
r a d VB (available on Windows only)
r x x Algol
r x x Pascal
r x x Fortran
x a x Forth
x x x Lisp
These are my own binary cutoffs. Your mileage may vary.
Languages seem to need two of the three to be viable.
Uhm, that's assuming that they don't have some misfeature.
For example, BASIC lacked block structure, local variables
and any kind of records for so long that when introduced,
they were non-standard. Fortran had its day, and
in 1977 it got block structure into the standard. Did it ever
have portable access to dynamic memory? If so, it was
too late. VB is currently viable. I don't use it as I don't
support Windows, and it isn't cross platform. It changes
much to fast for anything I'd want to accomplish. Expect
to have to rewrite everything every three years. This limits
you to what you can rewrite in three years. If Microsoft
decides it can't milk any more from VB, it's history.
I may be ready to use Lisp again. Maybe I'm getting old.
Lisp has demonstrated that I could write in it without having
to scrap my code instantly. An individual can produce a
working Lisp environment. So, it doesn't take 90% of the
computer industry language market to ensure availability,
cross platform.
"Stephen Uitti" <······@ford.com> writes:
> "Duane Rettig" <·····@franz.com> wrote in message
> ··················@beta.franz.com...
> > "Stephen Uitti" <······@ford.com> writes:
> > > Naturally, when
> > > my friends asked me what was so strange about the language, I'd say
> > > "Lisp stands for LISt Processor. Everything is a list". Since my friends
> > > were only interested in the summary, this answer was correct.
> >
> > I disagree; the second part of your answer was incorrect, and you then
> > succeeded in propagating the inaccuracy to your friends.
>
> It's a summary. It's no more inaccurate than saying "You need to
> breathe to live." Sure it's more complicated than that. And some
> microbes don't breathe.
The problem I have with the summary is its exclusivity. Saying "Lisp
has/uses lists" or even "Lisp needs lists" is _not_ the same as
saying "In lisp, everthing is a list", just as in your analogy saying
"You need to breathe to live", is much different than saying "You
only need to breathe to live" (the first statement is true because
it is not exclusive, but the second is not true.
> For what it's worth, the subject of Lisp hasn't really come up much
> in the last, uhm, 23 years. I can't imagine that I've commited any real
> crime against the language. Except maybe by silence. I did not, for
> example, bitch that EMACS changed from TECO (to some extent
> a Forth like stack language) to Lisp as the extension language.
I can well believe that the subject of lisp hasn't come up for you in
the last 23 years. There are some people for whom that is true. Most
programmers, though, who had anything to do with AI _at_ _least_ heard
about lisp during the AI heyday in the early '80s. And lisp never died;
it has have continuity for the past 40 years, even through ups and downs.
> And at least I didn't say "The manuals suck", based on the MacLisp
> manuals of the 70s. (I hope this has been fixed.)
Well, let's see - the lisps of today include Emacs lisp, Common
Lisp, and (arguably) many Scheme implementations. Nope, Maclisp
is not one of them. I know there is much Maclisp legacy code out
there, but most of it has been ported to Common Lisp. The Maclisp
manuals may suck, but it has very little to do with today. As for
today's lisps, you'll have to try them out and make your own
determination as to the state of their manuals...
> [ ... ]
> Lisp also suffers from being twenty or thirty years ahead of its time.
I agree here.
> >Or, Other languages are compiled; let's show the students a Lisp
> >interpreter (the fact that Lisp code can be interpreted does not
> >preclude extremely efficient compilation, but teaching about
> >interpreters in lisp can lead one to incorrectly conclude that lisp is
> >only interpreted; incorrect for over 20 years).
>
> Executive summaries, like prejudice, do not change quickly.
> This is likely what irritates so much. Lisp has been around
> forever (it predates Fortran).
Actually, no; Lisp is slightly younger than Fortran.
> Lisp has changed in that time.
Absolutely. That includes the time from your last lisp course
and now. You should check out the newer look of lisp.
> Computers have changed. The problems that are considered
> hard have changed. In particular, dynamic memory use
> has become considerably more important. Explicit
> memory management (C) is generally faster, but this is
> increasingly less important (and for many programmers,
> increasingly less true).
Actually, it has seldom been true; unless you are using a malloc/free
algorithm that sacrifices space for speed, the best-fit algorithms of
malloc() implementations and the coagulation algorithms of free()
implementations tend to slow it down to a point where many C
programmers are extremely surprised whgen they have it demonstrated to
them. Dynamic memory management has at least the potential for being
on the same par as explicit; sometimes slower, sometimes faster.
> > > Perhaps Lisp needs a better executive summary.
> >
> > Yes, it does. You shouldn't use the one you are currently using.
>
> Since I'm not the Lisp expert, perhaps someone closer to the industry
> should generate one.
You should look at the ones that are already available: go to
http://www.lisp.org/table/lisp.htm, and see an executive summary
(said by someone in the industry) along with other myth-busting
information.
> [ ... ]
> Of the thirty odd languages I've used, the ones I continue
> using are the ones that are pretty easy to read, are
> available, and are in demand for some task. Annoyingly,
> programmers don't choose their own tools these days.
> The winners at the moment are C and uhm...
>
> r = read
> a = available
> d = demand
> x = no
>
> r a d C
> x a d C++
> x a d Perl
> r x x BASIC
> r a d VB (available on Windows only)
> r x x Algol
> r x x Pascal
> r x x Fortran
> x a x Forth
> x x x Lisp
>
> These are my own binary cutoffs. Your mileage may vary.
> Languages seem to need two of the three to be viable.
Um, yes, mileage will vary.
Since the languages you list are Turing Complete (any
application could thus be ported to any of the languages)
the third column makes little sense to me except from a
marketing point-of-view, or an ease-of-use and competitive
standpoint. And Lisp wins in this area; one of the very
reasons why you don't hear about applications written in
Lisp is precisely because the vendor doesn't _want_ their
competitors to know what makes them successful.
The first column is entirely subjective and based on the
programmer. It is possible to create good (readable) programs
in all of the above languages, and it is possible to create
horrible programs in any of the above languages [some easier
that others].
For the second column, you are clearly missing some information.
Lisp is one of the most widely available languages around.
For Common Lisp implementations alone (both commercial and
free) see http://www.lisp.org/table/systems.htm and take your
pick.
> Uhm, that's assuming that they don't have some misfeature.
> For example, BASIC lacked block structure, local variables
> and any kind of records for so long that when introduced,
> they were non-standard. Fortran had its day, and
> in 1977 it got block structure into the standard. Did it ever
> have portable access to dynamic memory? If so, it was
> too late. VB is currently viable. I don't use it as I don't
> support Windows, and it isn't cross platform. It changes
> much to fast for anything I'd want to accomplish. Expect
> to have to rewrite everything every three years. This limits
> you to what you can rewrite in three years. If Microsoft
> decides it can't milk any more from VB, it's history.
The above paragraph looks like it comes from a programmer who
has seen a lot, and who needs to rediscover lisp to complete
his experience.
> I may be ready to use Lisp again. Maybe I'm getting old.
> Lisp has demonstrated that I could write in it without having
> to scrap my code instantly. An individual can produce a
> working Lisp environment. So, it doesn't take 90% of the
> computer industry language market to ensure availability,
> cross platform.
I agree that you should give Lisp another try. I disagree that
there is _any_ problem whatsoever with availability.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
"Stephen Uitti" <······@ford.com> writes:
> Explicit memory management (C) is generally faster ...
Many (myself included) would disagree with this generality.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
"Stephen Uitti" <······@ford.com> writes:
> Should Lisp be considered "Object Oriented"?
More so that C++, since it has garbage collection and multiple
inheritance, which make it much easier to build OO models.
> Explicit memory management (C) is generally faster, but this is
> increasingly less important (and for many programmers,
> increasingly less true).
It's been less true in any large program for a long time, since
managing the free lists takes a surprising amount of time.
> Fortran had its day, and in 1977 it got block structure into the
> standard. Did it ever have portable access to dynamic memory? If so,
> it was too late.
In Fortran 90; it wasn't too late for those still using it in 2000.
<URL:http://www.xanalys.com/software_tools/mm/articles/lang.html>
describes how all the old languages are slowly acquiring decent memory
management features. When will Fortran have GC?
--
Pekka P. Pirinen, Adaptive Memory Management Group, Harlequin Limited
A programming language is low level when its programs require attention to
the irrelevant. - Alan Perlis
On 19 Dec 2000 15:16:52 +0000, ·····@harlequin.co.uk (Pekka P. Pirinen) wrote:
>"Stephen Uitti" <······@ford.com> writes:
>> Should Lisp be considered "Object Oriented"?
>
>More so that C++, since it has garbage collection and multiple
>inheritance, which make it much easier to build OO models.
C++ does have multiple inheritance.
//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------
Fernando Rodr�guez <·······@must.die> writes:
> On 19 Dec 2000 15:16:52 +0000, ·····@harlequin.co.uk (Pekka P. Pirinen) wrote:
> >> Should Lisp be considered "Object Oriented"?
> >
> >More so that C++, since it has garbage collection and multiple
> >inheritance, which make it much easier to build OO models.
>
> C++ does have multiple inheritance.
Indeed it does. Multiple dispatch was what I was thinking about while
trying to make that snide remark about C++.
Forgot to mention the MOP, too. And dispatching on built-in types,
not just classes.
--
Pekka P. Pirinen, Adaptive Memory Management Group, Harlequin Limited
Heard at OOPSLA 2000: We didn't pick the best solution [for the architecture],
because it would have required C++ programmers who understand the language.
"Pekka P. Pirinen" <·····@harlequin.co.uk> wrote in message
···················@harlequin.co.uk...
> Fernando Rodr�guez <·······@must.die> writes:
> > C++ does have multiple inheritance.
>
> Indeed it does. Multiple dispatch was what I was thinking about while
> trying to make that snide remark about C++.
>
> Forgot to mention the MOP, too. And dispatching on built-in types,
> not just classes.
Also, unlike C++, Lisp doesn't require that all the methods for a class
be mentioned in a class header. So, to generalise the above, you can
dispatch on classes (and other types) that are:
- built-in
- defined in other parts of your own application
- defined in libraries.
I like that a lot!
______________________________
Simon Katz
On Wed, 20 Dec 2000, Simon Katz wrote:
> Also, unlike C++, Lisp doesn't require that all the methods for a class
> be mentioned in a class header.
To be fair (not that I'm too concerned with being fair to C++), it's
trivial to work around that limitation by abandoning the message-passing
syntax. Just make the member variables public and use functions instead
of methods -- it makes C++ a little more bearable.
Tom
On Tue, 19 Dec 2000 06:38:22 GMT, in comp.lang.lisp you wrote:
>> I _resent_ the implication that I'm responsible for the Marcus G. Daniels
>> fan club and in particular the likes of Israel Thomas. As long as you
>> guys keep blaming me for other people's vile and reprehensible actions,
>
>Erik, you are being blamed for YOUR vile and reprehensible actions.
>Just a little while ago, you suggested that someone should kill
>Marcus.
>
>In most countries you could end up facing court over that.
>I suspect that if Marcus took it up with the authorities in Norway,
>you would be getting a friendly visit from your ISP.
Sorry, that should have read:
"I suspect that if Marcus took it up with the authorities in Norway,
you would be getting a friendly visit from your police."
Duane Rettig wrote:
> Any guesses as to the name of the next fad language
> to be replaced in the beginning of this eternal question?
java?
;)w
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91eafr$3u7@web.nmti.com>
In article <············@eccws12.dearborn.ford.com>,
Stephen Uitti <······@ford.com> wrote:
> Why do people use Perl and Java instead of Lisp and Forth?
Don't ask me about Java, but while I'm not a great fan of Perl I can answer
that one...
Because you can write two lines of Perl and have a program that does something
useful on just about any popular OS widely used today.
Provide a "lispscript" tool that lets you write, oh:
awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
in no more than a couple of lines of code, and you'll get people using
Lisp for casual scripting. Once they're doing that, they'll keep using it
for more complex projects. Or come up with a killer web page creation
tool written in lisp. Something to get people started on it.
The problem with lisp, and forth, for this kind of thing is there's no
hook to get people started on them any more. Emacs Lisp is about the closest
thing, but that depends on getting people using Emacs, and it's not for
everyone (me, for example).
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Peter da Silva wrote:
>
> In article <············@eccws12.dearborn.ford.com>,
> Stephen Uitti <······@ford.com> wrote:
> > Why do people use Perl and Java instead of Lisp and Forth?
Actually, thinking about the way I have stuff set up,
1. I always have a lisp window available, in an emacs buffer.
2. the structure of almost all important files in my world
are either straight text (.tex source), images, or are
trivially parsed as lisp. That is, when read in,
they produce their own data structures and/or programs.
When you, or perl program reads text, then you first have to
write a parsing program to make sense of it and so
you have lost a few hours. Or you are just using characters.
Just being able to read characters and spit them
out is really not of interest to me. Searching
can be done with an editor. Is the kind of processing
in that line of perl really of interest?
If it is not of interest to you, find a test
that makes sense and see if my world, not just
lisp programs in your world, is better
equipped to deal with it. Consider: I can
store programs, or expressions like
(+ a (* b c)) in a file and read them in,
MAKING PERFECT SENSE of them, by the function
(read).
>
> Don't ask me about Java, but while I'm not a great fan of Perl I can answer
> that one...
>
> Because you can write two lines of Perl and have a program that does something
> useful on just about any popular OS widely used today.
>
> Provide a "lispscript" tool that lets you write, oh:
>
> awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>
> in no more than a couple of lines of code, and you'll get people using
> Lisp for casual scripting. Once they're doing that, they'll keep using it
> for more complex projects. Or come up with a killer web page creation
> tool written in lisp. Something to get people started on it.
I don't know about a killer tool, but I use lisp for a server and
for creation of dynamic html pages.
I'm not sure this makes any difference to any of the readers of this
list, though.
RJF
From: Harald Hanche-Olsen
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <pcosnnoa5i8.fsf@thoth.home>
+ Richard Fateman <·······@cs.berkeley.edu>:
| Peter da Silva wrote:
| >
| > Provide a "lispscript" tool that lets you write, oh:
| >
| > awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
| >
| > in no more than a couple of lines of code, and you'll get people
| > using Lisp for casual scripting. Once they're doing that, they'll
| > keep using it for more complex projects. Or come up with a killer
| > web page creation tool written in lisp. Something to get people
| > started on it.
|
| Just being able to read characters and spit them
| out is really not of interest to me. Searching
| can be done with an editor. Is the kind of processing
| in that line of perl really of interest?
The life of a system administrator tends to be filled up with trivia
for which that little perl (awk actually, but it's the same with perl)
snippet can save a lot of time. But is it really of interest? Not
really, it just saves time. Many sysadmins of course wish to automate
more of their job, trying to work on higher levels of abstraction.
But then, as they have already gotten used to doing the simple stuff
with perl, they tend to keep on using perl for bigger and more
important tasks until they get to the point where the language gets in
the way rather than being a help. The sad thing is how they then
carry on way past that point. What I think Peter is calling for is a
way to ease the transition to a more reasonably high level language
like lisp by lowering the initial hurdle to overcome before you can do
immediately useful, if trivial, things with it.
--
* Harald Hanche-Olsen <URL:http://www.math.ntnu.no/~hanche/>
- Yes it works in practice - but does it work in theory?
Voice in the desert: Quiet, isn't it, Harald Hanche-Olsen?
> What I think Peter is calling for is a
> way to ease the transition to a more reasonably high level language
> like lisp by lowering the initial hurdle to overcome before you can do
> immediately useful, if trivial, things with it.
A bit like Guile?
http://www.gnu.org/software/guile/guile.html
A lot of work has still to be done, which is why I use Perl for the
endless trivial things and Scheme for everything else. Perl is fast
and brutal, making it ideal for small jobs, while Lisp (in all forms)
is elegant and gracefully scales to any size of project.
Perhaps the best way to describe Perl is as a "useful animal in the
desert." It's always available (and when it's not, it' trivial to
download and install) and there's a good chance that what you want,
admin-wise, has already been done.
But if you can avoid the desert, then a horse is preferable. ;)
--
Email address intentially munged | You can never browse enough
will write code that writes code that writes code for food
<URL:http://www.wildcard.demon.co.uk>
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91iho1$sgs@web.nmti.com>
In article <··························@news.demon.co.uk>,
Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk> wrote:
> Voice in the desert: Quiet, isn't it, Harald Hanche-Olsen?
> > What I think Peter is calling for is a
> > way to ease the transition to a more reasonably high level language
> > like lisp by lowering the initial hurdle to overcome before you can do
> > immediately useful, if trivial, things with it.
> A bit like Guile?
> http://www.gnu.org/software/guile/guile.html
The problem with Guile is that it was created to acieve a political goal, not
to solve any specific problem: most of the people politically motivated to
use it are likely using emacs lisp for the same purpose.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
On 17 Dec 2000, Peter da Silva wrote:
> In article <··························@news.demon.co.uk>,
> Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk> wrote:
> > Voice in the desert: Quiet, isn't it, Harald Hanche-Olsen?
> > > What I think Peter is calling for is a
> > > way to ease the transition to a more reasonably high level language
> > > like lisp by lowering the initial hurdle to overcome before you can do
> > > immediately useful, if trivial, things with it.
>
> > A bit like Guile?
> > http://www.gnu.org/software/guile/guile.html
>
> The problem with Guile is that it was created to acieve a political goal, not
> to solve any specific problem: most of the people politically motivated to
> use it are likely using emacs lisp for the same purpose.
Actually, it was created to solve a specific problem: easily writing
easily extensible applications. Embedding elisp in all GNU applications
would be a terrible idea, and who wants to write one's program logic in
Perl or Tcl? (Well, obviously some people do, but for those of us who
want to use a lisp or suitably-enhanced scheme, it's no good to have 10
applications linking to 10 different embedable lisp/scheme
implementations; at least not from the point of view of wanting a
somewhat-cohesive GNU system).
As for Guile-as-scripting-language, I think you're right: I use elisp for
most all of my scripting needs. I'm used to it and more competant in it,
and it has the advantage that it's quite a bit more reasonable to expect
it to be installed on a given system than guile.
Tom
Voice in the desert: Quiet, isn't it, Peter da Silva?
> > A bit like Guile?
> > http://www.gnu.org/software/guile/guile.html
>
> The problem with Guile is that it was created to acieve a political goal, not
> to solve any specific problem: most of the people politically motivated to
> use it are likely using emacs lisp for the same purpose.
Maybe that's how Stallman sees it. ;) If you look at the list of Guile
projects, you'll see that people are doing some real things with it.
http://www.gnu.org/software/guile/gnu-guile-projects.html
Nothing to do with Emacs. I'm not sure if that project is even listed.
I'd also suggest looking at Scsh, but I've not used that one...
--
Email address intentially munged | You can never browse enough
will write code that writes code that writes code for food
<URL:http://www.wildcard.demon.co.uk>
In article <··························@news.demon.co.uk>,
Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk>
wrote:
>I use Perl for the
>endless trivial things and Scheme for everything else. Perl is fast
>and brutal, making it ideal for small jobs, while Lisp (in all forms)
>is elegant and gracefully scales to any size of project.
>
>Perhaps the best way to describe Perl is as a "useful animal in the
>desert." It's always available (and when it's not, it' trivial to
>download and install) and there's a good chance that what you want,
>admin-wise, has already been done.
>
>But if you can avoid the desert, then a horse is preferable. ;)
A useful analogy that I often use is to compare programming languages
to screwdrivers. I have lots of different screwdrivers in my toolbox
at home -- some for Robertson-head ("square"), some for Phillips-head
("+ sign"), some for whatever we call a "-" sign slot, some big ones,
some little ones, some with multiple tips that can go into a single
handle. Having -- and being proficent in using -- a wide variety of
screwdrivers means that I can more easily use one well-matched to the
job at hand.
Similarly, these days I tend to use C++ for "traditional" programming,
heavy-duty number-crunching, and suchlike, calling other people's C
and Fortran routines (or my own old ones from before I learned C++)
since I dislike reinventing the wheel. But I often hack up a quick
couple thousand lines of Perl for some least-squares fitting, data analysis,
generating gnuplot command scripts to display movies, and other stuff
where nanoseconds don't count. But if I want to do heavy-duty symbolic
algebra, none of these languages are anyways near as good as a language
that's focused in that area, say Maple (Macsyma and Mathematica are some
other popular choices in that area).
I don't usually work in those problem domains, but if I needed to, say,
hack up a quick program to search a game tree, generate Scrabble patterns
or anagrams of "Florida doesn't vote rite", I might try Perl because I'm
familiar with it and it would be adequate. But I know that Icon would
likely be better. Similarly, if I needed to write code to generate other
code, well, I've done that in Maple (generating C/C++), but Lisp has some
very nice features to morph back and forth between code and data, so maybe
I'd summon up my 20-year-old Lisp skills and try that. And there's always
APL lurking in the back of my mind, and another half-dozen misc languages
that also have their own niches (think "screwdrivers I haven't used in a
long time, that now gather dust in the bottom of the rummage drawer...
but that might come in handy some day).
Summary #1: Use tools which are well-suited for the job!
Summary #2: Put square pegs in square holes, round pegs in round holes.
That means, for example:
* Don't try to use Fortran 66 to write a multigrid solver for which
dynamic data structures, recursion, heavy-duty number crunching,
*and* Fortran 77 compatability are all important.
Externalities also count:
* A language you already know well, but which isn't that intrinsically
well-suited for a given job, is often a better engineering solution
than an ideally-suited language you have to learn first.
* I tend to use C++ and Perl a lot more than PL/I, APL, and Lisp, at
least in part because C++ and Perl are already installed on all the
computers I use. And if they're not, or if they're broken or ancient
versions, there are excellent C++ and Perl implementations available
at no cost on the net, which I can install myself in a couple of hours
or less, without even needing root access. Even $50 to buy a piece
of software is a _big_ hassle factor, and makes it a lot less likely
I'm going to use the stuff:
- either I have to pay for it out of pocket (and getting a US$ cheque
generated takes a fair hassle for some of us in Europe who don't
speak the local banking system's language very well),
- or I have to persuade someone with purchasing authority (who may
be multiple levels removed from me or crosswise in the organizational
structure) to cut a P/O and order the stuff, which may take many weeks
The (very nice) book
Brian W. Kernighan and Rob Pike.
"The Practice of Programming",
Addison-Wesley, 1999, paperback ISBN 0-201-61586-X,
http://cm.bell-labs.com/cm/cs/tpop/index.html
has a fascinating section where they present solutions to the *same*
programming problems in 6 or so different languages: I seem to recall
C, C++, Java, Awk, Perl, and some others as well. Fascinating reading!
--
-- Jonathan Thornburg <······@thp.univie.ac.at>
http://www.thp.univie.ac.at/~jthorn/home.html
Universitaet Wien (Vienna, Austria) / Institut fuer Theoretische Physik
Q: Only 7 countries have the death penalty for children. Which are they?
A: Congo, Iran, Nigeria, Pakistan[*], Saudi Arabia, United States, Yemen
[*] Pakistan moved to end this in July 2000. -- Amnesty International,
http://www.web.amnesty.org/ai.nsf/index/AMR511392000
······@mach.thp.univie.ac.at (Jonathan Thornburg) writes:
> A useful analogy that I often use is to compare programming languages
> to screwdrivers.
That makes Lisp a milling machine: a very useful tool when you need
something other than a screwdriver, or you need the custom screwdriver
you don't have, or you don't know what tool you need but you need
*something*.
On the other hand, perhaps a bit of overkill when you have a simple
problem and a reasonable tool in hand.
Of course, people who prefer other languages generally have a few
screws loose *anyway*, so there'll always be a need for a few standard
screwdrivers.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjbsu775r2.fsf@tfeb.org>
Joe Marshall <···@content-integrity.com> writes:
> That makes Lisp a milling machine: a very useful tool when you need
> something other than a screwdriver, or you need the custom screwdriver
> you don't have, or you don't know what tool you need but you need
> *something*.
>
Milling machines are way too special purpose: Lisp is one of these
amazing CNC machine tools where you write elaborate programs for it
and it can basically make anything.
Needless to say you write these programs in Lisp.
--tim
······@mach.thp.univie.ac.at (Jonathan Thornburg) writes:
> A useful analogy that I often use is to compare programming languages
> to screwdrivers.
That makes Lisp a milling machine: a very useful tool when you need
something other than a screwdriver, or you need the custom screwdriver
you don't have, or you don't know what tool you need but you need
*something*.
On the other hand, perhaps a bit of overkill when you have a simple
problem and a reasonable tool in hand.
Of course, people who prefer other languages generally have a few
screws loose *anyway*, so there'll always be a need for a few standard
screwdrivers.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
"Joe Marshall" <···@content-integrity.com> wrote in message
·················@content-integrity.com...
> ······@mach.thp.univie.ac.at (Jonathan Thornburg) writes:
>
> > A useful analogy that I often use is to compare programming languages
> > to screwdrivers.
>
> That makes Lisp a milling machine: a very useful tool when you need
> something other than a screwdriver, or you need the custom screwdriver
> you don't have, or you don't know what tool you need but you need
> *something*.
Good analogy. I really have a milling machine and I do program it in Lisp.
As usual Lisp is really great for this.
I process lists like this '((:circle x y z dia)(:pocket x y z dx dy dz)...)
to give CNC G codes.
> On the other hand, perhaps a bit of overkill when you have a simple
> problem and a reasonable tool in hand.
It's exactly what the CNC operators tell me.
> Of course, people who prefer other languages generally have a few
> screws loose *anyway*, so there'll always be a need for a few standard
> screwdrivers.
I agree.
Marc
From: Bob Bradlee
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3A404351.C15EA086@CAVE.NET>
Marc Battyani wrote:
> Good analogy. I really have a milling machine and I do program it in Lisp.
> As usual Lisp is really great for this.
> I process lists like this '((:circle x y z dia)(:pocket x y z dx dy dz)...)
> to give CNC G codes.
Which CAD platform are you built around ?
--
* Bob Bradlee Lord High Fixer Technologist and Consultant *
* ···@Cave.net Cave Network Group Senior Partner & Sys Admin *
* ···@Bradlee.org Bradlee Genealogical Society Founder & Scribe *
Bob Bradlee <···@CAVE.NET> wrote
>
> Marc Battyani wrote:
> > Good analogy. I really have a milling machine and I do program it in
Lisp.
> > As usual Lisp is really great for this.
> > I process lists like this '((:circle x y z dia)(:pocket x y z dx dy
dz)...)
> > to give CNC G codes.
>
> Which CAD platform are you built around ?
None.
I just use Lisp to generate G codes that I feed to the CNC machine.
I mostly makes parts for my electronics: front/back panels, shielding cases
and small parts like an optical fiber to photodiode adatpter.
Marc
From: Bob Bradlee
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3A40C265.45D12881@CAVE.NET>
Marc Battyani wrote:
> None.
> I just use Lisp to generate G codes that I feed to the CNC machine.
> I mostly makes parts for my electronics: front/back panels, shielding cases
> and small parts like an optical fiber to photodiode adatpter.
A number of years ago I built a system that extracted geometry
from Autocad drawings and generated M&G code to reproduce
the drawing in steel. I did a panel design system for what
now has become Rockwell Automation. No cheese involved, just
the power of LISP :)
Bob
"Jonathan Thornburg" <······@mach.thp.univie.ac.at> wrote in message
················@mach.thp.univie.ac.at...
Snip
> A useful analogy that I often use is to compare programming languages
> to screwdrivers. I have lots of different screwdrivers in my toolbox
> at home -- some for Robertson-head ("square"), some for Phillips-head
> ("+ sign"), some for whatever we call a "-" sign slot, some big ones,
> some little ones, some with multiple tips that can go into a single
> handle. Having -- and being proficent in using -- a wide variety of
> screwdrivers means that I can more easily use one well-matched to the
> job at hand.
I think this analogy fails in one important area. The "learning curve" for
different tipped screw drivers is essentialy zero. While I certainly agree
that if you are going to do a lot of work in an area, larning a "well
suited" tool is best, I agree with your later (snipped) comment that
someitmes it isn't worth the trouble. As a result, few programmers know the
range of languages that you do. So that still leaves us with the questions
of range of suitability to various problems of various languages. The
correct choice or choices of course depend upon your particular mix of
problem types/sizes and domains.
On a related note, there is another analogy that may be apt here. It seeems
to me that PERL is like BASIC in that it is easy to learn, great for small
problems, and can be stretched for use with big problems. But like BASIC,
many of the things that make it easy to learn and use for small problems
hinder its use for larger ones. The challenge then is to find/invent
languages that are easy to learn and have minimal "hassle" for small
problems, yet scale well. I had thought that for general purpose imperitive
languages, that Turing (with its systems and object oriented extensions) was
a good candidate for that. But Turing seems to have fallen by the way side,
or at least been overtaken by Java, which has some other advantages, but
doesn't have the ease of use of Turing. Is Python a good candidate for this
role as a scripting language? I don't know enough to know, but it seems
like it.
Good stuff snipped
--
- Stephen Fuld
>
> --
> -- Jonathan Thornburg <······@thp.univie.ac.at>
> http://www.thp.univie.ac.at/~jthorn/home.html
> Universitaet Wien (Vienna, Austria) / Institut fuer Theoretische Physik
> Q: Only 7 countries have the death penalty for children. Which are
they?
> A: Congo, Iran, Nigeria, Pakistan[*], Saudi Arabia, United States,
Yemen
> [*] Pakistan moved to end this in July 2000. -- Amnesty
International,
>
http://www.web.amnesty.org/ai.nsf/index/AMR511392000
On Tue, 19 Dec 2000 18:19:45 GMT, "Stephen Fuld"
<······@worldnet.att.net> wrote:
[...]
>problems, yet scale well. I had thought that for general purpose imperitive
>languages, that Turing (with its systems and object oriented extensions) was
>a good candidate for that. But Turing seems to have fallen by the way side,
>or at least been overtaken by Java, which has some other advantages, but
>doesn't have the ease of use of Turing. Is Python a good candidate for this
>role as a scripting language? I don't know enough to know, but it seems
>like it.
Yes. Well, I think so, anyway.
Peter Norvig introduces Python to Lisp programmers at
http://www.norvig.com/python-lisp.html.
There is much additional info at www.python.org
Interest in Python seems to be growing, too. Last time I read (tried
to read) comp.lang.python (last summer) it was getting over 500
messages a day
Followups trimmed. Regards,
Icarus
--
"Where there is no fear, there can be no courage"
I Don't Do Windoze. ····@my-deja.com
[Followup posted to comp.lang.lisp only - Paolo]
On 19 Dec 2000 17:12:47 +0100, ······@mach.thp.univie.ac.at (Jonathan
Thornburg) wrote:
> A useful analogy that I often use is to compare programming languages
> to screwdrivers. I have lots of different screwdrivers in my toolbox
I don't think that this is a useful analogy. Screwdrivers are extremely
specialized tools compared to many software ones, especially programmable
tools like languages. That's why you need so many screwdrivers, while a
general-purpose language is, well, more general-purpose.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91fu1g$pjg@web.nmti.com>
In article <·················@cs.berkeley.edu>,
Richard Fateman <·······@cs.berkeley.edu> wrote:
> Just being able to read characters and spit them
> out is really not of interest to me. Searching
> can be done with an editor. Is the kind of processing
> in that line of perl really of interest?
Getting a list of disabled users on a UNIX system? Sure.
(and that was actualy in awk, one precursor to perl... I don't do perl
myself, but I guarantee you it wouldn't have taken more than one line)
Since I live in a world where I don't get to define all the tools and file
formats, languages that can trivially snarf data from random files and do
useful stuff with it are absolute necessities.
The problem is when people go from there and build huge applications in them
because that's what they know. The trick is to provide a hook to get people
started in lisp, and I don't know the answer... if I did I'd be doing it.
I do as much as possible in Tcl, a language which has a lot of lisp-nature,
but which also has the ability to grovel over foreign files easily. And, yes,
when I can I save data as Tcl lists in files, and they're just as easily
read and operated on as Lisp lists.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Peter da Silva wrote:
>
> In article <············@eccws12.dearborn.ford.com>,
> Stephen Uitti <······@ford.com> wrote:
> > Why do people use Perl and Java instead of Lisp and Forth?
>
> Don't ask me about Java, but while I'm not a great fan of Perl I can answer
> that one...
>
> Because you can write two lines of Perl and have a program that does something
> useful on just about any popular OS widely used today.
>
> Provide a "lispscript" tool that lets you write, oh:
>
> awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>
> in no more than a couple of lines of code, and you'll get people using
> Lisp for casual scripting. Once they're doing that, they'll keep using it
> for more complex projects. Or come up with a killer web page creation
> tool written in lisp. Something to get people started on it.
The very fact one needs scripting may indicate that the OS may
leave some to be desired. In over thirty years of a career as a
kernel-level developer, for example, I never had the need to use
a scripting language for anything.
>
> The problem with lisp, and forth, for this kind of thing is there's no
> hook to get people started on them any more. Emacs Lisp is about the closest
> thing, but that depends on getting people using Emacs, and it's not for
> everyone (me, for example).
You can go to the Rice University web site, download DrScheme.
Get yourself a copy of the Little Schemer, and get to work. As I
see it, if you're still using text-based editors and XTerms, you
may be becoming a Rip Van Winkle lookalike!
Forth, by the way, is the most powerful programming paradigm out
there. Not even Lisp can match it.
Alberto.
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91ggup$3vk@web.nmti.com>
In article <·················@moreira.mv.com>,
Alberto Moreira <········@moreira.mv.com> wrote:
> The very fact one needs scripting may indicate that the OS may
> leave some to be desired.
That's possible, but so what? People are still writing scripts and using Perl.
No amount of debate over better operating systems will change that.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
·····@abbnm.com (Peter da Silva) writes:
> In article <·················@moreira.mv.com>,
> Alberto Moreira <········@moreira.mv.com> wrote:
> > The very fact one needs scripting may indicate that the OS may
> > leave some to be desired.
>
> That's possible, but so what? People are still writing scripts and using Perl.
> No amount of debate over better operating systems will change that.
The debate towarts OS'es is superflou-ous anyway;
The mantra allways has been: Garbage in == Garbage out.
Even scripting can't change that, nor any OS.
A `PHB' can ask for more garbage (they need) any progger can think of....
cor
--
/* If GNU/LINUX has no solution, you've got the wrong problem */
/* Never install Slackware..........You might learn to use IT */
/* ······@amsat.org http://clsnet.dynip.nl */
Peter da Silva wrote:
> That's possible, but so what? People are still writing scripts and using Perl.
> No amount of debate over better operating systems will change that.
Well, people are still writing Cobol and RPG programs. But if I
remember, one of the points is about how Perl is so much better
because one can encode a lot of script into one line of
write-only code.
Alberto.
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91jeiu$agd@web.nmti.com>
In article <·················@moreira.mv.com>,
Alberto Moreira <········@moreira.mv.com> wrote:
> > That's possible, but so what? People are still writing scripts and using
> > Perl. No amount of debate over better operating systems will change that.
>
> Well, people are still writing Cobol and RPG programs.
Very few that haven't been writing COBOL and RPG for a very very long time.
Perl's mindshare, on the other hand, seems to be growing.
And, unfortunately, Lisp's isn't.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
* Peter da Silva
| Perl's mindshare, on the other hand, seems to be growing.
|
| And, unfortunately, Lisp's isn't.
How do you determine such things?
I see more and more people wanting to get away from Perl because they see
what Common Lisp can do to application development. Maybe that's because
I'm a Common Lisp programmer who argues that Perl solutions are ad hoc
and need to recoded in a real programming language if they are going to
be maintained. Management types seem to accept this line, not just where
I work, but where friends and colleagues work, too. World-wide, it also
seems interest in Lisp is increasing. You can see that by the number of
people who come to Lisp fora only to kick the language. At least they
have heard of it and find it threatening enough to fight. That's a start.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91l8t2$504@web.nmti.com>
In article <················@naggum.net>, Erik Naggum <····@naggum.net> wrote:
> I see more and more people wanting to get away from Perl because they see
> what Common Lisp can do to application development.
And I see people switching to Java and (get your airsick bags ready) Visual
Basic. Outside of Usenet, I don't think I've seen mention of Lisp in years.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
* Peter da Silva
| Outside of Usenet, I don't think I've seen mention of Lisp in years.
I have no reason to doubt that this is true for you, but this also means
that both your ability and mine to judge the growth or death cramps of
languages that have particular groups of followers is very limited. How
do you know that Lisp is in the decline? The Common Lisp vendors are
reporting more users. Various Common Lisp implementations have made it
into Linux distributions and are quite popular. Where I work, Perl is
generally acknowledge to lead to management disasters, largely because of
one guy who quit and left us with more than 100,000 lines of utter crap
that it has taken month to decode and re-engineer and sometimes we can't
even figure out how it could possibly have solved anything at all, so
Perl has a real bad rap. The Common Lisp-based application has been
running for almost a year (325 days today), surviving upgrades and lots
of changes and patches, responding to ever changing needs, and the
interest in Common Lisp is growing. Friends and former colleagues make
up a pretty stron growth rate for Common Lisp in Oslo, Norway, and many
other parts of Norway are experiencing rapid increasing growth. This is
of course a "local" phenomenon, but it's all I _really_ know about, so
when you haven't seen mention of Lisp in years, that's just as true, but
it is not enough to conclude that Lisp is on the decline -- you're just
in an area where Lisp is not used, and I am.
What are we going to about these differences? I thikn it is sage advice
to stay away from "language X is winning and language Y is losing"
statements simply because the world of language choice is bigger than
anyone can reasonably expect to get a grip on.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91m4bg$nti@web.nmti.com>
In article <················@naggum.net>, Erik Naggum <····@naggum.net> wrote:
> * Peter da Silva
> | Outside of Usenet, I don't think I've seen mention of Lisp in years.
> I have no reason to doubt that this is true for you, but this also means
> that both your ability and mine to judge the growth or death cramps of
> languages that have particular groups of followers is very limited. How
> do you know that Lisp is in the decline?
I apologise if I gave the impression that I thought it was on the decline.
I think it's pretty unquestionable that it's less popular than Perl, and
that Perl is growing aggressively.
> Where I work, Perl is
> generally acknowledge to lead to management disasters,
I won't debate that. But in a world where people are running critical
systems on Windows NT and Windows 2000, logic clearly has little to do
with these sorts of decisions. When you have people running industrial
plants with Visual Basic applications in the critical path, what can you
expect?
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
"Erik Naggum" <····@naggum.net> wrote in message
·····················@naggum.net...
> * Peter da Silva
> | Outside of Usenet, I don't think I've seen mention of Lisp in years.
>
> I have no reason to doubt that this is true for you, but this also means
> that both your ability and mine to judge the growth or death cramps of
> languages that have particular groups of followers is very limited.
Its very possible for you both to be correct. Lisp could easily be
gaining programmers in count and losing in the percentage of
programmers battle. Exponential growth, being what it is
allows statements like "over half of the humans that have
ever been born over all of human history are still alive".
OTOH, you could both be wrong.
> Where I work, Perl is
> generally acknowledge to lead to management disasters
It's easy enough to write bad code in any language. I've
written Perl web apps, as part of a team. The whole thing
worked fine. I've also rewritten Perl web apps in C
to overcome performance and maintainability problems.
I've even seen good code that was poorly installed.
Here's a laundry list of management disasters I've seen
of late:
1. Outsourcing the development of a multimillion dollar
application without getting source code. The result was
an application that didn't do the job, since the requirements
were never well defined or communicated. The application
quality was no better (actually worse) than in house apps.
Maintainability was entirely dependant on the out sourcing
company, which jacked prices through the roof. Oddly,
the "fix" was to rewrite the app from scratch - using
an outsourcing strategy that did not include getting
the whole source code. "Hey, we shot ourselves
in the foot, but we still have another."
2. Failure to provide the right tools for the job. For
example, a site that ran Oracle would not spring for
the licenses required, and would not allow a free database
such as Postgres or MySQL to be used (since it would
add yet another database). The resulting applications
complexity resulted in substantially longer development
time, increased risk, and decreased maintainability.
3. Treating programmers as plug and play interchangable.
4. Ignoring the advice of the staff.
5. Failure to recognize when a project is dead.
6. Failure to use the pieces that did work.
7. Use of known unstable platforms.
8. Use in production of pre-release products without
a feasibility study.
9. Underestimation of project scope.
10. Uncontrolled scope creep.
Compared to these, language choice is secondary.
In comp.arch Erik Naggum <····@naggum.net> wrote:
> * Peter da Silva
> | Perl's mindshare, on the other hand, seems to be growing.
> |
> | And, unfortunately, Lisp's isn't.
> How do you determine such things?
Job openings/month? Books published/year?
> I see more and more people wanting to get away from Perl because they see
> what Common Lisp can do to application development. Maybe that's because
Perl isn't 'application development' language, it is 'script development'
language. There is a big difference. Or not, if your application is a
set (possibly big) of scripts, possibly using some support from compiled
and loaded libraries.
> I'm a Common Lisp programmer who argues that Perl solutions are ad hoc
> and need to recoded in a real programming language if they are going to
> be maintained. Management types seem to accept this line, not just where
Perl scripts can and are maintained. It's not harder than lisp - it mostly
depends on coding style.
> I work, but where friends and colleagues work, too. World-wide, it also
> seems interest in Lisp is increasing. You can see that by the number of
> people who come to Lisp fora only to kick the language. At least they
> have heard of it and find it threatening enough to fight. That's a start.
> #:Erik
> --
> The United States of America, soon a Bush league world power. Yeee-haw!
--
Sander
FLW: "I can banish that demon"
* Sander Vesik <······@haldjas.folklore.ee>
| Job openings/month?
The population does not grow if more people die than are born.
If people get hired without advertising, you don't see the job openings
as easily. Not even compulsory government statistics can help you, since
it takes _years_ for the statistics guys to get reasonable figures.
| Books published/year?
The remainder and recall rate is maybe a good indicator. Publishers play
the lottery with books in popular areas, they don't create popularity.
| > I see more and more people wanting to get away from Perl because they
| > see what Common Lisp can do to application development.
|
| Perl isn't 'application development' language, it is 'script development'
| language.
Well, duh. That's why they are leaving Perl: It is unfit for what people
tend to do with it. Scripting only goes so far, but they need to see
something better before they leave Perl, and being better than Perl is
actually not _that_ easy.
| Perl scripts can and are maintained. It's not harder than lisp - it
| mostly depends on coding style.
Yeah, yeah, whatever.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
From: TTK Ciar
Subject: script devel vs app devel, was Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91mi0h03ip@news2.newsguy.com>
> From: Erik Naggum <····@naggum.net>
> Date: 18 Dec 2000 22:44:49 +0000
[..snip..]
>
>| > I see more and more people wanting to get away from Perl because they
>| > see what Common Lisp can do to application development.
>|
>| Perl isn't 'application development' language, it is 'script development'
>| language.
>
> Well, duh. That's why they are leaving Perl: It is unfit for what people
> tend to do with it. Scripting only goes so far, but they need to see
> something better before they leave Perl, and being better than Perl is
> actually not _that_ easy.
I'm deliberately not cross-posting this to any of the perl groups,
because I'd like to keep things civil.. I will readily admit that
there are applications which should not be developed in perl, but I
also believe that there are applications which can be done fairly
well in perl -- small nntp, smtp, and http servers, mail clients,
chat clients, web development tools, and the like.
My experience with using perl in a commercial setting is limited,
however. The two of you seem to have seen perl try and fail in
commercial application development, so I ask you, to what degree is
perl suited/unsuited to being an application development language?
-- TTK
From: Joe Marshall
Subject: Re: script devel vs app devel, was Re: Could CDR-coding be on the way back?
Date:
Message-ID: <snnkci0i.fsf@content-integrity.com>
TTK Ciar writes:
> I also believe that there are applications which can be done fairly
> well in perl -- small nntp, smtp, and http servers, mail clients,
> chat clients, web development tools, and the like.
Given the extraordinarily poor quality and high complexity of smtp
servers, I doubt perl would be a good choice for implementing one.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
From: Craig Brozefsky
Subject: Re: script devel vs app devel, was Re: Could CDR-coding be on the way back?
Date:
Message-ID: <87bsu8ayft.fsf@piracy.red-bean.com>
TTK Ciar writes:
> My experience with using perl in a commercial setting is limited,
> however. The two of you seem to have seen perl try and fail in
> commercial application development, so I ask you, to what degree is
> perl suited/unsuited to being an application development language?
I've done some perl web application development in the past. The
biggest issue I had with it is that using data structures of any
complexity becomes a major nightmare, and wrapping them up as Objects
doesn't help all that much. No perl project had more than three
people hacking on it, and we had a pretty consistent code standard
that got rid of some of the issues that perl's slutty syntax has. We
didn't have the same level of problems that Erik Naggum has talked
about, but I don't discount his experience. Not too brag, but the
people we had doing perl were pretty damn good coders.
Using web application from third parties written in perl, and trying
to hack them in some way, has been nothing but pure hell tho. In
general the quality of Perl code floating around the net is pretty
bad.
Compared to the complexity of the Common Lisp based web applications
we've developed recently, the perl applications were child's play.
It's a bit of a cliche that a good programmer can write decent perl,
but when you give that good coder a Lisp environment to work in, the
quality of work is just exponentially better.
--
Craig Brozefsky <·····@red-bean.com>
LispWeb -- Mailing list for Lisp Based Web Development
Homepage: http://www.red-bean.com/lispweb
In comp.arch Erik Naggum <····@naggum.net> wrote:
> * Sander Vesik <······@haldjas.folklore.ee>
> | Job openings/month?
> The population does not grow if more people die than are born.
> If people get hired without advertising, you don't see the job openings
> as easily. Not even compulsory government statistics can help you, since
> it takes _years_ for the statistics guys to get reasonable figures.
> | Books published/year?
> The remainder and recall rate is maybe a good indicator. Publishers play
> the lottery with books in popular areas, they don't create popularity.
If you don't agree with these metrics which metrics tell you that the LISP
mindshare is growing? Or growing fatser than mindshare for language XXX?
> | > I see more and more people wanting to get away from Perl because they
> | > see what Common Lisp can do to application development.
> |
> | Perl isn't 'application development' language, it is 'script development'
> | language.
> Well, duh. That's why they are leaving Perl: It is unfit for what people
> tend to do with it. Scripting only goes so far, but they need to see
> something better before they leave Perl, and being better than Perl is
> actually not _that_ easy.
> | Perl scripts can and are maintained. It's not harder than lisp - it
> | mostly depends on coding style.
> Yeah, yeah, whatever.
Heh.
> #:Erik
> --
> The United States of America, soon a Bush league world power. Yeee-haw!
--
Sander
FLW: "I can banish that demon"
Sander Vesik <······@haldjas.folklore.ee> writes:
> > Well, duh. That's why they are leaving Perl: It is unfit for what people
> > tend to do with it. Scripting only goes so far, but they need to see
> > something better before they leave Perl, and being better than Perl is
> > actually not _that_ easy.
>
> > | Perl scripts can and are maintained. It's not harder than lisp - it
> > | mostly depends on coding style.
>
> > Yeah, yeah, whatever.
Yeah is right. I maintain some of the stuff I've written in Perl, and
I find it pretty easy to maintain, even if I havn't looked at it for
six months or so. And I also find it easier to make use of Perl-style
libraries that I find than the Lisp ones. Of course, if I'm writing
them, then CLOS beats the life out of Perl by a long shot (no
comparison). Perl's OO mechanism is great, but cumbersome, and not
even that efficient per unit line of code. I find that when it comes
to OO-style programming, NOTHING can compare with CL. Really,
anything else is a toy (especially when you consider the MOP
extensions).
There's a lot in Perl to facilitate writing larger systems to be
maintainable, well-documented, and reasonably efficient. I even know
of a company in healthcare which is writing the bulk of their flagship
software in Perl (a huge project), and are doing very well. Part of
the wisdom of that decision is maintainability if the right standards
are set. Also, there's a pretty healthy bunch of Perl people out
there.
Dave.
[Followup posted to comp.lang.lisp only - Paolo]
On 18 Dec 2000 21:34:47 GMT, Sander Vesik <······@haldjas.folklore.ee>
wrote:
[about Lisp's mindshare and popularity]
> > How do you determine such things?
>
> Job openings/month? Books published/year?
While high values of job openings/month and books published/year may be
useful data for estimating the popularity and mindshare of a programming
language, low values may not be conclusive hints that a language is not
popular or doesn't have mindshare.
How many job openings are there for US presidents each month? How many
tutorials for US presidents are published each year? What kind of
conclusions can you draw about the job of being a US president from this
information only?
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
In article <···········@mach.thp.univie.ac.at>, I wrote:
> * I tend to use C++ and Perl a lot more than PL/I, APL, and Lisp, at
> least in part because C++ and Perl are already installed on all the
> computers I use. And if they're not, or if they're broken or ancient
> versions, there are excellent C++ and Perl implementations available
> at no cost on the net, which I can install myself in a couple of hours
> or less, without even needing root access. Even $50 to buy a piece
> of software is a _big_ hassle factor, and makes it a lot less likely
> I'm going to use the stuff:
> [[...]]
I should perhaps add here that I'm well aware that there are some
very nice no-cost Lisp systems around (Emacs is a notable example).
My statements were (are) in the context of full-featured Common Lisp
systems (and "full-featured" here includes "good enough for serious work"
performance, too). I don't know of any that are available as source
distributions supporting every computer I'll every use at no cost, in
the manner of (say) Perl or gcc. If such Common Lisp systems exist,
I'd be delighted to see followups pointing to their web pages...
--
-- Jonathan Thornburg <······@thp.univie.ac.at>
http://www.thp.univie.ac.at/~jthorn/home.html
Universitaet Wien (Vienna, Austria) / Institut fuer Theoretische Physik
Q: Only 7 countries have the death penalty for children. Which are they?
A: Congo, Iran, Nigeria, Pakistan[*], Saudi Arabia, United States, Yemen
[*] Pakistan moved to end this in July 2000. -- Amnesty International,
http://www.web.amnesty.org/ai.nsf/index/AMR511392000
On Sat, 16 Dec 2000 09:06:38 -0500, Alberto Moreira <········@moreira.mv.com>
wrote:
>Forth, by the way, is the most powerful programming paradigm out
>there. Not even Lisp can match it.
Why, what's so special about it?
//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------
Fernando Rodr�guez wrote:
>
> On Sat, 16 Dec 2000 09:06:38 -0500, Alberto Moreira <········@moreira.mv.com>
> wrote:
>
> >Forth, by the way, is the most powerful programming paradigm out
> >there. Not even Lisp can match it.
>
> Why, what's so special about it?
The ability to do incremental compilation on the fly. The
availability of having *all* language functionality, even those
only present in a compiler, available to an application writer.
The ability to control what's done at compile time and what's
done at run time. The ability to do delayed evaluation at the
programmer's fingertips. The ability to be syntax independent
and to mold one's syntax to the application. The ability to
control, even dynamically, the balance between
performance-oriented bit-twiddling machine code and higher level
code. The ability to move all the way from the lowest level to
the highest abstraction within the same language framework. The
ability to reduce applications to Forth words and to embed
applications within applications just by invoking those words.
OLE ? COM ? These things, in Forth, come in the wash.
I could go on, mind you. I can, in a rather small space too,
write a Lisp compiler in Forth, I could just as easily add to
it, for example, as much Prolog-like logic programming as I
want, and still have the full Forth paradigm superimposed on top
of it, and be able to control when, how, where, and to what
extent, I compile or intepret code.
I know of no other language that comes even close.
Alberto.
Alberto Moreira <········@moreira.mv.com> wrote:
> > [W]hat's so special about [Forth]?
>
> The ability to do incremental compilation on the fly.
Exactly like Common Lisp (see the COMPILE function).
> The availability of having *all* language functionality, even those only
> present in a compiler, available to an application writer.
My feeling is that Common Lisp gets closer than most. Perhaps more
specific examples would help?
> The ability to control what's done at compile time and what's done at run
> time.
Like Common Lisp (with CL you get to play with read- and load-times as
well).
> The ability to do delayed evaluation at the programmer's fingertips.
I might be misunderstanding you grossly, but don't QUOTE and EVAL give
you this? To the best of my knowledge neither languages give you lazy
evaluation as standard. (Well, Forth words don't have arguments, as
such!).
> The ability to be syntax independent and to mold one's syntax to the
> application.
Exactly like Common Lisp.
> The ability to control, even dynamically, the balance between
> performance-oriented bit-twiddling machine code and higher level code.
[and]
> The ability to move all the way from the lowest level to the highest
> abstraction within the same language framework.
Common Lisp is pretty good in these respects. Especially when (either
CL and Forth are) compared to other languages.
My gut feeling is that Forth has the edge in going lower and CL has the
edge on getting higher. But I'm biased: I tend not to do much
bit-bashing these days and CL just fits my head.
> The ability to reduce applications to Forth words and to embed
> applications within applications just by invoking those words.
Change 'word' to 'function' and you could say Common Lisp (or at least
Lisp). Then again if you did that you could name almost any other
language. Whether words/functions are enough for non-trivial examples
is another discussion.
Lisp (of one form or another) has been embedded in a number of
applications (AutoCad, Emacs and InterLeaf to name the famous ones). I
can think of none that use Forth (but would be interested to hear about
any).
Forth seems to be embedded in lower-level types of soft- and firmware
(I'm thinking about OpenBoot specifically).
> I could go on, mind you. I can, in a rather small space too, write a Lisp
> compiler in Forth,
And vice versa.
> I could just as easily add to it, for example, as much Prolog-like logic
> programming as I want, and still have the full Forth paradigm superimposed
> on top of it, and be able to control when, how, where, and to what extent,
> I compile or intepret code.
Likewise.
> I know of no other language that comes even close.
Well, my analysis (such as it is!) is that Common Lisp comes pretty
close.
When I last looked into these issues (albeit extremely superficially)
Forth and Lisp seemed to me startlingly similar in a number of respects,
even though they are at opposite ends of the spectrum. (Well, some
imagined spectrum of mine.)
Off the top of my head, I would claim that their similarities are the
ways in which they both differ from other, more 'mainstream' languages.
So I'd still like to hear what you think is special about Forth. Even
it's about trade-offs rather than specific features. I mean, they are
/very/ different languages! ;)
--
Duncan Harvey,
"The mainstream is just a tidal wave of morons."
Duncan Harvey wrote:
>
> Alberto Moreira <········@moreira.mv.com> wrote:
>
> > > [W]hat's so special about [Forth]?
> >
> > The ability to do incremental compilation on the fly.
>
> Exactly like Common Lisp (see the COMPILE function).
> My feeling is that Common Lisp gets closer than most. Perhaps more
> specific examples would help?
> Like Common Lisp (with CL you get to play with read- and load-times as
> well).
>
> > The ability to do delayed evaluation at the programmer's fingertips.
>
Let me warn you, you've opened the floodgates. I hadn't touched
this in many a year, but it's one of my favorite subjects, let
me just hope I don't flood you with nonsense!
I believe Lisp gives us some of the abilities of Forthlike
languages. You can kind of compile on the fly. You can sort of
do delayed evaluation. In fact, that alone places Lisplike
languages in a separate plane, inaccessible to most other
languages. But Forth goes beyond, because its building blocks
are even more elementary: you get more raw power.
> I might be misunderstanding you grossly, but don't QUOTE and EVAL give
> you this? To the best of my knowledge neither languages give you lazy
> evaluation as standard. (Well, Forth words don't have arguments, as
> such!).
You can define lazy evaluation on the fly because you can
compile on the fly, at an atomic level if you wish. If you want
to delay the evaluation of a word, say, PLUS, you can compile
LIT PLUS EXECUTE instead of executing the plus. Now, you can
easily compile a bracketed sequence of words on the fly, by
defining the openbracket as the equivalent of "UNTIL
CLOSEBRACKET WORD LOOKUP EXECUTE", and here EXECUTE is easily
redefined to compile the sequence, so that for example you could
define << 1 2 + >> to generate "LIT 1 EXECUTE LIT 2 EXECUTE LIT
+ EXECUTE" or something like that, and that will delay the
evaluation as you wish. The compile facility of Forth is built
into the language, from the inside out, you actually define new
Forth words by using the compile mechanism, and you have a
"COMPILE" word to do it on the fly, word by word.
For example, you define a new function in Scheme using (define
foo .... ). Well, in Forth you would use : foo ... ; It looks
similar, but the similarity is superficial. If I have : foo bar
baz burp ; What I'm telling the Forth system to do is the
conceptual equivalent of something like "UNTIL SEMICOLON WORD
LOOKUP EXECUTE", which will build a forth word consisting in
threaded code tokens for FOO, BAR and BURP, and then do the
equivalent of "FOO MAKEDICTIONARY ENTRY" which will put the
entry in the current dictionary context. That context, mind you,
is also controlled by the programmer and not by the language
interpreter, so that I can stack and unstack vocabularies as I
wish and mimic whatever runtime management philosophy I care to
mimic. The EXECUTE word is accessible to any program, so that if
I want I can define my own compile loop by redefining its
meaning. More, I can compile "immediate" words which are
executed at compile time, and the behavior of the immediate
compilation is, again, overridable by the programmer. So it's I
the programmer who am in charge of the compilation, not the
language interpreter.
It isn't that the standard language gives it, the standard
language Forth only gives us the very basic mechanisms: threaded
code, compile, execute, the dictionary, and a few others. In
Forth, everything else is built on top of this very minimalistic
frame. So, delayed evaluation, like much else, is build on top
of the basic mechanism. But the core machinery is so flexible,
that there's barely anything that cannot be build with it.
> Exactly like Common Lisp.
No that's not true. In Lisp you're a slave of the list. In a
Forthlike language, you have words. I can for example define the
open parenthesis as a word that scans for the matching close
parenthesis and builds a list, but I can easily redefine it to
denote a vector, an object, a runtime statement construct, or
anything else I want to. Furthermore, I can define "(" and "(("
to be two different words if I so desire, I can overload the
meaning of the open parenthesis with something else depending on
the context, and so on.
> Common Lisp is pretty good in these respects. Especially when (either
> CL and Forth are) compared to other languages.
Common Lisp cannot go down like Forth can. I can assemble
individual instructions on the fly if I so desire, and I can
control the extent and depth of that assembly. I can make Forth
look like any language that's out there, even languages that are
totally foreign to the Lisp way of being. I can make Forth look
just like Prolog, for example, syntax and all, because I have
all the basic mechanisms at my fingertips.
> My gut feeling is that Forth has the edge in going lower and CL has the
> edge on getting higher. But I'm biased: I tend not to do much
> bit-bashing these days and CL just fits my head.
Forth has been sorely underutilized and demonized by the
computer science world. Some day we may see somebody stand up
and put Forthlike languages at their right place vis-a-vis to
other languages, but I don't expect to live to see that day.
> Change 'word' to 'function' and you could say Common Lisp (or at least
> Lisp). Then again if you did that you could name almost any other
> language. Whether words/functions are enough for non-trivial examples
> is another discussion.
But a word is not a function, although it can be made to look
like one. A word is sensitive to compile or run mode; a word has
side effects; a word executes differently depending on the
dictionary context it's being used; a word can compile something
on the fly and even jump into it. A word can create new syntax
or new semantics. Note well, the stack is just a simple way to
make data visible with the least amount of effort, but that
doesn't mean that words see "parameters" on the stack. A word
may only receive one parameter, but it may - legally - drop more
than that parameter from the stack, it may not drop anything, it
may push things onto the stack before it comes back, it may
flush the stack. It may pass its stack to a further word,
because Forth uses *two* stacks, a data stack and a program
stack, and they're totally independent. You see, I can make
Forth execution look like C, or like Pascal, or like Algol, or
like Scheme, or make it be dynamic binding like Lisp, and I can
do all of that and more all at the same time in different places
in my program. In Forth, the programmer controls the semantics,
not the language standard.
An example, say I want to build an x86 assembler into my
application. So I want to handle something like "mov eax foo"
inside my Forth program. So I define "mov" as an immediate Forth
word, that is, it is executed at compile time, That word can be
made to (1) parse "eax" and match it against the proper
vocabulary, (2) parse "foo" and match against its own
vocabulary, (3) build the instruction, and (4) compile it
incrementally into the word I'm compiling. So I can, from the
moment I define my "mov" word, intersperse x86 assembler
instructions into my code any time I want to. So, "mov" is a
word, a compile-time immediate word, but it is not really a
function. More, I can choose to expose that "mov" word to the
user of the application, or not, at my discretion. You see, with
Forth words I am in command, not the Interpreter or the
Compiler.
Another example, say I'm writing a linker. I could design my
object format such that the relocation information is encoded as
Forth threaded code. That threaded code can do anything the
basic language supports. So, when I link, I run the words that
set up relocation information to be commonly visible, and then I
run the words that fill up the relocatable modules with that
information. The words go in the object module headers, and do
their job with whatever information is available. So, to link,
you Forth-execute the object modules!
As another example, the functionality of the Java virtual
machine has been available in Forth years before Java ever was.
In fact, I wonder if the fact that Sun was a bit into Forth
didn't play a role in the way they designed that VM; the
bytecode concept was, as far as I know, pioneered in the UK in
the seventies with the SNIBBOL language, that they later renamed
to MINT, and it looked remarkably like a higher level Forth.
> Lisp (of one form or another) has been embedded in a number of
> applications (AutoCad, Emacs and InterLeaf to name the famous ones). I
> can think of none that use Forth (but would be interested to hear about
> any).
>
> Forth seems to be embedded in lower-level types of soft- and firmware
> (I'm thinking about OpenBoot specifically).
Forth has traditionally been used in tight places, I've seen
real nice systems buried into 4K Eproms. If I remember
correctly, some Sun machines use Forth for their Bios.
> Well, my analysis (such as it is!) is that Common Lisp comes pretty
> close.
>
> When I last looked into these issues (albeit extremely superficially)
> Forth and Lisp seemed to me startlingly similar in a number of respects,
> even though they are at opposite ends of the spectrum. (Well, some
> imagined spectrum of mine.)
>
> Off the top of my head, I would claim that their similarities are the
> ways in which they both differ from other, more 'mainstream' languages.
There are, indeed, similarities, but there are fundamental
differences.
> So I'd still like to hear what you think is special about Forth. Even
> it's about trade-offs rather than specific features. I mean, they are
> /very/ different languages! ;)
I believe the main difference is the atomicity of Forth, and the
almost total absense of imposed syntax or semantics. I can, in
Forth, build programs word by word, machine instruction by
machine instruction. I can build a syntatical environment for a
user language, again, token by token, rule by rule, run-time
facility by run-time facility. I am not restricted to the
mechanisms the language give me, because the basic mechanism is
so simple and so flexible that I can morph it into just about
anything I want. It isn't that I can implement Prolog's
functionality in Forth, I can make Forth *look like* Prolog,
build a Prolog-on-the-fly for a Prolog programmer if I want too.
It doesn't work quite like that in Lisp, because of the Lambda
Calculus inheritance and the functional approach. Forth is *not*
functional, nor is it block-structured, but it can become either
of those and more, all at the same time, in intensities that are
controlled by the programmer. More, the symbol table mechanisms
in Forth are available to both Forth and to the Forth
programmer, so I don't need to program environments and the
like: I use the basic dictionary mechanism of the language, and
I extend it as I need, either at compile or at run time as it
may soothe me the best.
It's hard to explain, one has to get down to it and do some
programming to feel the difference.
Alberto.
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkj66kiqc5o.fsf@tfeb.org>
I think this is going nowhere fast. It's evident from a number of
things you say that you're not really familiar enough with Common Lisp
and its facilities to make meaningful comments, and I suspect few Lisp
people are familiar enough with Forth to argue usefully. Why don't we
just stop before it turns into some futile war of attrition?
--tim
Tim Bradshaw <···@tfeb.org> writes:
> I think this is going nowhere fast. It's evident from a number of
> things you say that you're not really familiar enough with Common Lisp
> and its facilities to make meaningful comments, and I suspect few Lisp
> people are familiar enough with Forth to argue usefully. Why don't we
> just stop before it turns into some futile war of attrition?
I do know Forth (and like it), but I agree completely that the previous
poster does not know Lisp (some of his Forth examples are _directly_
translatable into Common Lisp), but I would not want to have such a
discussion in this newsgroup mix; it would be completely off-topic.
Such a discussion should probably be moved to comp.lang.lisp and/or
comp.lang.forth, or better yet some *.advocacy list or even email.
Followup set to c.l.l only.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Alberto Moreira <········@moreira.mv.com> writes:
> Forth, by the way, is the most powerful programming paradigm out
> there. Not even Lisp can match it.
Ridiculous troll.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Joe Marshall <···@content-integrity.com> writes:
> Alberto Moreira <········@moreira.mv.com> writes:
> > Forth, by the way, is the most powerful programming paradigm out
> > there. Not even Lisp can match it.
>
> Ridiculous troll.
I think that's accidental; all of his points have been _known_ to
apply to Lisp, at one time or another, and are true of some
implementations even today.
On the other hand, every single feature that he names as "superior"
characteristics of Forth are in fact features of practically any CL
implementation that is fancier than CLISP.
Ridiculous ignorance, yes. Whether or not it was a dishonest mistake
is a whole other question. [I sat through a "marketing presentation"
by a Smalltalk vendor a year ago that made roughly equivalent claims,
and were certainly loathe to point out that Lisp vendors could offer
highly similar sets of features...]
--
(reverse (concatenate 'string ····················@" "454aa"))
<http://www.ntlug.org/~cbbrowne/>
"Never make any mistaeks." (Anonymous, in a mail discussion about to a
kernel bug report.)
Joe Marshall wrote:
>
> Alberto Moreira <········@moreira.mv.com> writes:
>
> > Forth, by the way, is the most powerful programming paradigm out
> > there. Not even Lisp can match it.
>
> Ridiculous troll.
Show me one thing you can do in Lisp that I can't do in a
Forthlike language.
Alberto.
Alberto Moreira <········@moreira.mv.com> writes:
> Joe Marshall wrote:
> >
> > Alberto Moreira <········@moreira.mv.com> writes:
> >
> > > Forth, by the way, is the most powerful programming paradigm out
> > > there. Not even Lisp can match it.
> >
> > Ridiculous troll.
>
> Show me one thing you can do in Lisp that I can't do in a
> Forthlike language.
I never said that there are things that can be done in Lisp that
cannot be done in Forth. Forth is `turing complete' so it can do
anything any other computer language can do.
I just deny your assertion that Lisp cannot match Forth in power
which implies that there are things that can be done in Forth that
a. solve a problem in a particularly perspicuous way
b. cannot be done nearly as easily in Lisp
and furthermore suggests that there are a multitude of these things.
(the ridiculous part)
I think posting such an assertion in comp.lang.lisp is inviting
dissent. (the troll part)
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
From: Michael Parker
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91nsg8$sh5$1@nnrp1.deja.com>
> Show me one thing you can do in Lisp that I can't do in a
> Forthlike language.
Let me rephrase that: Show me one thing you can do in Lisp that I can't
do in assembler. Or some hypothetical Turing machine. Or Lisp, since
Lisp *is* a forthlike language with a different polarity on the polish
notation, and a fancier memory model.
Alberto, I'm as big a Forth fan as anybody (I helped write a
commercially successful PostScript interpreter in Forth in the late
80's), but this argument isn't going anywhere, at least not in this NG.
Lisp and Forth offer similar levels of hackability, just at different
levels.
For an interesting look at at the parallels between the two, you might
want to take a look at the Poplog system. The low-level guts of Poplog
is a forthlike 0-address stack machine with tagged data elements, and an
open stack. The code stream is available via code-generating words.
The primary language in this system is Pop-11, which is sort of
Forth-on-steroids (there's some precedence associated with some of the
words, so you can use a more infix-like notation if you wish, but the
postfix notation is also available). On this foundation they also built
Prolog, SML, Common Lisp, and Scheme compilers. It's worth a look for
anybody that's a fan of both Forth and Lisp.
Sent via Deja.com
http://www.deja.com/
Michael Parker wrote:
>
> > Show me one thing you can do in Lisp that I can't do in a
> > Forthlike language.
>
> Let me rephrase that: Show me one thing you can do in Lisp that I can't
> do in assembler. Or some hypothetical Turing machine. Or Lisp, since
> Lisp *is* a forthlike language with a different polarity on the polish
> notation, and a fancier memory model.
>
> Alberto, I'm as big a Forth fan as anybody (I helped write a
> commercially successful PostScript interpreter in Forth in the late
> 80's), but this argument isn't going anywhere, at least not in this NG.
> Lisp and Forth offer similar levels of hackability, just at different
> levels.
Michael, I'm well aware of the parallels. But my point is of a
different nature. You know, I knew I was opening a Pandora's box
when I jumped into this argument, and I'm going to stop
immediately after this post. As for the NG, I apologize: I
caught this thread from comp.arch, and I got carried away in a
moment of folly. But I promise I'm going to leave you guys alone
after this post.
But consider: the thing about Forthlike languages is, in a
nutshell, that they're not really languages but metalanguages.
Take for example the way many Schemes implement local state and
frames. You say something like
(define (make-withdraw balance)
(lambda (amount) ... ))
(define W1 (make-withdraw 100))
(this is from SICP) and you get a global environment, another
environment where balance is bound to 100, a pointer from one to
the other, parameter bindings, and more. You see, what the
programmer gets is defined by the compiler, and the semantics
you get is what you get: you have little control over the
language's semantics, or over its syntax.
Now, contrast it with a Forthlike language. I have a word to
bind a variable to a value, another word to build a vocabulary,
another word to stack or link vocabularies, another word to
resolve a symbol, and so on. I have the *choice* of what runtime
system to use, because all the building blocks are made
available to me by the language. I may decide to use them
exactly the way Scheme uses it, but then, I may not; while in
Scheme the compiler decides it for me, in Forth I decide it
myself.
So, Forth isn't so much a language to build program as it is a
language to build languages: it's not a toy, it's a Lego set
from which we can build different toys at different times.
> For an interesting look at at the parallels between the two, you might
> want to take a look at the Poplog system. The low-level guts of Poplog
> is a forthlike 0-address stack machine with tagged data elements, and an
> open stack. The code stream is available via code-generating words.
> The primary language in this system is Pop-11, which is sort of
> Forth-on-steroids (there's some precedence associated with some of the
> words, so you can use a more infix-like notation if you wish, but the
> postfix notation is also available). On this foundation they also built
> Prolog, SML, Common Lisp, and Scheme compilers. It's worth a look for
> anybody that's a fan of both Forth and Lisp.
I must admit I don't know Poplog. The zero-addressing stack
machine, as I see it, is of peripheral importance. The
cornerstones of Forthlike languages, as I see it, are of other
nature: threaded-code based incremental compilation; the concept
of interpretation where anything I want can be made a word and
associated to an EXECUTE word so that I can look at a program
basically as a loop "DO FETCH LOOKUP EXECUTE FOREVER"; the
ability to jump between compile and run time on the fly; the
dual level of interpretation; the multilevel expandability of
threaded code into machine code, again, at the programmer's
control; and the central concept that *every* piece of
functionality available to the compiler or interpreter is also
incrementally available to the runtime system; the concept that
every piece of syntax is associated to a runtime piece of
threaded or machine code.
You see, infix is easily implemented in Forth, therefore there
may be no point making it a cast-in-stone feature of the
language. For example, if "PLUS" is a word, I can make it be
infix by defining it to perform something like "FETCH LOOKUP
EXECUTE EXECUTE", that is, I can associate a Forth word to
"PLUS", then I can have it fetch the next word, look it up,
execute it (that can trigger umpteen levels of nesting, it makes
no difference), which leaves the second augend on the stack (the
first was already there!), and the second "execute" does the
sum. Now, the little program I gave you, which I call an
"interpretation", can be made to compile when in compile mode or
execute when in execute mode, you know, the same program. So,
the point being, if I can easily reach for the functionality
with a few lines of code, why cast it in stone in a language ?
I could go on, but I respect your annoyance, and that of other
people in your NG, and I'm now going to leave you guys alone.
Sorry for the inconvenience!
Alberto.
Alberto Moreira <········@moreira.mv.com> writes:
> So, Forth isn't so much a language to build program as it is a
> language to build languages: it's not a toy, it's a Lego set
> from which we can build different toys at different times.
Please go to http://www.lisp.org/table/lisp.htm and note the
first statement on that page.
The problem, and the troll, is not that you are promoting
Forth (a language I like very much). Instead, the problem
is that you have not the first clue what lisp is, nor how
lisp-like Forth is.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjae9r7566.fsf@tfeb.org>
Alberto Moreira <········@moreira.mv.com> writes:
>
> So, Forth isn't so much a language to build program as it is a
> language to build languages: it's not a toy, it's a Lego set
> from which we can build different toys at different times.
s/Forth/Lisp/ and you get a statement which is a good description of
Lisp. And *that* is why this is so annoying: Lisp and Forth are
languages which both regard building languages as a good way to solve
problems, and both provide powerful tools to do that. And in a way
they are very similar. So seeing someone who is obviously familiar
with only one (note familiarity with scheme is *not* familiarity with
Common Lisp) make comments about what the other can not do is
upsetting.
--tim
Michael Parker <··········@my-deja.com> wrote in message
·················@nnrp1.deja.com...
>
> Or Lisp, since
> Lisp *is* a forthlike language with a different polarity on the polish
> notation, and a fancier memory model.
glad you said that. i've been playing with Scheme over the weekend and the
though kept popping in my head that it was Forth run through a reversing
filter -- specially after doing the metacompiler stuff from SICP.
In article <·····················@newsfeed.slurp.net>,
bowman <······@montana.com> wrote:
>
>Michael Parker <··········@my-deja.com> wrote in message
>·················@nnrp1.deja.com...
>>
>> Or Lisp, since
>> Lisp *is* a forthlike language with a different polarity on the polish
>> notation, and a fancier memory model.
>
>glad you said that. i've been playing with Scheme over the weekend and the
>though kept popping in my head that it was Forth run through a reversing
>filter -- specially after doing the metacompiler stuff from SICP.
>
I had much the same feeling when first learning each language: the
slow realization that I really couldn't be changing the application
this fast in any of the more conventional languages I was familiar with.
The difference is that I was working on a small problem in Forth, and
a much larger one (a four-week class project) in Common Lisp. My
perception is that the difference here is primarily one of scale.
One of the things that amazed me about Forth is the amount of
functionality one could get in a small machine (16K, say), whereas
running some sort of Lisp took a lot more resources. Nowadays, I
don't program on anything where memory is a problem (like an embedded
application) so I don't even keep a Forth system around anymore.
They gave me very similar feelings, though.
--
David H. Thornley | If you want my opinion, ask.
·····@thornley.net | If you don't, flee.
http://www.thornley.net/~thornley/david/ | O-
Peter da Silva <·····@abbnm.com> wrote:
+---------------
| Provide a "lispscript" tool that lets you write, oh:
| awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
| in no more than a couple of lines of code, and you'll get people using
| Lisp for casual scripting.
+---------------
I already frequently use it for casual scripting (well, Scheme, mostly).
With only a couple of new utility macros & functions, your example could
be expressed in Common Lisp as:
(with-lines-from-file (line "/etc/passwd")
(let ((fields (string-split line :fs #\:)))
(when (string= (aref fields 5) "/sbin/nologin")
(format t "~A~%" (aref fields 0))))))
But seriously, how many "one-liners" do you *actually* write anyway?
Not many. And by the time you've done coded up something that's complex
enough to be *useful*, Perl's tricky little code-compression notation
usually expands to be about the same length as any other language, and
six weeks or six months later I'd *much* rather be reading Lisp than
decoding a Perl puzzle.
-Rob
-----
Rob Warnock, 31-2-510 ····@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043
In article <··············@fido.engr.sgi.com>,
Rob Warnock <····@rigden.engr.sgi.com> wrote:
>Peter da Silva <·····@abbnm.com> wrote:
>+---------------
>| Provide a "lispscript" tool that lets you write, oh:
>| awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>| in no more than a couple of lines of code, and you'll get people using
>| Lisp for casual scripting.
>+---------------
>
>I already frequently use it for casual scripting (well, Scheme, mostly).
>With only a couple of new utility macros & functions, your example could
>be expressed in Common Lisp as:
>
> (with-lines-from-file (line "/etc/passwd")
> (let ((fields (string-split line :fs #\:)))
> (when (string= (aref fields 5) "/sbin/nologin")
> (format t "~A~%" (aref fields 0))))))
>
>But seriously, how many "one-liners" do you *actually* write anyway?
I do this sort of thing a lot. I would not want to type four lines of
scheme instead of one line of awk.
>Not many. And by the time you've done coded up something that's complex
>enough to be *useful*, Perl's tricky little code-compression notation
>usually expands to be about the same length as any other language, and
>six weeks or six months later I'd *much* rather be reading Lisp than
>decoding a Perl puzzle.
I avoid perl, but checking through my small collection of random
awk/sed/shell scripts that were interesting enough to save I find I
can read them. Most of the time I don't save the script so reuse
is not important.
(There's someone around MIT who excels at writing dense, impenetrable
bc and dc scripts, one of my favorites of which is this one to calculate
the checksum digit of a credit card:
head -1 | sed -e 's,\(.\)\(.\),\1 2*d10/++\2+,g' \
-e 's,\([0-9]\)$,\1 2*d10/++,' -e 's,\(.*\),10 0 \110%-10%p,' | dc
)
(This is probably off-topic by now for all the newsgroups it is in
so I'll stop writing.)
--
John Carr (···@mit.edu)
From: Rahul Jain
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91hikd$125$1@joe.rice.edu>
In article <················@senator-bedfellow.mit.edu> on
<················@senator-bedfellow.mit.edu>, ···@mit.edu wrote:
> In article <··············@fido.engr.sgi.com>, Rob Warnock
> <····@rigden.engr.sgi.com> wrote:
>>I already frequently use it for casual scripting (well, Scheme, mostly).
>>With only a couple of new utility macros & functions, your example could
>>be expressed in Common Lisp as:
>>
>> (with-lines-from-file (line "/etc/passwd")
>> (let ((fields (string-split line :fs #\:)))
>> (when (string= (aref fields 5) "/sbin/nologin")
>> (format t "~A~%" (aref fields 0))))))
>>
>>But seriously, how many "one-liners" do you *actually* write anyway?
>
> I do this sort of thing a lot. I would not want to type four lines of
> scheme instead of one line of awk.
Maybe what we need is a set of lisp string-manipulation macros. I'm sure
they wouldn't be too much harder to use than perl, and we could
definitely make them quite powerful with the flexibility that keyword
args and dynamic scoping provide. Anyone interested in starting a project
to do this?
--
-> -/- - Rahul Jain - -\- <-
-> -\- http://linux.rice.edu/~rahul -=- ·················@usa.net -/- <-
-> -/- "I never could get the hang of Thursdays." - HHGTTG by DNA -\- <-
|--|--------|--------------|----|-------------|------|---------|-----|-|
Version 11.423.999.220020101.23.50110101.042
(c)1996-2000, All rights reserved. Disclaimer available upon request.
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91gb5u$1re@web.nmti.com>
In article <··············@fido.engr.sgi.com>,
Rob Warnock <····@rigden.engr.sgi.com> wrote:
> But seriously, how many "one-liners" do you *actually* write anyway?
Five or ten a day. And they're all useful right away. that's the point.
> Not many. And by the time you've done coded up something that's complex
> enough to be *useful*, Perl's tricky little code-compression notation
> usually expands to be about the same length as any other language, and
> six weeks or six months later I'd *much* rather be reading Lisp than
> decoding a Perl puzzle.
I quite agree, but I *like* lisp. I'm just explaining why a lot of people
never get started with it, they use Perl or the equivalent, and as someone
else mentioned they keep on using it way past the point of diminishing
returns.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
In article <··············@fido.engr.sgi.com>,
Rob Warnock <····@rigden.engr.sgi.com> wrote:
>Peter da Silva <·····@abbnm.com> wrote:
>+---------------
>| Provide a "lispscript" tool that lets you write, oh:
>| awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>| in no more than a couple of lines of code, and you'll get people using
>| Lisp for casual scripting.
>+---------------
>
>I already frequently use it for casual scripting (well, Scheme, mostly).
>With only a couple of new utility macros & functions, your example could
>be expressed in Common Lisp as:
>
> (with-lines-from-file (line "/etc/passwd")
> (let ((fields (string-split line :fs #\:)))
> (when (string= (aref fields 5) "/sbin/nologin")
> (format t "~A~%" (aref fields 0))))))
>
>But seriously, how many "one-liners" do you *actually* write anyway?
An average of at least one every single working day.
And I am a pure developer, I do essentially no system
administration. A sysadmin probably writes a one-liner in some
scripting language for at least 1 in 10 commands they type
per day.
The line between '1 line program' and 'a moderately complex
command or pileline' is a little fuzzy, especially on a Unix box
(by design), but if you applied a 'most people would consider it..'
test I bet you'd come up with 1 in 10, at least.
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkj7l4yqcn5.fsf@tfeb.org>
····@rigden.engr.sgi.com (Rob Warnock) writes:
>
> But seriously, how many "one-liners" do you *actually* write anyway?
> Not many. And by the time you've done coded up something that's complex
> enough to be *useful*, Perl's tricky little code-compression notation
> usually expands to be about the same length as any other language, and
> six weeks or six months later I'd *much* rather be reading Lisp than
> decoding a Perl puzzle.
>
I write hundreds and hundreds or, rather, I write complex commandlines
which do this kind of thing all the time.
But the argument that because these take longer in Lisp than they
would in sh/awk/perl/whatever then Lisp is no good is a bit dubious.
They take longer in C too, but no one is claiming that C is not a
useful language as a result. I don't think Lisp is a good solution to
these problems any more than I think C is.
--tim
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91l7q2$4am@web.nmti.com>
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
> But the argument that because these take longer in Lisp than they
> would in sh/awk/perl/whatever then Lisp is no good is a bit dubious.
Then it's a good thing that nobody's making that argument.
It's really much simpler than that. People tend to be conservative. They build
on what they already know, and if they already know Perl then they will tend
to go on working in it rather than switching to something else. Sure, I went
through a period where I'd implement "Adventure" in every new language that
came along, but most people aren't that... well, there's a variety of words
for that kind of behaviour.
The bottom line, though, is people get into perl very easily. There's Perl
hackers hanging around all the classrooms whispering "hey, kid, wanna write
a script... first one's free". It would just be keen if there was something
like that with Lisp, other than GNU Emacs.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkj7l4xg3nw.fsf@tfeb.org>
[followups set to comp.lang.lisp]
·····@abbnm.com (Peter da Silva) writes:
> In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
> > But the argument that because these take longer in Lisp than they
> > would in sh/awk/perl/whatever then Lisp is no good is a bit dubious.
>
> Then it's a good thing that nobody's making that argument.
>
I think I misphrased my statement. `no good' should have been
something like `has no hook' (I can't think of the right term). And I
think people (though perhaps not you) *are* saying more-or-less that,
because they're bringing up things like guile and scsh.
What I'm trying to say is that language X does not have to be a good
tool for writing shell-script-style programs for it to have a `hook' to
attract people: it can have other hooks. C and Java are the classic
examples -- C's hook was not unix scripting, but Unix systems
programming; Java's hook was the web. I have no idea what C++'s hook
was (OO while being compatible with C? sounds pretty weak).
*I* think that if Lisp does have a hook then it definitely is not
shell-script type programs.
--tim
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91lckp$72f@web.nmti.com>
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
> What I'm trying to say is that language X does not have to be a good
> tool for writing shell-script-style programs for it to have a `hook' to
> attract people: it can have other hooks.
That's true. And the hook can change over time. For C it used to be that
it was a reasonable systems programming language that could be implemented
fairly easily on the systems of the time. Now it's that there's all these
other people using it who can help you/work for you/provide code you can
work on...
For Java and C++, it's that they're familiar to C coders. God knows there's
little else to recommend them. Java and the web? I think you're thinking of
Javascript... I have Java disabled on my browser and maybe once a month I have
to turn it on... Javascript, though, I can't get by without. And Javascrpt has
little to do with Java, other than being an OO langugae with C-style syntax:
the object model is completely different.
> *I* think that if Lisp does have a hook then it definitely is not
> shell-script type programs.
Then what? What's going to get your average naive young geek interested in it?
What problems does it solve for someone just starting out, and how do you
convince them to try it?
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkj66khfx8x.fsf@tfeb.org>
·····@abbnm.com (Peter da Silva) writes:
> For Java and C++, it's that they're familiar to C coders. God knows
> there's little else to recommend them. Java and the web? I think
> you're thinking of Javascript... I have Java disabled on my browser
> and maybe once a month I have to turn it on... Javascript, though, I
> can't get by without. And Javascrpt has little to do with Java,
> other than being an OO langugae with C-style syntax: the object
> model is completely different.
Oh, I don't necessarily think that Java is/was actually particularly
useful for web programming. But Sun did a *very* good job of selling
it as such. Hooks like this don't have to be true or good reasons to
use the language, they just have to work. Actually I suspect some of
the reasons people use Java now are that C++ projects are coming
unstuck all over the place and Java looks less toxic than C++ to
people running these projects, while not being too alien, so C++ is
Java's hook.
> Then what? What's going to get your average naive young geek
> interested in it? What problems does it solve for someone just
> starting out, and how do you convince them to try it?
I don't know: if I did I'd be doing it.
One thing I think is important though -- if you want a hook for some
language or system, it's probably not a good idea to use a hook which
is already used by some well-established language: you need to find a
new space to occupy. Of course this can work -- I guess it did for
perl which beat awk &c, but it seems to me that it's better to try
some new thing. That's why Java did so well: there was this huge
`new' thing -- the internet -- and Java was The Language For The
Internet Age. Whether it actually *was* or not is another question of
course. This is why I don't think scripting is a way in for Lisp --
there are already well-established languages there, perl being the
most well-known, and I really doubt if Lisp will displace those.
Similarly for web stuff -- Lisp is actually very well-suited to
server-side web applications in lots of ways, and if someone had been
smart enough to really push it 4 or 5 years ago it might have done
well. But now there are n different languages & systems in this area
all fighting for mindshare, and it's just too late: there's no big win
in mindshare left to have (especially since the whole dot.com bubble
has deflated if not burst).
--tim
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
> ...
>Oh, I don't necessarily think that Java is/was actually particularly
>useful for web programming. But Sun did a *very* good job of selling
>it as such. Hooks like this don't have to be true or good reasons to
>use the language, they just have to work. Actually I suspect some of
wasn't the hook for java that it had a vm implementation that was
supposed to be abele to run 'anywhere'?
> ...
hs
·····@abbnm.com (Peter da Silva) writes:
> in no more than a couple of lines of code, and you'll get people using
> Lisp for casual scripting. Once they're doing that, they'll keep using it
> for more complex projects. Or come up with a killer web page creation
> tool written in lisp. Something to get people started on it.
Speaking of killer web page creation tools, see IMHO at:
http://alpha.onshore.com/lisp-software
It's not just a "page" creation tool, rather it's more like an web
application server toolkit. It works well with UncommonSQL, an SQL
database interface with drivers for PostgreSQL and Oracle.
I know I keep bringing this up in different forums I read, but the fact
that people consistently ask for something like it indicates to me
that it might do the community some good to know about it. Thre
archives of the mailing list mentioned in my .sig is also useful for
finding lisp based web development tools.
--
Craig Brozefsky <·····@red-bean.com>
LispWeb -- Mailing list for Lisp Based Web Development
Homepage: http://www.red-bean.com/lispweb
·····@abbnm.com (Peter da Silva) writes:
> Provide a "lispscript" tool that lets you write, oh:
> awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
For this sort of use, you'd probably want equivalents of Perl's -n, -p
and -i options, which autogenerate lumps of code which perform the
specified code for each line of a file, print the transformed lines
and modify files in place. You might want a -c that wrapped a (cond
...) around the given code too, so you could do matches like awk.
lispscript -nc -e '((equal (field ":" 6) "/sbin/nologin") (print line))' \
/etc/passwd
--
Adam Sampson
···@gnu.org
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91ihur$slj@web.nmti.com>
In article <··············@cartman.azz.net>, Adam Sampson <···@gnu.org> wrote:
> ·····@abbnm.com (Peter da Silva) writes:
> > Provide a "lispscript" tool that lets you write, oh:
> > awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
> For this sort of use, you'd probably want equivalents of Perl's -n, -p
> and -i options, which autogenerate lumps of code which perform the
> specified code for each line of a file, print the transformed lines
> and modify files in place. You might want a -c that wrapped a (cond
> ....) around the given code too, so you could do matches like awk.
> lispscript -nc -e '((equal (field ":" 6) "/sbin/nologin") (print line))' \
> /etc/passwd
Now *that* has potential. Damn near as compact, and a lot more readable.
Just don't call the interpreter "ls". :->
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Peter da Silva wrote:
> Because you can write two lines of Perl and have a program that does something
> useful on just about any popular OS widely used today.
>
> Provide a "lispscript" tool that lets you write, oh:
>
> awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>
> in no more than a couple of lines of code, and you'll get people using
> Lisp for casual scripting. Once they're doing that, they'll keep using it
> for more complex projects. Or come up with a killer web page creation
> tool written in lisp. Something to get people started on it.
Two remarks.
- In practice, why bother typing that? Just say
egrep /sbin/nologin /etc/passwd
if you want a list you can eyeball. If you want to do something
more interesting with the userIDs, you're not talking about a
one-liner any more.
- If you're going to do a lot of /etc/passwd hacking like that,
stick some suitable definitions in whatever file your Lisp
reads at startup so that you can feed Lisp something like
(with-passwd-fields
(when (equal shell "/sbin/nologin") (print userid)))
which saves you having to remember which field number is
which. It's a few more characters than the awk solution,
but much less error-prone.
--
Gareth McCaughan ················@pobox.com
sig under construc
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91iire$t04@web.nmti.com>
In article <······························@g.local>,
Gareth McCaughan <················@pobox.com> wrote:
> - In practice, why bother typing that? Just say
> egrep /sbin/nologin /etc/passwd
> if you want a list you can eyeball.
Because the resulting list is harder to cut-and-paste into a report, feed
to mail, or whatever.
> If you want to do something
> more interesting with the userIDs, you're not talking about a
> one-liner any more.
But you're also not necessarily talking about something you can do easily in
a chunk of lisp, compared to gluing the output together with another program.
> - If you're going to do a lot of /etc/passwd hacking like that,
Problem is the next file I'm doing this on is probably not going to be
/etc/passwd.
> stick some suitable definitions in whatever file your Lisp
> reads at startup so that
So that you have to make sure that each of the 150 computers here has
all the right things in root's lisp startup file, and that none of the
other sysadmins munge it, and when I'm working a problem in someone
else's account I remember to explicitly load that file.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Voice in the desert: Quiet, isn't it, Peter da Silva?
> So that you have to make sure that each of the 150 computers here has
> all the right things in root's lisp startup file, and that none of the
> other sysadmins munge it, and when I'm working a problem in someone
> else's account I remember to explicitly load that file.
Hmm. Doesn't Perl have the same problem? It does here. In fact, it
applies to everything with a config file.
I prefer to explicitly load most code, but that's a different issue.
(I used to have a C header called portab.h for defining basic macros
that I used in every C program. Now I declare everything locally.)
One of the risks of using other peoples' code is that they may have
forgotten a quirk of their setup that may differ from, say, mine.
Is there such a thing as a standard environment? How often do you
write code that assumes details beyond that standard? Every time you
do, that's something to either break on another machine, or something
you must document/install.
It's tedious building and syncing environments, but that's just admin.
--
Email address intentially munged | You can never browse enough
will write code that writes code that writes code for food
<URL:http://www.wildcard.demon.co.uk>
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91iuun$41d@web.nmti.com>
In article <··························@news.demon.co.uk>,
Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk> wrote:
> Voice in the desert: Quiet, isn't it, Peter da Silva?
> > So that you have to make sure that each of the 150 computers here has
> > all the right things in root's lisp startup file, and that none of the
> > other sysadmins munge it, and when I'm working a problem in someone
> > else's account I remember to explicitly load that file.
> Hmm. Doesn't Perl have the same problem?
Only if you're in a habit of writing scripts that depend on stuff that
isn't the same from one installation to the next. It doesn't take long to
realise that changing any of the default settings is a bad idea.
> Is there such a thing as a standard environment?
You'd be surprised.
> How often do you write code that assumes details beyond that standard?
Almost never. Porting elm to Xenix-286 is a great way to have that lesson
drilled into you.
> It's tedious building and syncing environments, but that's just admin.
That's one reason I resist co-worker's requests to install custom software
on deliverable boxes. Unless the standard tools simply don't work, we build
our systems in a vanilla environment.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Voice in the desert: Quiet, isn't it, Peter da Silva?
> > Hmm. Doesn't Perl have the same problem?
>
> Only if you're in a habit of writing scripts that depend on stuff that
> isn't the same from one installation to the next. It doesn't take long to
> realise that changing any of the default settings is a bad idea.
I agree that tweaking default settings can create problems, but
environments themselves can vary. This is true even for different
distributions of Linux, but Perl runs on a lot of other systems, too.
As for other languages...There are many useful things that are beyond
the scope of a language standard.
> > Is there such a thing as a standard environment?
>
> You'd be surprised.
Yes, I would. See my comment above.
> > How often do you write code that assumes details beyond that standard?
>
> Almost never. Porting elm to Xenix-286 is a great way to have that lesson
> drilled into you.
Porting code is indeed a great way to learn important lessons. Code
written in C? ANSI C says little about the OS. As for Perl...
There's code in CPAN that makes assumptions about platform details. Of
course, this helps makes it useful for admin tasks.
> > It's tedious building and syncing environments, but that's just admin.
>
> That's one reason I resist co-worker's requests to install custom software
> on deliverable boxes. Unless the standard tools simply don't work, we build
> our systems in a vanilla environment.
Standard tools? Hmm. No such thing. There may be a standard for Unix,
or perhaps a collection of standards (defacto and official, POSIX,
GNU, etc), and MS have theirs, but the world is bigger. Even the world
of Perl is big enough to reveal variations in the environment.
Even if we just assume Linux, how many HOWTO docs are there that list
different directory layouts between distributions? Enough. Code that
works with one version of a distribution may fail on a later version.
Perhaps this only tells us what a toy Linux is. ;) I hope that systems
like Solaris are more stable. Anyway, automake exists for a damn good
reason. It even works on Windows, via Cygwin. However, I don't expect
it to work for every platform that Perl is available for.
Otherwise, I think we agree.
We're drifting away from Lisp, here. Followup set to comp.arch.
--
Email address intentially munged | You can never browse enough
will write code that writes code that writes code for food
<URL:http://www.wildcard.demon.co.uk>
From: Peter da Silva
Subject: Could CDR-coding be on the way back?
Date:
Message-ID: <91jf4d$aoe@web.nmti.com>
In article <··························@news.demon.co.uk>,
Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk> wrote:
> Voice in the desert: Quiet, isn't it, Peter da Silva?
> > > Hmm. Doesn't Perl have the same problem?
> > Only if you're in a habit of writing scripts that depend on stuff that
> > isn't the same from one installation to the next. It doesn't take long to
> > realise that changing any of the default settings is a bad idea.
> I agree that tweaking default settings can create problems, but
> environments themselves can vary. This is true even for different
> distributions of Linux, but Perl runs on a lot of other systems, too.
Sure does. I don't use Linux. At all. But Perl or Tcl ship *by default* on an
increasing number of commercial OSes. And the differences between Perl
implementations on different platforms has negligable effect on scripting.
The same is true of Perl, Tcl, Python, and the bevy of tools that are
collectively known as "shell scripts". It's not true of "the code I stuck
into my Lisp configuration file".
> > That's one reason I resist co-worker's requests to install custom software
> > on deliverable boxes. Unless the standard tools simply don't work, we build
> > our systems in a vanilla environment.
> Standard tools? Hmm. No such thing.
Sure there is. Every OS has a set of standard tools they ship with. If I know
the guy at the other end of the phone is running Tru64 4.0E I know that he's
got a /usr/bin/tclsh and a /bin/awk. I can talk him through problems with
those.
> Even if we just assume Linux, how many HOWTO docs are there that list
> different directory layouts between distributions?
That's one of the reasons I don't *DO* Linux.
Followups set back to comp.lang.lisp... this has more to do with Lisp than
it does with computer architecture.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Voice in the desert: Quiet, isn't it, Peter da Silva?
> And the differences between Perl implementations on different platforms
> has negligable effect on scripting.
That may depend on what your scripts do.
> The same is true of Perl, Tcl, Python, and the bevy of tools that are
> collectively known as "shell scripts". It's not true of "the code I stuck
> into my Lisp configuration file".
Your Lisp code differs from mine, then. Very little of my Common Lisp
environment depends on the platform. Same with my Scheme environment,
altho that may contain a little more OS dependant code (mainly to do
the things that CL does for me, like paths.) My C code, of which I
write very little these days, has always tended to contain a small
percentage of OS specific code, for pretty much the same reasons.
> > Standard tools? Hmm. No such thing.
>
> Sure there is. Every OS has a set of standard tools they ship with.
Not every OS uses the _same_ tools. Thank goodness. (Some systems
don't even have a command line.)
> > Even if we just assume Linux, how many HOWTO docs are there that list
> > different directory layouts between distributions?
>
> That's one of the reasons I don't *DO* Linux.
[grin]
> Followups set back to comp.lang.lisp... this has more to do with Lisp than
> it does with computer architecture.
Hmm. Gimme a LispOS and I might agree. ;)
The only environment I was truely happy with was Forth, partly because
it booted without an OS but mainly because I could change anything and
everything. _That_ was my "standard" environment, 16 years ago.
Then I took another look at Lisp...
--
Email address intentially munged | You can never browse enough
will write code that writes code that writes code for food
<URL:http://www.wildcard.demon.co.uk>
>>>>> "Peter" == Peter da Silva <·····@abbnm.com> writes:
Peter> In article <··························@news.demon.co.uk>,
Peter> Martin Rodgers <···@thiswildcardaddressintentiallyleftmunged.demon.co.uk> wrote:
>> Standard tools? Hmm. No such thing.
Peter> Sure there is. Every OS has a set of standard tools they ship
Peter> with.
True, but there may be considerable differences between the
implementation of standard tools between varioous OS's; examples
include even such standard things as `make', `m4', `awk', and `sed'
which has significant differences (at least between Linux and Solaris,
which are the two OS's I am most familiar with).
Now `perl' and `tcl' differs very little between platforms, but this
is not very surprising since they have the same software base being
ported to different platforms, whereas Solaris `sed' and Linux `sed'
has been independently developed.
This probably is also a contributing factor to the success of `perl'
and `tcl'. For instance, `perl' looks like nothing else so there is
little incentive to reimplement it. At the same time, if you write a
solution using `perl', it would readily work everywhere whereas a
scripted solution combining a number of lowerlevel tools more often
than not runs into compability problems when moved to other platforms,
giving programmers the incentive to use `perl' even in situations where
there is a simple commandline pipe to do the same.
------------------------+-----------------------------------------------------
Christian Lynbech | Ericsson Telebit, Skanderborgvej 232, DK-8260 Viby J
Phone: +45 8938 5244 | email: ···@tbit.dk
Fax: +45 8938 5101 | web: www.ericsson.com
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
- ·······@hal.com (Michael A. Petonic)
Peter da Silva wrote:
> In article <······························@g.local>,
> Gareth McCaughan <················@pobox.com> wrote:
> > - In practice, why bother typing that? Just say
> > egrep /sbin/nologin /etc/passwd
> > if you want a list you can eyeball.
>
> Because the resulting list is harder to cut-and-paste into a report, feed
> to mail, or whatever.
It's just as easy to cut-and-paste into a report or into
a mail message. If by "feed to mail" you mean "give to a
mailer as a list of people to send to" then I agree, but
sending mail to all the people who *can't* use your system
isn't often what you want to do.
Perhaps what you mean is "something that contains only
the usernames will look nicer". That's certainly true,
but if what you want is looking nice then a list of
usernames one-per-line probably isn't optimal either.
> > If you want to do something
> > more interesting with the userIDs, you're not talking about a
> > one-liner any more.
>
> But you're also not necessarily talking about something you can do easily in
> a chunk of lisp, compared to gluing the output together with another program.
And you're not necessarily talking about something you can do
easily by gluing the output together with another program, compared
to using a chunk of Lisp.
> > - If you're going to do a lot of /etc/passwd hacking like that,
>
> Problem is the next file I'm doing this on is probably not going to be
> /etc/passwd.
Depends what "this" means, of course. If you spend a lot of
time splitting files into lines and lines into fields, then
the strategy I described applies. As well as the rather specific
WITH-PASSWD-LINES macro (or whatever I called it), you would
also have WITH-SPLIT-LINES or something:
(with-split-lines ("/etc/passwd" ":")
(when (equal (field 6) "/sbin/nologin") (print line)))
Maybe I should demonstrate that this stuff is feasible.
The following code is oversimplified a bit to make it clearer.
For instance, the SPLIT-STRING function is inefficient and
could be made somewhat more general. In practice you'd want
to do it properly and hide it away in a file you never need
to refer to. :-)
(defmacro with-file-lines (filename &body body)
`(with-open-file (file ,filename :direction :input)
(loop for line = (read-line file nil)
while line
do ,@body)))
(defun split-string (string separator)
(coerce
(loop for i = 0 then (+ j (length separator))
for j = (search separator string :start2 i)
collect (subseq string i j)
while j)
'vector))
(defmacro with-split-lines ((filename separator) &body body)
`(with-file-lines ,filename
(let ((fields (split-string line ,separator)))
(flet ((field (n) (if (< n (length fields))
(aref fields n)
"")))
,@body))))
(defmacro with-passwd-fields (&body body)
`(with-split-lines ("/etc/passwd" ":")
(let ((name (field 0))
(password (field 1))
(uid (field 2))
(gid (field 3))
(gecos (field 4))
(home-dir (field 5))
(shell (field 6)))
,@body)))
Writing this (which took, oooh, all of 5 or 10 minutes) revealed
something interesting: your original example was *wrong*, at least
for my system. The "username" and "shell" fields are 6 apart, not 5.
(I don't know offhand how awk numbers its fields, so I don't know
whether it was the $1 or the $6 that was wrong.) So the order of
things is now "write one-liner, run it and see that it's wrong,
fix it and run it again". Slightly less streamlined. And if that
one-liner is plumbed into something more complicated, you might
not even know you'd typed "5" instead of "6" until it was too late...
I'd estimate that if I were writing a lot of one-liners then
updating a file of utilities like the ones above would enable
me to write 90% of the one-liners almost as briefly in Lisp
as in perl after I'd spent, say, a couple of hours on the
utilities. Would that make it worthwhile? I don't know; I don't
spend much of my time writing one-liners, so I don't have much
incentive to Lispify them. What it means is that the total
extra time spent as a result of writing one-liners in Lisp
rather than perl would be at most on the order of a few hours.
> > stick some suitable definitions in whatever file your Lisp
> > reads at startup so that
>
> So that you have to make sure that each of the 150 computers here has
> all the right things in root's lisp startup file, and that none of the
> other sysadmins munge it, and when I'm working a problem in someone
> else's account I remember to explicitly load that file.
Not quite. You write a little script that calls Lisp and loads
the init file, and put it somewhere that's in everyone's path.
Yes, you're dependent on other sysadmins not breaking it. If
you can't trust them not to do that then you have higher-priority
problems than finding the neatest way to grovel through /etc/passwd.
--
Gareth McCaughan ················@pobox.com
sig under construc
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91l8em$4ll@web.nmti.com>
In article <·······························@g.local>,
Gareth McCaughan <················@pobox.com> wrote:
> I'd estimate that if I were writing a lot of one-liners then
> updating a file of utilities like the ones above would enable
> me to write 90% of the one-liners almost as briefly in Lisp
> as in perl after I'd spent, say, a couple of hours on the
> utilities.
Fair enough. Here's a suggestion... take those utilities, package them up
with a small lisp or scheme interpreter, and call it "lispscript". Push
it out to all the usual free software sites. Then you'll have a hook for
people to get into Lisp with.
Because nobody's going to write them who doesn't know Lisp first. So it's
not a really effective way to convince people to use Lisp instead of Perl.
> What it means is that the total
> extra time spent as a result of writing one-liners in Lisp
> rather than perl would be at most on the order of a few hours.
So your young, naive system support guy asks... "A couple of hours, on
top of a couple of weeks to get effective enough in Lisp to do it, or I
can copy this line of code from Randall Schwartz' column in ;login:. What
to do, what to do..."
> Not quite. You write a little script that calls Lisp and loads
> the init file, and put it somewhere that's in everyone's path.
I'm not the guy who's complaining that people use Perl instead of Lisp. I'm
the guy explaining how it happens. I use Tcl... it's got a lot of the nice
features of Lisp, and I didn't have to write it myself, and c|o|m|p|a|q ships
it in /usr/bin.
Oh, and I don't get to dictate people's paths to them.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
(What's this doing in comp.arch, anyway?)
Peter da Silva wrote:
[I said:]
> > I'd estimate that if I were writing a lot of one-liners then
> > updating a file of utilities like the ones above would enable
> > me to write 90% of the one-liners almost as briefly in Lisp
> > as in perl after I'd spent, say, a couple of hours on the
> > utilities.
>
> Fair enough. Here's a suggestion... take those utilities, package them up
> with a small lisp or scheme interpreter, and call it "lispscript". Push
> it out to all the usual free software sites. Then you'll have a hook for
> people to get into Lisp with.
>
> Because nobody's going to write them who doesn't know Lisp first. So it's
> not a really effective way to convince people to use Lisp instead of Perl.
Clearly true. Maybe we're at cross purposes here, I thought
you were arguing, roughly, "Lisp is no good for this sysadmin
stuff because you can't write one-liners with it". If that
wasn't your point, then I've probably been wasting my time
disagreeing with it. Too bad.
> > What it means is that the total
> > extra time spent as a result of writing one-liners in Lisp
> > rather than perl would be at most on the order of a few hours.
>
> So your young, naive system support guy asks... "A couple of hours, on
> top of a couple of weeks to get effective enough in Lisp to do it, or I
> can copy this line of code from Randall Schwartz' column in ;login:. What
> to do, what to do..."
Sure.
> > Not quite. You write a little script that calls Lisp and loads
> > the init file, and put it somewhere that's in everyone's path.
>
> I'm not the guy who's complaining that people use Perl instead of Lisp. I'm
> the guy explaining how it happens.
Well, if all we need is an explanation for the fact that people
use perl, there's no need to look at technical issues at all
(beyond verifying that it at least sort-of-works). Everyone in
the sysadmin world uses perl, so newcomers get told to learn it,
and every book in sight has examples of how to do common sysadmin
tasks in perl, and so on and on. The technical strengths and
weaknesses of perl barely come into it.
> I use Tcl... it's got a lot of the nice
> features of Lisp, and I didn't have to write it myself, and c|o|m|p|a|q ships
> it in /usr/bin.
>
> Oh, and I don't get to dictate people's paths to them.
I wasn't thinking of dictating people's paths. I was thinking of
installing a program somewhere that happens to be in everyone's
path. /usr/local/bin or something. We were, I thought, talking
about sysadmins, who almost by definition get to put things in
/usr/local/bin if they need to.
--
Gareth McCaughan ················@pobox.com
sig under construc
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <92g7k4$4q8@web.nmti.com>
In article <·······························@g.local>,
Gareth McCaughan <················@pobox.com> wrote:
> > > Not quite. You write a little script that calls Lisp and loads
> > > the init file, and put it somewhere that's in everyone's path.
> > I'm not the guy who's complaining that people use Perl instead of Lisp. I'm
> > the guy explaining how it happens.
> Well, if all we need is an explanation for the fact that people
> use perl, there's no need to look at technical issues at all
> (beyond verifying that it at least sort-of-works). Everyone in
> the sysadmin world uses perl, so newcomers get told to learn it,
> and every book in sight has examples of how to do common sysadmin
> tasks in perl, and so on and on. The technical strengths and
> weaknesses of perl barely come into it.
That's the result, not the cause.
> I wasn't thinking of dictating people's paths. I was thinking of
> installing a program somewhere that happens to be in everyone's
> path.
I don't have any such place.
> /usr/local/bin or something. We were, I thought, talking
> about sysadmins, who almost by definition get to put things in
> /usr/local/bin if they need to.
There is no /usr/local/bin on delivered systems unless there's some specific
application that's needed for that system that isn't part of our standard load.
Testbeds end up with the same configuration as the system they're testing.
I understand some companies are less rigorous about developer's environments,
but we can't afford all the support costs that would result.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "A well-rounded geek should be able to geek about anything."
-- ·······@esperi.org
Disclaimer: WWFD?
Peter da Silva wrote:
[I said:]
> > Well, if all we need is an explanation for the fact that people
> > use perl, there's no need to look at technical issues at all
> > (beyond verifying that it at least sort-of-works). Everyone in
> > the sysadmin world uses perl, so newcomers get told to learn it,
> > and every book in sight has examples of how to do common sysadmin
> > tasks in perl, and so on and on. The technical strengths and
> > weaknesses of perl barely come into it.
>
> That's the result, not the cause.
It's the result of the perl-not-lisp culture in the past,
and the cause of the perl-not-lisp culture in the future.
It seems clear to me that non-technical reasons played a
large part in determining the culture of the past, just as
they play a large part today.
Note: I am not saying that Lisp is a better language than perl
for system administration. I'm not even sure that such a claim,
in either direction, is really meaningful. I'm just saying
that to answer the question "how come everyone uses X and no one
uses Y" one needs to look just as hard at social as technical
issues. Perhaps technical reasons explain why perl has (for many
people) won out over awk and sed and other such tools, but they
don't explain why it's won out over, say, Common Lisp and OCaml
and Erlang and Ada, because most people who use perl haven't
even considered using those languages.
And no, I am not suggesting that Ada is a better tool than perl
for anything a sane person would consider doing in perl.
> > I wasn't thinking of dictating people's paths. I was thinking of
> > installing a program somewhere that happens to be in everyone's
> > path.
>
> I don't have any such place.
>
> > /usr/local/bin or something. We were, I thought, talking
> > about sysadmins, who almost by definition get to put things in
> > /usr/local/bin if they need to.
>
> There is no /usr/local/bin on delivered systems unless there's some specific
> application that's needed for that system that isn't part of our standard load.
>
> Testbeds end up with the same configuration as the system they're testing.
>
> I understand some companies are less rigorous about developer's environments,
> but we can't afford all the support costs that would result.
If you're saying what I think you are, then *provably* you
can't use Lisp, or indeed anything that isn't part of the
already-decided configuration of these systems. I'm not
sure what's supposed to follow from this, except that if
you are required to use a system in a certain configuration
that doesn't include Lisp then you shouldn't use Lisp. (Gosh.)
The same would, of course, apply to perl or tcl or anything
else.
*
I am not sure this discussion is going anywhere. Shall we drop it?
--
Gareth McCaughan ················@pobox.com
sig under construc
················@pobox.com (Gareth McCaughan) writes:
> Note: I am not saying that Lisp is a better language than perl for
> system administration. I'm not even sure that such a claim, in
> either direction, is really meaningful.
It probably isn't. What _is_ meaningful is what people can grok, what
works on most platforms, and being free. Perl was, to my knowledge,
the first language of its kind to really build a community and attack
the problems that sysadmins and web scripters had. When it was
gaining momentum, it was competing with C for CGI (how could it lose?)
Perl, with all its idiosyncrasies, is much easier for most people to
become useful than Common Lisp. More people know it, there's more
stuff out there, it's much better organized, it's more uniform, more
free in general, and more active in terms of solving problems in a
_wider_ variety than Common Lisp. For example, I take myself. I've
spent mounds more time using Common Lisp, reading up on it, learning
its nuances, and yet I find myself completely unable to figure out how
to do certain things with CL that are trivial in Perl (i.e. because
someone wrote a module). You read the synopsis of the single dispatch
multiple inheritance model that Perl follows, and it's usually enough
for a huge amount of power.
For what I'm working on now, CL happens to be the ultimate langauge.
A Perl-based solution would be a nightmare, but done right it would be
the next best thing, as far as I know.
I like Perl because it makes hard things easy and some _really_ hard
things a helluva lot easier. Lispers can fuss all they want about it,
but unless you're a Lisp guru, Perl is a great skill to have for a
Lisp programmer. Comes in handy for me; kinda makes me ``complete''
in a way. I feel incomplete with CL. Probably more to do with me
than CL, but that's the bottom line.
getting back, there are many metrics you should use when judging
language. One important one is maintainability. Perl is very
maintainable, despite what people say, as long as it's properly
written. Even my code, without `use strict;' is easily maintained,
and very readable. I just stay away from the insane shortcuts. A
really heinus macro can be harder to grok than Perl code sometimes.
I get this amazing sense (and I mention it again and again) that I
suspect that half (if not more) of the anti-Perl posts on this forum
are by people that either did not use Perl sufficiently to judge it,
or did not use it at all, and probably some who just glanced at some
Perl code and made snap decisions.
I learned Perl because I was told to; not because I wanted to. It was
forced on me through my job. I resisted it because of all the reasons
that people dislike it. But I feel that the pain in Perl is
relatively short-lived, unlike C, worse C++, and even CL. The only
thing is that I _enjoy_ the learning curve CL and Perl, and don't
enjoy it in other languages. I think that's what people here are
really talking about -- i.e. what they enjoy. Most Common Lispers
probably do not enjoy learning Perl, writing Perl, reading it, etc.
That I understand. But if that's the problem, then they should just
say that.
dave
David Bakhash <·····@alum.mit.edu> writes:
> Perl, with all its idiosyncrasies, is much easier for most people to
> become useful than Common Lisp. More people know it, there's more
> stuff out there, it's much better organized, it's more uniform, more
> free in general, and more active in terms of solving problems in a
> _wider_ variety than Common Lisp. For example, I take myself. I've
> spent mounds more time using Common Lisp, reading up on it, learning
> its nuances, and yet I find myself completely unable to figure out
> how to do certain things with CL that are trivial in Perl
> (i.e. because someone wrote a module). You read the synopsis of the
> single dispatch multiple inheritance model that Perl follows, and
> it's usually enough for a huge amount of power.
I think the library thing is really, really important. Not only that
Perl has a large array of very useful libraries, but that they are all
under the same license. You know what you can do with them.
Of course, Common Lisp is very rich in its own way; I just discovered
`rotatef' the other day. But it does feel kind of like its own closed
world sometimes - as soon as I want to use sockets or interface with
databases, I have to go outside the realm of standardized language.
Not that I am really complaining - it would take a struggle to make me
program anything over a single line in Perl again (I still use it from
the shell sometimes). I just wish I spent less time searching the
internet for free Lisp libraries for speaking modern standards like
SMTP, HTTP, SQL, and HTML.
David Bakhash <·····@alum.mit.edu> writes:
> I get this amazing sense (and I mention it again and again) that I
> suspect that half (if not more) of the anti-Perl posts on this forum
> are by people that either did not use Perl sufficiently to judge it,
> or did not use it at all, and probably some who just glanced at some
> Perl code and made snap decisions.
>
> I learned Perl because I was told to; not because I wanted to. It was
> forced on me through my job. I resisted it because of all the reasons
> that people dislike it. But I feel that the pain in Perl is
> relatively short-lived, unlike C, worse C++, and even CL. The only
> thing is that I _enjoy_ the learning curve CL and Perl, and don't
> enjoy it in other languages. I think that's what people here are
> really talking about -- i.e. what they enjoy. Most Common Lispers
> probably do not enjoy learning Perl, writing Perl, reading it, etc.
> That I understand. But if that's the problem, then they should just
> say that.
I've learned a lot of programming languages over the years, and Perl
is the only one I still can't write 5 lines straight without thumping
through the camel book looking for how they massacred the syntax for
that case again. I can live with its object model, its scalar and list
contexts, its way of defining functions and passing parameters and
most of its other oddities, but I just can't remember the syntax. It's
simply insane that you need 2 man pages to explain general
datastructures, and that putting a list into a hash table is major
wizardry. A man page to explain how to create a list of lists and you
maintain it's easier to learn than Lisp?
I need it for the modules giving access to all sorts of protocols but
I simply write cook book code to get the data from the network, write
it out in SEXPs and do the rest of the work with CL, generating and
running throw away Perl scripts to change stuff on the network if
necessary.
--
Lieven Marchand <···@village.uunet.be>
Gla�r ok reifr skyli gumna hverr, unz sinn b��r bana.
Lieven Marchand <···@village.uunet.be> writes:
> I need it for the modules giving access to all sorts of protocols
> but I simply write cook book code to get the data from the network,
> write it out in SEXPs and do the rest of the work with CL,
> generating and running throw away Perl scripts to change stuff on
> the network if necessary.
If I wern't more proficient at Perl, then I'd probably use it
minimally in this way or a similar way.
the syntax stuff is a mystery to me. I can't grok C, much less C++,
yet I get Perl. I think it has to do with declarations and pointers
that Perl doesn't have that allow me to get Perl while failing hard at
C.
If CL had similar libraries, a nice CPAN-like searchable website, a
single free implementation that worked with all the advertised
libraries (also free), and nice interfaces to the platform-dependent
stuff (e.g. primarily threads, sockets, Corba, SQL), then I'd never
write another line of Perl.
dave
David Bakhash <·····@alum.mit.edu> writes:
> the syntax stuff is a mystery to me. I can't grok C, much less C++,
> yet I get Perl. I think it has to do with declarations and pointers
> that Perl doesn't have that allow me to get Perl while failing hard
> at C.
Perl and C have some similarities in the way they handle data. For
example, in Perl, you can't put an array object in a hash table; you
have to use a "reference" to it, and explicitly dereference it when
needed. Most C hash tables I've used also expect a pointer
(reference) to the value.
> If CL had similar libraries, a nice CPAN-like searchable website, a
> single free implementation that worked with all the advertised
> libraries (also free), and nice interfaces to the platform-dependent
> stuff (e.g. primarily threads, sockets, Corba, SQL), then I'd never
> write another line of Perl.
I'd also like a pony :)
Seriously though, what bugs me the most is sockets. Lisp is so great
for manipulating and processing data, but I have to figure out the
(four or more) interfaces to sockets on the free implementations.
On 5 Jan 2001, Erik Naggum wrote:
> * Colin Walters <·········@cis.ohio-state.edu>
> | Seriously though, what bugs me the most is sockets. Lisp is so great for
> | manipulating and processing data, but I have to figure out the (four or
> | more) interfaces to sockets on the free implementations.
>
> So why not collect this information and build a more general interface on
> top of the bogus ones that seem to bother people so much? Clearly, that
> way also lies standardization, as opposed to the whining and griping that
> <whatever> is not standardized, as if Perl were _standardized_. *snicker*
The CLOCC <http://clocc.sourceforge.net/> uses its PORT collection of
cross-platform portability tools (LGPL). Notably, it contains an
interface to sockets, etc.; and multiprocessing. If you don't want to
learn N different, similar interfaces to the missing-from-the-standard
features, don't: just learn one portable one. And someone else already
did the grunt work of making the general interface :)
Incidentally, by my count, N = 3 for the free implementations (CMUCL,
CLISP, GCL/AKCL), not >= 4; am I missing any?
Tom
Thomas Francis Burdick <···@u.washington.edu> writes:
> The CLOCC <http://clocc.sourceforge.net/> uses its PORT collection
> of cross-platform portability tools (LGPL). Notably, it contains an
> interface to sockets, etc.; and multiprocessing. If you don't want
> to learn N different, similar interfaces to the
> missing-from-the-standard features, don't: just learn one portable
> one. And someone else already did the grunt work of making the
> general interface :)
Honestly I haven't actually looked at the CLOCC code yet. I do plan
to though.
> Incidentally, by my count, N = 3 for the free implementations
> (CMUCL, CLISP, GCL/AKCL), not >= 4; am I missing any?
Actually, I was counting the CLOCC interface as one of the different
versions, as well as :sockets
(http://ww.telent.net/lisp/sockets.html), which would make five.
And the Lisp package I'm using (Araneida, which is incidentally pretty
cool), seems to redefine portions of CMUCL's interface. Whee...
From: Chris Double
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <wk8zopyo3l.fsf@double.co.nz>
Colin Walters <·········@cis.ohio-state.edu> writes:
> Actually, I was counting the CLOCC interface as one of the different
> versions, as well as :sockets
> (http://ww.telent.net/lisp/sockets.html), which would make five.
There is also a sockets and multiprocessing package for Corman Lisp
at:
http://www.double.co.nz/cl
Chris.
On Fri, 5 Jan 2001 19:23:50 -0800, Thomas Francis Burdick
<···@u.washington.edu> wrote:
> Incidentally, by my count, N = 3 for the free implementations (CMUCL,
> CLISP, GCL/AKCL), not >= 4; am I missing any?
There are also at least Poplog Common Lisp and ECL-Spain.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
Paolo Amoroso <·······@mclink.it> writes:
> On Fri, 5 Jan 2001 19:23:50 -0800, Thomas Francis Burdick
> <···@u.washington.edu> wrote:
>
> > Incidentally, by my count, N = 3 for the free implementations (CMUCL,
> > CLISP, GCL/AKCL), not >= 4; am I missing any?
>
> There are also at least Poplog Common Lisp and ECL-Spain.
>
I must say that I am impressed by the Poplog Environment. The CL is
not quite ANSI, but the overall system is very nice.
Cheers
--
Marco Antoniotti =============================================================
NYU Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://galt.mrl.nyu.edu/valis
Like DNA, such a language [Lisp] does not go out of style.
Paul Graham, ANSI Common Lisp
Erik Naggum <····@naggum.net> writes:
> FWIW, Allegro CL's sockets implementation poses very little strain
> on its users. Sockets are streams and obey all expected protocol.
> I have found its functional interface to be superior to the "free"
> versions, but it's pretty bizarre that there can be so much
> variation over this simple theme.
Yes. ACL's interface to sockets (which include datagram sockets) is
really simple, and works well. I havn't been happy with just about
any other interface, except CLISP, which is really good, nice and
simple, works on MS Windows and Unix, but lacks datagram support last
I checked. CLISP is the only other implementation that does it right,
in my opinion, and keeps it simple (when it comes to simple sockets,
socket streams, and server sockets). I don't like the LispWorks
socket interface (in the COMM package) as much, but not too long ago
wrote some code to give LispWorks a similar API that ACL has (though
it uses unexported symbols, which I don't like, but without which I
couldn't seem to do it). It's at
http://mit.edu/cadet/www/socket.lisp
It's public domain stuff.
Chris Double also put out some excellent stuff at:
http://www.double.co.nz/cl
for Corman Lisp. One of the nice things about his work is that he
also wrote the corman-to-acl mapping for his socket API, similar to
the LispWorks one I wrote, though more complete. I don't know if it
appears on his site, though.
Given that He has AllegroServe, SQL, multiprocessing, and sockets
(even SSL), working in his CormanLisp, it almost seems like CormanLisp
isn't such a bad alternative to other Common Lisp implementations. I
avoid it because it's Windows only, because it lacks several ANSI CL
features, and some other reasons, but it's definitely right for the
price, in my opinion.
dave
David Bakhash <·····@alum.mit.edu> writes:
> Chris Double also put out some excellent stuff at:
>
> http://www.double.co.nz/cl
>
> for Corman Lisp. One of the nice things about his work is that he
> also wrote the corman-to-acl mapping for his socket API, similar to
> the LispWorks one I wrote, though more complete. I don't know if it
> appears on his site, though.
Not yet. I've yet to upload updates to a number of my libraries but
will get around to it soon as I get a chance to package everything
up. This includes the corman-to-acl mapping, AllegroServe port, etc.
Chris.
--
http://www.double.co.nz/cl
Peter da Silva <·····@abbnm.com> wrote:
+---------------
| Gareth McCaughan <················@pobox.com> wrote:
| > - In practice, why bother typing that? Just say
| > egrep /sbin/nologin /etc/passwd
| > if you want a list you can eyeball.
|
| Because the resulting list is harder to cut-and-paste into a report,
| feed mail, or whatever.
+---------------
Sheesh! Hey, c'mon, Peter, you know better than that:
egrep /sbin/nologin /etc/passwd | sed -e 's/:.*$//'
And it's even fewer characters to type than the Perl version!! ;-} ;-}
-Rob
-----
Rob Warnock, 31-2-510 ····@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043
····@rigden.engr.sgi.com (Rob Warnock) writes:
> Sheesh! Hey, c'mon, Peter, you know better than that:
>
> egrep /sbin/nologin /etc/passwd | sed -e 's/:.*$//'
>
> And it's even fewer characters to type than the Perl version!! ;-} ;-}
It doesn't, however, give the same answer: it will be confused if any
of the users decide to put the string "/sbin/nologin" in their GECOS
information.
I can't imagine offhand why they'd want to, without knowing what the
script is used for. Perhaps Peter is generating a list of usernames
for the accounting department to cancel billing for.
Incidentally, if you had used Perl you could have had a version that
works with NIS as well:
perl -e 'while(@line=getpwent()) {($line[8] eq "/sbin/nologin") && print $line[1]."\n"}'
-dan
--
http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91l8lu$4p2@web.nmti.com>
In article <··············@fido.engr.sgi.com>,
Rob Warnock <····@rigden.engr.sgi.com> wrote:
> Sheesh! Hey, c'mon, Peter, you know better than that:
> egrep /sbin/nologin /etc/passwd | sed -e 's/:.*$//'
> And it's even fewer characters to type than the Perl version!! ;-} ;-}
What Perl version? I haven't given a Perl version.
And what happens if someone's GCOS name is /sbin/nologin? Try:
egrep ":/sbin/nologin$" /etc/passwd | sed -e 's/:.*$//'
Yes, I know it's unlikely that someone's gonna call themselves "/sbin/nologin"
but I did once have a user call himself "/dev/null".
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
[Trimmed newsgroups, retargeted subject to match content...]
Peter da Silva <·····@abbnm.com> wrote:
+---------------
| Rob Warnock <····@rigden.engr.sgi.com> wrote:
| > Sheesh! Hey, c'mon, Peter, you know better than that:
| > egrep /sbin/nologin /etc/passwd | sed -e 's/:.*$//'
| > And it's even fewer characters to type than the Perl version!! ;-} ;-}
|
| What Perl version? I haven't given a Perl version.
+---------------
Sorry, I guess I mis-read your article <···················@web.nmti.com>
that started this whole sub-thread, where you said:
Because you can write two lines of Perl and have a program that
does something useful on just about any popular OS widely used today.
Provide a "lispscript" tool that lets you write, oh:
awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
and not being very fluent in Perl myself, I assumed that you were using
some "awk" sub-command of Perl and not the old Unix command-line Awk
program (since obsoleting Awk is one of Perl's main claims to fame).
+---------------
| And what happens if someone's GCOS name is /sbin/nologin? Try:
| egrep ":/sbin/nologin$" /etc/passwd | sed -e 's/:.*$//'
+---------------
Thanks for the bug fix.
But even with the fix, it's *still* shorter than the Awk version.
And not only that, it works regardless of how many fields (7 or 8)
the passwd file contains (as someone else was worrying about).
-Rob
-----
Rob Warnock, 31-2-510 ····@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043
Peter da Silva wrote:
> And what happens if someone's GCOS name is /sbin/nologin? Try:
>
> egrep ":/sbin/nologin$" /etc/passwd | sed -e 's/:.*$//'
>
> Yes, I know it's unlikely that someone's gonna call themselves "/sbin/nologin"
> but I did once have a user call himself "/dev/null".
If it comes to that, what if there are comments in /etc/passwd?
(I don't know whether all unices allow them, but FreeBSD certainly
does.)
This one-liner is steadily becoming longer and longer,
and the details easier and easier to forget. That's the
problem with one-liners.
--
Gareth McCaughan ················@pobox.com
sig under construc
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <92g7n9$4qi@web.nmti.com>
In article <·······························@g.local>,
Gareth McCaughan <················@pobox.com> wrote:
> Peter da Silva wrote:
> > And what happens if someone's GCOS name is /sbin/nologin? Try:
> > egrep ":/sbin/nologin$" /etc/passwd | sed -e 's/:.*$//'
> > Yes, I know it's unlikely that someone's gonna call themselves "/sbin/nologin"
> > but I did once have a user call himself "/dev/null".
> If it comes to that, what if there are comments in /etc/passwd?
Not in any /etc/passwd I have anything to do with. There's too many tools out
there already that "know" there aren't any comments in there.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "A well-rounded geek should be able to geek about anything."
-- ·······@esperi.org
Disclaimer: WWFD?
[Followup posted to comp.lang.lisp only - Paolo]
On 15 Dec 2000 23:46:03 GMT, ·····@abbnm.com (Peter da Silva) wrote:
> Provide a "lispscript" tool that lets you write, oh:
>
> awk 'BEGIN {FS=":"}; $6=="/sbin/nologin" {print $1}' /etc/passwd
>
> in no more than a couple of lines of code, and you'll get people using
> Lisp for casual scripting. Once they're doing that, they'll keep using it
Something similar to such a tool may already exist, i.e. `scsh' by Olin
Shivers:
http://www-swiss.ai.mit.edu/ftpdir/scsh/
While this is definitely not a one liner, it's a useful approximation to
the above code and is intended to be put into a file:
#!/usr/local/bin/scsh -s
!#
(call-with-input-file "/etc/passwd"
(lambda (port)
(awk ((field-reader (infix-splitter ":" 7)) port) (record fields) ()
((string=? (nth fields 5) "/sbin/nologin")
(format #t "~&~a~%" (nth fields 0))))))
Note that although scsh is based on Scheme, its design is general enough
that it can be implemented also in Lisp and possibly other languages.
> for more complex projects. Or come up with a killer web page creation
> tool written in lisp. Something to get people started on it.
Does any of the entries in this list fit your definition of killer tool?
http://ww.telent.net/cliki/Web
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
On 15 Dec 2000 11:52:39 +0000, Erik Naggum <····@naggum.net> wrote:
>* ······@nospam-cs.wisc.edu
>| Or the uniformity and simplicity of the data model (everything is a list)?
>
> It would be helpful if you could explain why you believe this "everything
> is a list" part to be true and how you came to believe it. (Hint: It has
> actually _never_ been true, and it mystifies Lisp people that so many
> people who come to Lisp hold this myth. If we could only track it down
> and find ways to replace it with something less false...)
A few months ago in an interview about emacs (at freshmeat or
slashdot, I'm not sure), RMS himself said that "in Lisp everything is a list",
that why it's so flexible and bla, bla, bla... I guess this sort of thing
helps. ;-)
//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------
> * ······@nospam-cs.wisc.edu
> | Or the uniformity and simplicity of the data model (everything is a list)?
>
> It would be helpful if you could explain why you believe this "everything
> is a list" part to be true and how you came to believe it. (Hint: It has
> actually _never_ been true, and it mystifies Lisp people that so many
> people who come to Lisp hold this myth. If we could only track it down
> and find ways to replace it with something less false...)
That's certainly how it was presented to me, when I first wrote a LISP
program in my undergraduate Data Structures class in IIRC 1977. My
text was the LISP 1.5 manual; I'd have to check (and it's at school
while I'm at home at the moment), but I remember it as presenting
S-expressions, then lists... and never mentioning any other data
structure in the language.
> The United States of America, soon a Bush league world power. Yeee-haw!
A huge step up.
--
Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
Department of Computer Science FAX -- (505) 646-1002
New Mexico State University http://www.cs.nmsu.edu/~pfeiffer
VL 2000 Homepage: http://www.cs.orst.edu/~burnett/vl2000/
······@nospam-cs.wisc.edu writes:
> In comp.arch Johan Kullstam <········@ne.mediaone.net> wrote:
> > ······@nospam-cs.wisc.edu writes:
> >> In comp.arch Duane Rettig <·····@franz.com> wrote:
> >> > ······@nospam-cs.wisc.edu writes:
> >> >> Out of curiosity: what kinds of large commercial
> >> >> applications use LISP? Do they use it as an embedded
> >> >> scripting engine, or to write the application proper?
> >>
> >> > Both are possible, as well as using lisp as glue to hold
> >> > together systems with subsystems written in different languages.
> >> > However, the applications I refer to, which _must_ use lisp for
> >> > its ability to handle large problem sets, tend to be largely
> >> > written in lisp.
> >>
> >> I guess I'm trying to figure out why LISP is unique
> >> in its ability to handle these large problem sets.
> >> What is it about these data sets that prevents use
> >> of C or C++, and *requires* LISP?
> > nothing *requires* Lisp, per se, as you can always write a common-lisp
> > in, for example, C and C++.
>
>
> [ snipped lots of good stuff about LISP ]
>
>
> Oh sure, I agree. But I'm curious as to what exactly
> the original poster meant while writing "applications [...]
> which _must_ use lisp for its ability to handle large
> problem sets".
>
> What is it about LISP that enables handling
> large data sets? Is it just the fact that objects can be
> easily serialized to storage and back (as you described
> earlier)? Or the garbage collection? Or the uniformity
> and simplicity of the data model (everything is a list)?
everything is not a list. there are a number of simple types such as
number or character. other aggregate types such as array and
hashtable exist in common-lisp.
> I guess it would be good if the original poster
> described the applications in question and explained
> how LISP helped in their implementation.
agreed.
--
J o h a n K u l l s t a m
[········@ne.mediaone.net]
Don't Fear the Penguin!
From: ······@nospam-cs.wisc.edu
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91dm94$k8v@spool.cs.wisc.edu>
In comp.arch Johan Kullstam <········@ne.mediaone.net> wrote:
> ······@nospam-cs.wisc.edu writes:
>> and simplicity of the data model (everything is a list)?
>
> everything is not a list. there are a number of simple types such as
> number or character. other aggregate types such as array and
> hashtable exist in common-lisp.
Mea culpa. Apologies to all the LISP fans out there.
I've never programmed in LISP myself, only read a bit
about it.
Manoj
"Duane Rettig" <·····@franz.com> wrote in message
··················@beta.franz.com...
> ······@best.com (Chuck Fry) writes:
>
> > And with 64 bits to play with, who's to say we can't spare a couple for
> > CDR-coding?
>
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke. These programs, which
> used the upper 8 bits of address words for tags and flags, assumed
(correctly,
> but without foresight) that these bits were available for use. However,
> whether these programs were justified in the original designs or not, it
> would be safe for me to venture a guess that _none_ of them exist in their
> previous design form, and that all have been either rewritten or scrapped.
Unfortunately, your guess would be wrong! In MVS and its successors, there
is still the concept of below/above the line, where the line is the address
(16MB) that fits within 24 bits. Some code and data structures must still
run "below the line". It is truely amazing how long a bad decision can
haunt one :-(.
--
- Stephen Fuld
Snip
>
> --
> Duane Rettig Franz Inc. http://www.franz.com/ (www)
> 1995 University Ave Suite 275 Berkeley, CA 94704
> Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig wrote:
[huge snip, lots of interesting stuff about CDR vs EQ]
> I believe that EQ testing does speak directly to the point of cdr-coding;
> if you measure the number of times CDR is taken vs the number of times
> EQ is performed in a typical program, you find that EQ usage overwhelms
> CDR usage. Thus, there is a strong impetus to optimize EQ testing
> in preference to CDR operations.
>
> Remember that any boolean test, e.g. "(when x ...)" is equivalent to
> "(unless (eq x nil) ...)", in a similar manner that in C "if (x) {...}"
> is equivalent to "if (x!=0) {...}", so there are many more EQ tests than
> meets the eye in lisp code. If you use a bit which changes the value
> of a pointer without changing its object's identity, then tests for
> identity can no longer be performed in one instruction, and lisp then
> loses to other languages...
At least on x86, this is much less costly than it would seem:
if (x & SIGNIFICANT_BITS)
takes exactly (modulo slightly increased opcode size) the same time as
if (x)
The relevant asm code would be:
test eax,eax
jnz skip
vs
test eax,SIGNIFICANT_BITS ; 32-bit immediate
jnz skip
On other architectures you would probably (semi-)permanently allocate a
register to contain that 'SIGNIFICANT_BITS' mask, which does carry some
cost.
Terje
--
- <··············@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
Terje Mathisen <··············@hda.hydro.com> writes:
> Duane Rettig wrote:
> [huge snip, lots of interesting stuff about CDR vs EQ]
> > I believe that EQ testing does speak directly to the point of cdr-coding;
> > if you measure the number of times CDR is taken vs the number of times
> > EQ is performed in a typical program, you find that EQ usage overwhelms
> > CDR usage. Thus, there is a strong impetus to optimize EQ testing
> > in preference to CDR operations.
> >
> > Remember that any boolean test, e.g. "(when x ...)" is equivalent to
> > "(unless (eq x nil) ...)", in a similar manner that in C "if (x) {...}"
> > is equivalent to "if (x!=0) {...}", so there are many more EQ tests than
> > meets the eye in lisp code. If you use a bit which changes the value
> > of a pointer without changing its object's identity, then tests for
> > identity can no longer be performed in one instruction, and lisp then
> > loses to other languages...
>
> At least on x86, this is much less costly than it would seem:
>
> if (x & SIGNIFICANT_BITS)
>
> takes exactly (modulo slightly increased opcode size) the same time as
Not a real issue, but can't resist this: 5 bytes instead of 2 bytes
is slight? :-)
> if (x)
>
> The relevant asm code would be:
>
> test eax,eax
> jnz skip
>
> vs
>
> test eax,SIGNIFICANT_BITS ; 32-bit immediate
> jnz skip
This magic works only when you are testing against integer 0. C
happens to pun zero as the null or false value, but in Common Lisp,
zero is a non-null integer and NIL is the false value (Scheme is even
more "pure", with nil being the empty list and a special false value,
both distinct from 0).
So such a test would necessarily require mask and compare instructions,
and could not use test.
Our lisp implementation keeps NIL in edi, and tests against it with
a long compare instruction, which is definitely faster than a mask
followed by a compare.
My second paragraph was intended as a reminder to lisp programmers
in case they forget how many times they do EQ comparisons. However,
EQ is a general comparison test, and is the most primitive operation
in lisp. It is in effect, a single instruction to compare between
to longwords (besides, of course, the jump instruction, although some
architectures have compare-and-jump instructions, thus eliding the final
jump at the end). It is the general comparison "instruction" that would
have to be enhanced to first mask off any bits that are "noise" with
respect to what lispers know to be two identical lisp objects. It is
this enhancement required by cdr-coding that I object to.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig wrote:
>
> Terje Mathisen <··············@hda.hydro.com> writes:
> > At least on x86, this is much less costly than it would seem:
> >
> > if (x & SIGNIFICANT_BITS)
> >
> > takes exactly (modulo slightly increased opcode size) the same time as
>
> Not a real issue, but can't resist this: 5 bytes instead of 2 bytes
> is slight? :-)
Yes, as long as this isn't the only instruction used. I.e. even if 10%
of all code was test for C NULL values, it would only increase the total
code size by 7.5%.
> > test eax,SIGNIFICANT_BITS ; 32-bit immediate
> > jnz skip
>
> This magic works only when you are testing against integer 0. C
> happens to pun zero as the null or false value, but in Common Lisp,
> zero is a non-null integer and NIL is the false value (Scheme is even
> more "pure", with nil being the empty list and a special false value,
> both distinct from 0).
>
> So such a test would necessarily require mask and compare instructions,
> and could not use test.
OK, this is much more serious!
Terje
--
- <··············@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
In article <·············@beta.franz.com>,
Duane Rettig <·····@franz.com> wrote:
>······@best.com (Chuck Fry) writes:
>
>> And with 64 bits to play with, who's to say we can't spare a couple for
>> CDR-coding?
>
>Let's not make this mistake again. When IBM (and Amdahl) created their XA
>architectures for the 370 compatibles (extending the address range from 24
>bits to 32 bits), several programs suddenly broke.
Can anyone come up with a plausible program in the next two decades, to
run on conventional architectures, that will require an address space
between 60 and 64 bits? If you say "let's map everything into a flat
space" you will need more than 64 bits. If you stick to conventional
addressing, including memory mapping individual objects, you use a lot
less than 64 bits.
--
John Carr (···@mit.edu)
···@mit.edu (John F Carr) writes:
> In article <·············@beta.franz.com>,
> Duane Rettig <·····@franz.com> wrote:
> >······@best.com (Chuck Fry) writes:
> >
> >> And with 64 bits to play with, who's to say we can't spare a couple for
> >> CDR-coding?
> >
> >Let's not make this mistake again. When IBM (and Amdahl) created their XA
> >architectures for the 370 compatibles (extending the address range from 24
> >bits to 32 bits), several programs suddenly broke.
>
> Can anyone come up with a plausible program in the next two decades, to
> run on conventional architectures, that will require an address space
> between 60 and 64 bits? If you say "let's map everything into a flat
> space" you will need more than 64 bits. If you stick to conventional
> addressing, including memory mapping individual objects, you use a lot
> less than 64 bits.
It won't take two decades.
This kind of thinking comes dangerously close to the "it'll never happen"
mantra that intelligent programmers and engineers have been (incorrectly)
reciting for at least the last 6 decades.
The mantra comes in two forms: "it can't be done", and "you'll never
use it all".
The first form is probably the most familiar, and usually occurs when
engineers see a wall in the technology they work with, past which the
technology doesn't work. It happened with vacuum tubes, magnetic core
memory, transistors, and even various forms of integrated circuits.
It has also happened in the mechanical world; I am amazed myself to see
gears and motors only a few molecules across. Are we at the limit there?
I wouldn't count on it...
The second form is more fun, because instead of picturing limits on
one's technology, we are picturing limits on one's own imagination.
My favorite example was in college, when I was in the university
computer lab, and talking to the resident guru about wanting to wait
to buy my own computer because I wanted one of those new machines with
64 kbytes of memory. His response was "what do you want with one of
those? You'll never use all that memory". Did I know myself precisely
what I would do with all that memory? No, but I knew I would find a
use...
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
On 12 Dec 2000 13:53:04 GMT, ···@mit.edu (John F Carr) said:
> Can anyone come up with a plausible program in the next two decades, to
> run on conventional architectures, that will require an address space
> between 60 and 64 bits? If you say "let's map everything into a flat
> space" you will need more than 64 bits. If you stick to conventional
> addressing, including memory mapping individual objects, you use a lot
> less than 64 bits.
18 446 744 073 709 551 616 bits
2 305 843 009 213 693 952 bytes
2 147 483 648 gigabytes
Today, a "full" Usenet feed is around 250 GB/day, and the yearly
growth is around 200%. With two decades of continued growth at the
same rate, that'll be 250*3^20 gigabytes, or:
871 696 100 250 gigabytes
So the answer is, yes, there is a plausible program that will require
far more, if it is expected to use a system with flat addressing.
Since EMC delivers systems with storage capacity well into the
terabyte range today (enough for storing a full Usenet feed for well
over a week, perhaps as much as a month, I haven't checked their
products lately), it doesn't seem unlikely that they will have
something that can store more than 2 billion gigabytes (or 2 exabytes)
in 2020. It isn't as if it's _that_ much data. Perhaps, if you think
of it on a block level, and let each block be 16 KB, then you'll sort
of postpone the necessity of moving beyond 64 bit for a while.
Assumptions:
- Usenet continues to grow exponentially at approximately the same
rate (the growth rate has been slightly more than exponential for
the past 7 years or so, but was higher again before 1993)
- Some people will still want to store all of Usenet for more than
one day (i.e. a week or so)
- Storage technology can keep up with this growth
- Network technology can keep up
- Other computer technology can keep up
Sources:
- Comparative studies in my own ongoing thesis on Usenet
distribution
- Henry E. Hardy's "The Usenet System", ITCA Yearbook, 1993
- Jeremy Nixon's old news statistics on SuperNews (now unavailable)
- http://newsfeed.mesh.ad.jp/flow/
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
In article <···············@skidbladnir.ifi.uio.no>,
Jan Ingvoldstad <····@ifi.uio.no> wrote:
>On 12 Dec 2000 13:53:04 GMT, ···@mit.edu (John F Carr) said:
>
>> Can anyone come up with a plausible program in the next two decades, to
>> run on conventional architectures, that will require an address space
>> between 60 and 64 bits? If you say "let's map everything into a flat
>> space" you will need more than 64 bits. If you stick to conventional
>> addressing, including memory mapping individual objects, you use a lot
>> less than 64 bits.
>
>
>18 446 744 073 709 551 616 bits
> 2 305 843 009 213 693 952 bytes
> 2 147 483 648 gigabytes
>
>Today, a "full" Usenet feed is around 250 GB/day, and the yearly
>growth is around 200%. With two decades of continued growth at the
>same rate, that'll be 250*3^20 gigabytes, or:
>
> 871 696 100 250 gigabytes
>
>So the answer is, yes, there is a plausible program that will require
>far more, if it is expected to use a system with flat addressing.
I don't believe usenet will grow a million times over the
next 20 years or be mapped into a flat address space.
There's a limit to the amount of data a human can read or
write, in any medium.
When the company I worked for was bought the new owners told us
to expect their stock to continue growing at the rate of 80% per
year for many years to come. A few moments of thought made it
clear that the stock value would exceed the US GNP, or even the
world GNP, in only a few years. (We've actually lost about 30%
over 7 months.)
--
John Carr (···@mit.edu)
On 12 Dec 2000 16:21:12 GMT, ···@mit.edu (John F Carr) said:
> I don't believe usenet will grow a million times over the
> next 20 years or be mapped into a flat address space.
> There's a limit to the amount of data a human can read or
> write, in any medium.
You clearly don't know what Usenet is being used for these days. If
you thought it was plain text articles like those we're blessed with
seeing in comp.arch, you're dead wrong.
Have a looksie at alt.binaries, and come back when you've uploaded a
handful of CD images, like so many others seem to do.
Then watch out for those DVD images.
In the future, expect other storage formats to be uuencoded or
base64-coded just as successfully as today's formats are.
> When the company I worked for was bought the new owners told us
> to expect their stock to continue growing at the rate of 80% per
> year for many years to come. A few moments of thought made it
> clear that the stock value would exceed the US GNP, or even the
> world GNP, in only a few years. (We've actually lost about 30%
> over 7 months.)
This isn't stock. Usenet feeds will keep on growing until technology
can't let it grow anymore; the users will see to that.
(BTW, the text-only feed for Usenet is probably far less than 5 GB
today, and it isn't growing nearly as fast as the full feed.)
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
In article <···············@skidbladnir.ifi.uio.no>,
Jan Ingvoldstad <····@ifi.uio.no> wrote:
>On 12 Dec 2000 16:21:12 GMT, ···@mit.edu (John F Carr) said:
>
>> I don't believe usenet will grow a million times over the
>> next 20 years or be mapped into a flat address space.
>> There's a limit to the amount of data a human can read or
>> write, in any medium.
>
>You clearly don't know what Usenet is being used for these days. If
>you thought it was plain text articles like those we're blessed with
>seeing in comp.arch, you're dead wrong.
There is a limit regardless of medium -- even video can't usefully go
over tens of megabytes per second. But let's assume that usenet will
grow 2x per year forever, and the world will come to an end if a day of
usenet can't fit into memory. (Or a week, month, or all of time.)
Continued exponential growth actually makes my case stronger. Because
no number of bits will be adequate in the long term, reducing the
effective address space by 1 bit only advances the transition to a
bigger address by one doubling period.
--
John Carr (···@mit.edu)
···@mit.edu (John F Carr) writes:
> In article <···············@skidbladnir.ifi.uio.no>,
> Jan Ingvoldstad <····@ifi.uio.no> wrote:
> >On 12 Dec 2000 16:21:12 GMT, ···@mit.edu (John F Carr) said:
> >
> >> I don't believe usenet will grow a million times over the
> >> next 20 years or be mapped into a flat address space.
> >> There's a limit to the amount of data a human can read or
> >> write, in any medium.
You are making the unwarranted assumption that address space is only
used for addressing memory. There are a number of good reasons that
you might want 64 address bits, but not use (expt 2 64) bytes.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <9164lv$hca@web.nmti.com>
In article <············@content-integrity.com>,
Joe Marshall <···@content-integrity.com> wrote:
> You are making the unwarranted assumption that address space is only
> used for addressing memory. There are a number of good reasons that
> you might want 64 address bits, but not use (expt 2 64) bytes.
Oooh, it's a trick question! I kno the answer!
"And CDR-coding is one of those reasons."
:->
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
John F Carr wrote:
>
> In article <···············@skidbladnir.ifi.uio.no>,
> Jan Ingvoldstad <····@ifi.uio.no> wrote:
> >On 12 Dec 2000 16:21:12 GMT, ···@mit.edu (John F Carr) said:
> >
> >> I don't believe usenet will grow a million times over the
> >> next 20 years or be mapped into a flat address space.
> >> There's a limit to the amount of data a human can read or
> >> write, in any medium.
Why should the data be human written? (current examples:
satellite weather pictures, executables)
Or for that matter human read? (current examples: mirror engines,
executables again)
Arbitrarily large bandwidth will have arbitrarily astounding (and
unpredictable) impact.
> >
> >You clearly don't know what Usenet is being used for these days. If
> >you thought it was plain text articles like those we're blessed with
> >seeing in comp.arch, you're dead wrong.
>
> There is a limit regardless of medium -- even video can't usefully go
> over tens of megabytes per second.
You obviously don't play video intensive computer games :->
Even at 60 fps (still discernable flicker) at 1280x1024x24 bit
you'll be nudging 200 MB/sec ... and that's still way below
(e.g.) IMAX film bandwidth requirements for real-time playback.
<snip>
--
Russell Crook
Not speaking officially for Sun (or anyone else, for that matter).
[about the 70 MB harddisk of the MicroVAX]
"Why waste a huge disk on a single user Workstation?"
- Ken Olson, ex-President of ex-company Digital Equipment Corporation
In article <·················@canada.Sun.COM>,
Russell Crook <·············@canada.Sun.COM> wrote:
>You obviously don't play video intensive computer games :->
>Even at 60 fps (still discernable flicker) at 1280x1024x24 bit
>you'll be nudging 200 MB/sec ... and that's still way below
>(e.g.) IMAX film bandwidth requirements for real-time playback.
Video compresses a lot. To within a factor of a few, I see the maximum
useful video rate for human viewing at 10 megapixels times 1 bit per
pixel per frame times 100 fps, or tens of MB/sec.
--
John Carr (···@mit.edu)
In article <················@senator-bedfellow.mit.edu>, ···@mit.edu (John
F Carr) wrote:
>In article <···············@skidbladnir.ifi.uio.no>,
>Jan Ingvoldstad <····@ifi.uio.no> wrote:
>>On 12 Dec 2000 16:21:12 GMT, ···@mit.edu (John F Carr) said:
>>
>>> I don't believe usenet will grow a million times over the
>>> next 20 years or be mapped into a flat address space.
>>> There's a limit to the amount of data a human can read or
>>> write, in any medium.
>>
>>You clearly don't know what Usenet is being used for these days. If
>>you thought it was plain text articles like those we're blessed with
>>seeing in comp.arch, you're dead wrong.
>
>There is a limit regardless of medium -- even video can't usefully go
>over tens of megabytes per second. But let's assume that usenet will
>grow 2x per year forever, and the world will come to an end if a day of
>usenet can't fit into memory. (Or a week, month, or all of time.)
>
>Continued exponential growth actually makes my case stronger. Because
>no number of bits will be adequate in the long term, reducing the
>effective address space by 1 bit only advances the transition to a
>bigger address by one doubling period.
Meanwhile, there's another limit that might be harder to deal with:
250 GB/day is about 23 megabits/sec average, i.e. half a t-3 for the
news backbone. The expanded version of usenet will require about
80 exabits/sec for the backbone; to give you a rough idea of what
that means, using straightforward modulation techniques that would
be about 10 signal transitions per cycle of yellow light.
I expect that doesn't make it impossible; instead, if anything, it
makes it likely that Usenet-N will be some kind of horrific distributed
database that absolutely requires a flat address space to work...
I was rather struck by someone else's comparison to ECC, which is
transparent to user-level programs. What kind of smart memory
architecture would you want so that CDR-coding was similary
transparent, and would you want it? (Would it be just one of a
host of possibilities for memory subsystems that didn't store your
data as written but simply promised to return that value when you
asked for it?)
paul
* Paul Wallich
| I expect that doesn't make it impossible; instead, if anything, it
| makes it likely that Usenet-N will be some kind of horrific distributed
| database that absolutely requires a flat address space to work...
I expect that only some headers will be passed around very shortly,
with really clever cache propagation protocols that makes clients
retrieve an article by message-id on demand, ultimately from the
originating server, cutting traffic and disk requirements down to what
is actually used, and killing half the stupid spam problem, too. News
traffic will follow the readers, not always flood the whole Net. And
since we're all fully connected (in N months, anyway), the off-line
generation of news readers and the desire to keep a full copy of
today's cache/catch of USENET locally will simply go away. Nobody in
their right mind has a desire to keep a "local copy" of the whole
World Wide Web just to read an infinitesimally small fragment of it,
yet that is what people do with USENET because getting out of how it
has been done up to now is hard to get away from, but it really is no
worse than some large ISP doing it the right way. The bonus of such a
design is that most originating servers would be able to hold on to an
article _much_ longer than the receiving servers of today are, and
that, too, would relieve most people of the need to keep local copies
of everything.
#:Erik
--
"When you are having a bad day and it seems like everybody is trying
to piss you off, remember that it takes 42 muscles to produce a
frown, but only 4 muscles to work the trigger of a good sniper rifle."
-- Unknown
Erik Naggum <····@naggum.net> writes:
> Nobody in their right mind has a desire to keep a "local copy" of
> the whole World Wide Web just to read an infinitesimally small
> fragment of it, yet that is what people do with USENET because
> getting out of how it has been done up to now is hard to get away
> from
You're probably right, but a good argument against it is that USENET
actually works in a responsive and reliable manner. WWW seems always
to be too slow and produce too many 40x's.
It can probably be solved with an intelligent limited distribution
system ensuring redundancy of paths and storage, but at the moment, I
think I'd rather just cut all the porn and warez from my feed.
-kzm
--
If I haven't seen further, it is by standing in the footprints of giants
On Wed, 13 Dec 2000 06:57:46 GMT, Ketil Z Malde <·····@ii.uib.no>
said:
> You're probably right, but a good argument against it is that USENET
> actually works in a responsive and reliable manner. WWW seems always
> to be too slow and produce too many 40x's.
One of the reasons for that is that not all news servers get all of
Usenet. Thank goodness.
Another argument against changing it to fetch-by-demand is that the
way things are feeded now, it works like crude pre-caching.
(Fetch-by-demand can be implemented in a smart way, too, of course,
lessening the problem, and even combined with pre-caching. Isn't that
nice.)
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
Erik Naggum <····@naggum.net> writes:
> I expect that only some headers will be passed around very shortly,
> with really clever cache propagation protocols that makes clients
> retrieve an article by message-id on demand, ultimately from the
> originating server, cutting traffic and disk requirements down to what
> is actually used, and killing half the stupid spam problem, too.
I've been hoping for such a thing for about 5 years now, but is there
any work goin gon...?
--
(espen)
On 13 Dec 2000 10:48:19 +0100, Espen Vestre
<·····@*do-not-spam-me*.vestre.net> said:
> Erik Naggum <····@naggum.net> writes:
>> I expect that only some headers will be passed around very shortly,
>> with really clever cache propagation protocols that makes clients
>> retrieve an article by message-id on demand, ultimately from the
>> originating server, cutting traffic and disk requirements down to what
>> is actually used, and killing half the stupid spam problem, too.
This is a bit too crude, unless it's your local server doing the
fetching for you, not the client (user agent). But that may be what
you mean with "clever cache propagation".
However, the originating/injecting server must have the bandwidth or
other capacity for dealing with external requests for articles or even
storing articles for as long as necessary (that is, until some
reasonable amount of time has passed). If those requirements aren't
met, the current model seems much better to me.
Considering that there are great variances in how long articles are
stored from news provider to news provider, it seems likely that there
is a significant amount of users who want to read older articles. It
isn't unreasonable to assume there will be a good amount of arbitrary
requests for older articles at the originating server, say up to a
month after the article was posted. Someone with a large user/poster
base will have to upgrade their injecting servers. :)
Another issue is that if the injecting server is somewhere remote in
Australia and your client is in Norway, response will be slow,
reducing the usefulness of Usenet compared to the web.
Ketil Z Malde has a point when he talks about the responsiveness of
today's Usenet; it's very important for the user that the articles
requested appear "immediately". (There hasn't been much research on
Usenet, but I believe it's safe to apply relevant aspects of usability
studies of the web.)
> I've been hoping for such a thing for about 5 years now, but is there
> any work goin gon...?
From what I understand, it isn't uncommon to deal with header-only
feeds, and Diablo supports fetching Message-IDs from other servers by
demand (automatic redirecting to the other news server). The latter
seemed to work well when I tested it when the news servers in the
chain were topologically close. I didn't test with servers on the
other side of the globe, though.
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
* Jan Ingvoldstad <····@ifi.uio.no>
| This is a bit too crude, unless it's your local server doing the
| fetching for you, not the client (user agent). But that may be what
| you mean with "clever cache propagation".
I consider today's USENET distribution a crude cache propagation
mechanism and propose something that uses much less bandwidth and disk
space while it maintains the principle of nearby caches. A few years
ago, for instance, the five largest USENET sites in Oslo, Norway, were
all located within a square mile, duplicating each other with great
precision and speed because they had different user bases and owners.
The extant news software could not propagate news articles by any
other means than flooding every server with everything, but if the
news protocols were only slightly smarter, they could have waited
until they needed the article body before they requested it, if they
had the headers that they could show to users. Users would not have
noticed those delays, and if there were any millisecond delays, it
would be only for the first reader that day/week/whatever. Now, scale
this up and consider large (network-topological) regions cooperating
to avoid duplicating news articles and traffic needlessly. How many
USENET servers are there around the core interchange points these
days? Distributed, redundant load balancing is not exactly news to
people who care about distributed systems, but we still have people
who worry needlessly if they cannot have their local USENET feed. If
you get upset because you cannot read news because a remote server is
down, you are a USENET addict and need treatment, not better servers.
The problem with the whole current design is, however, that you do not
know who the originator (initial injector) and owner is and you cannot
request the article except from another nearby cache, so if you have a
mesasge-id that some article is a response to, you are basically hosed
if you have not been so lucky as to have it flood by you. That does
not mean there _is_ no originator and owner, so I consider it useful
to think of the injecting user as _virtually_ reachable.
| However, the originating/injecting server must have the bandwidth or
| other capacity for dealing with external requests for articles or even
| storing articles for as long as necessary (that is, until some
| reasonable amount of time has passed). If those requirements aren't
| met, the current model seems much better to me.
The current in-flow rate is a problem. I have never heard of any
USENET site that has out-flow problems for articles originating at
their site. Keeping your "own" articles around for months should be
the smallest issue compared to keeping everybody else's articles
around for weeks and days. You can increase your out-flow a hundred-
fold and still be _far_ short of the current in-flow, and this makes
it possible for leaf sites to have (relatively speaking) _very_ small
caches while their preferred nearest cache (akin to the site they get
their feed from today) holds on to articles as long as one of their
many leaf sites request it. The great thing with a design like this
is that you can upgrade from the leaf sites "inward" to the core
servers, because as long as there are major core servers who still
operate in the "old way", there will be much less pressure on the
injecting site. Given how slowly changes occur in the USENET core
technology, the chances are very, very good that there will remain a
number of huge servers who can and will act as proxies for the many
injecting sites that will never see any significant network load.
| Considering that there are great variances in how long articles are
| stored from news provider to news provider, it seems likely that there
| is a significant amount of users who want to read older articles.
Yes, that's the idea, to keep the original article available longer.
| It isn't unreasonable to assume there will be a good amount of
| arbitrary requests for older articles at the originating server, say
| up to a month after the article was posted. Someone with a large
| user/poster base will have to upgrade their injecting servers. :)
Methinks you have lost all sense of proportion and need to back up and
look at the numbers you give for the current situation and its growth
and consider who the people are who contribute the volume of data that
you refer to. Yes, there are some large sites, perhaps responsible
for as much as 1/1000th of the total volume on USENET each, but they
_already_ have 1000 times the capacity to handle "injections" and if
you ask them a 100 times for every article they have published, they
are still at 1/10th their old bandwidth, and they aren't going to fill
the remaining 9/10th with requests from other servers, either.
| Another issue is that if the injecting server is somewhere remote in
| Australia and your client is in Norway, response will be slow,
| reducing the usefulness of Usenet compared to the web.
Really? How come I can pick up the article from any number of very
nearby caches today? Hmmm. Mystifying! How _did_ they get there?
| Ketil Z Malde has a point when he talks about the responsiveness of
| today's Usenet; it's very important for the user that the articles
| requested appear "immediately". (There hasn't been much research on
| Usenet, but I believe it's safe to apply relevant aspects of usability
| studies of the web.)
Yeah, I think you guys have got it _exactly_ right. Of course I'm out
to destroy USENET and its usability when I suggest that we invent a
better way to propagate articles. Of course I'm trying my very best
to screw with the minds of people who implement the software so _they_
also think "Hey, this USENET thing really was a bad idea to begin
with, so let's just do something utterly braindamaged that will kill
it and hopefully everybody using it, too". Get a _grip_, guys!
If you don't understand cache propagation mechanisms, say so. If you
cannot even _imagine_ that somebody out there have tried to think
about the way USENET propagates _while_ trying to keep it working for
its users, I suggest you try to insult people openly instead of just
assuming they have extra spicy garlic meatballs for brains. Sheesh.
Today's USENET propagation model does not try to keep track of where
articles are read, that is the task of local news admins, most of whom
take some pride in providing "everything" to their users. If we did
not ship the articles until they were read, only enough header stuff
that people could see newsgroups listings and stuff, the traffic would
change patterns to adapt to the readership instead of the global flood
algorithm used today. This would cause an increased ability to read
"fringe" newsgroups (it's always a hassle to get a news admin to get
another weird newsgroup hierarchy, because only one group is too much
work and the whole hierarchy is too much data), with actual _user_
input to the distribution of news articles.
| From what I understand, it isn't uncommon to deal with header-only
| feeds, and Diablo supports fetching Message-IDs from other servers by
| demand (automatic redirecting to the other news server). The latter
| seemed to work well when I tested it when the news servers in the
| chain were topologically close. I didn't test with servers on the
| other side of the globe, though.
I'm aware of Diablo and strongly encourage further development along
those lines, but as the network of servers picks up messages, they
will naturally be cached many places along the way, not very much
different from today's system. Having to follow a chain of forwarding
servers to get a particular article is therefore very unlikely, unless
you read "fringe" newsgroups that nobody else in your vicinity reads.
When you do that, you might also well tolerate longer access times.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
There's already "leafnode" and "leafnode+"
http://www.io.com/~kazushi/leafnode+/
As you've suggested, most of the operation of Usenet can
probably (and almost certainly will, evolutionaly) change to
these sorts of servers, as the admins of current news servers
learn about them. I doubt that NNTP-client users ever need to
know about these changes.
--
Andrew
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjhf46htnu.fsf@tfeb.org>
Erik Naggum <····@naggum.net> writes:
> [interesting things about news]
While I don't think it does really what you propose, the leafnode and
leafnode+ news systems are perhaps moves in the right direction. They
fetch groups on demand from upstream servers (the first time you ask
you see a little fake article saying it will be here in a minute, if
you stop reading a group then it stops fetching it after a bit), so
they're pretty good for small leaf sites. If, like me, you read a
small number of groups then you can also set the expiry times for
stuff that is fetched to be huge -- I think I cache comp.risks for a
year and c.l.l for several months.
--tim
* Tim Bradshaw <···@tfeb.org>
| While I don't think it does really what you propose, the leafnode and
| leafnode+ news systems are perhaps moves in the right direction.
They don't do anything close to what I propose. I use leafnode myself,
and while it goes to fetch news for every once in a while, it is even
harder than it usually is to fetch an article by message-id because
someone thinks cross-posting is a good way to "move" discussions.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
I don't think I should promote more discussion on this subject here in
comp.lang.lisp/comp.arch; my original point was on topic, but now
we're a bit too far removed from that, unless there are some valuable
lessons to be learned in computer architecture or with Lisp from this
topic.
(I _do_ find your points interesting -- which is why I question them
-- and I'd like to know if you've done any formal research on this,
published or not -- this is exactly the kind of thing I'm writing
about in my thesis, although there are many things I don't know yet
and still need to learn.)
My apologies to comp.arch and comp.lang.lisp for yet another off-topic
thread drift.
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
On Tue, 12 Dec 2000 14:45:18 -0500, ··@panix.com (Paul Wallich) said:
> Meanwhile, there's another limit that might be harder to deal with:
> 250 GB/day is about 23 megabits/sec average, i.e. half a t-3 for the
> news backbone. The expanded version of usenet will require about
> 80 exabits/sec for the backbone;
Eh, no, 811 exabytes/day is not 80 exabits/sec, but roughtly 80
petabits/sec, which is much easier to handle. ;)
> to give you a rough idea of what
> that means, using straightforward modulation techniques that would
> be about 10 signal transitions per cycle of yellow light.
Does this mean that it's only (roughly) 0.01 signal transitions per
cycle of yellow light, or is that what you get for petabits?
But still, the time for this problem to arise might still come.
Usenet news isn't the only thing in the world requiring larger amounts
of storage and bandwidth. Video-on-demand has (so far) largely been a
failure for the mass market because of the lack of bandwidth for the
necessary quality (although MPEG-4 and later MPEG-7 may make subtle
changes to this), and sometime, people may actually have those nice
HDTV (or better) displays in their homes. So yes, I think we'll reach
that limit one day. Maybe it won't be in 20 years, but I'll be around
when it happens.
--
In the beginning was the Bit, and the Bit was Zero. Then Someone
said, Let there be One, and there was One. And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.
In article <···············@skidbladnir.ifi.uio.no>, Jan Ingvoldstad
<····@ifi.uio.no> wrote:
>On Tue, 12 Dec 2000 14:45:18 -0500, ··@panix.com (Paul Wallich) said:
>
>> Meanwhile, there's another limit that might be harder to deal with:
>> 250 GB/day is about 23 megabits/sec average, i.e. half a t-3 for the
>> news backbone. The expanded version of usenet will require about
>> 80 exabits/sec for the backbone;
>
>Eh, no, 811 exabytes/day is not 80 exabits/sec, but roughtly 80
>petabits/sec, which is much easier to handle. ;)
Oops, got my prefixes swapped.
>
>> to give you a rough idea of what
>> that means, using straightforward modulation techniques that would
>> be about 10 signal transitions per cycle of yellow light.
>
>Does this mean that it's only (roughly) 0.01 signal transitions per
>cycle of yellow light, or is that what you get for petabits?
The latter. Exponent was right, name was wrong.
>But still, the time for this problem to arise might still come.
>Usenet news isn't the only thing in the world requiring larger amounts
>of storage and bandwidth. Video-on-demand has (so far) largely been a
>failure for the mass market because of the lack of bandwidth for the
>necessary quality (although MPEG-4 and later MPEG-7 may make subtle
>changes to this), and sometime, people may actually have those nice
>HDTV (or better) displays in their homes. So yes, I think we'll reach
>that limit one day. Maybe it won't be in 20 years, but I'll be around
>when it happens.
I don't think so, mostly because I don't think that the notion of "backbone"
will make any sense at that point. There still will be
places where enormous amount of data get shipped around, but the idea of
big, fat, long pipes connecting a bunch of narrow ends seems less likely.
Btw, video-on-demand requires less bandwidth into each house than does
regular cable TV. It's the aggregate bandwidth but more important the
decentralization that gets you.
paul
From: Anne & Lynn Wheeler
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <uzoi160i4.fsf@earthlink.net>
Jan Ingvoldstad <····@ifi.uio.no> writes:
> Today, a "full" Usenet feed is around 250 GB/day, and the yearly
> growth is around 200%. With two decades of continued growth at the
> same rate, that'll be 250*3^20 gigabytes, or:
>
> 871 696 100 250 gigabytes
lets say that there is currently 6billion people ... then 250gb/day
amounts to about 40bytes/person/day. say in 20 years there are twice
the number of people or 12 billion people on the planet ... then the
871 696 100 250 gigabytes ... works out to around 7-8 million bytes
per person/day.
already the majority of the bytes are not text ... but binary of one
form or another. lets say that text is only 10% of the total ... and
something is worked out to handle the binary in some other way. That
would still say in 20 years that every person on the planet would have
to be generating around 750,000 text characters per day; say 6
characters per word ... around 120,000 words/day ... say 300 words/page
... around 400 pages/person/day.
majority of the usenet growth has been binary with some increase due
to more people involved. however, once every person in the world is
spending all their time only generating material for usenet ... i
would think that the growth would start to level off.
--
Anne & Lynn Wheeler | ····@garlic.com - http://www.garlic.com/~lynn/
Anne & Lynn Wheeler <····@garlic.com> writes:
> Jan Ingvoldstad <····@ifi.uio.no> writes:
>
> > Today, a "full" Usenet feed is around 250 GB/day, and the yearly
> > growth is around 200%. With two decades of continued growth at the
> > same rate, that'll be 250*3^20 gigabytes, or:
> >
> > 871 696 100 250 gigabytes
>
> lets say that there is currently 6billion people ... then 250gb/day
> amounts to about 40bytes/person/day. say in 20 years there are twice
> the number of people or 12 billion people on the planet ... then the
> 871 696 100 250 gigabytes ... works out to around 7-8 million bytes
> per person/day.
>
> already the majority of the bytes are not text ... but binary of one
> form or another. lets say that text is only 10% of the total ... and
==============================================^^
> something is worked out to handle the binary in some other way. That
> would still say in 20 years that every person on the planet would have
> to be generating around 750,000 text characters per day; say 6
> characters per word ... around 120,000 words/day ... say 300 words/page
> ... around 400 pages/person/day.
Why 10%? Why not 1%, or .00001%? How many characters in a picture?
Or a movie, or a symphony, or...
My boss' wife just had a baby. Almost immediately, he had four pictures
available for viewing. Very few words, the pictures said it all.
Much less than 10% text, I would say.
> majority of the usenet growth has been binary with some increase due
> to more people involved. however, once every person in the world is
> spending all their time only generating material for usenet ... i
> would think that the growth would start to level off.
The assumption that people will spend all of their time generating
material for usenet is based on a presumption that the rate of
material generation itself will remain constant. A dubious
presumption, I think.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: George Neuner
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3a43cdcc.20062759@helice>
On 12 Dec 2000 09:57:42 -0800, Duane Rettig <·····@franz.com> wrote:
>The assumption that people will spend all of their time generating
>material for usenet is based on a presumption that the rate of
>material generation itself will remain constant. A dubious
>presumption, I think.
Duane,
Someone said, "[data] expands to fill the space allotted". I think
you seriously underestimate the propensity for generating useless,
space filling, crap that is inherent in most people. Witness the
proliferation of spam and joke emails, webcams and streaming
audio/video sites and "personal" web pages.
IMO it will only get worse as available bandwidth increases.
George
-----
Disclaimer: This posting is an example of "space filling crap". The
propensity of the author to generate this and similar examples may be
considered typical of a particular, but herein undesignated, class of
Internet users.
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <9165g2$hq5@web.nmti.com>
In article <·············@earthlink.net>,
Anne & Lynn Wheeler <····@garlic.com> wrote:
> lets say that there is currently 6billion people ... then 250gb/day
> amounts to about 40bytes/person/day. say in 20 years there are twice
> the number of people or 12 billion people on the planet ... then the
> 871 696 100 250 gigabytes ... works out to around 7-8 million bytes
> per person/day.
That's only 5-6 frames per day at a modest 1280x1024 resolution.
(http://www.eecg.toronto.edu/~mann/)
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
···@mit.edu (John F Carr) writes:
> In article <·············@beta.franz.com>,
> Duane Rettig <·····@franz.com> wrote:
> >······@best.com (Chuck Fry) writes:
> >
> >> And with 64 bits to play with, who's to say we can't spare a couple for
> >> CDR-coding?
> >
> >Let's not make this mistake again. When IBM (and Amdahl) created their XA
> >architectures for the 370 compatibles (extending the address range from 24
> >bits to 32 bits), several programs suddenly broke.
>
> Can anyone come up with a plausible program in the next two decades, to
> run on conventional architectures, that will require an address space
> between 60 and 64 bits? If you say "let's map everything into a flat
> space" you will need more than 64 bits. If you stick to conventional
> addressing, including memory mapping individual objects, you use a lot
> less than 64 bits.
Take a plastic ball and cover the bottom half with microwave detecters
and A/Ds. Hand wave that some horrid reasearcher will come up with a way
to get 8 bits per sample, even at, say, 500GHz. Hand wave that the packages
is about 5cm. So that is about 200 per ball, with out getting too thing
about cramming them all in. So that comes to 1TB per sec.
That per 'ball'. You have a square Km of them, plus extras. Say a spacing
of 2M, thats 500x500, 250,000. So that comes to 250 EB/sec... Now run it
24 hrs/days for a decade... Or, just take 10 hrs data for analysis. 64
bits does not go far here, just in data size.
Plus, the accounting software may need to be 64+ bits as well :)
--
Paul Repacholi 1 Crescent Rd.,
+61 (08) 9257-1001 Kalamunda.
West Australia 6076
Raw, Cooked or Well-done, it's all half baked.
>>>>> "Paul" == Paul Repacholi <····@prep.synonet.com> writes:
Paul> ···@mit.edu (John F Carr) writes:
>> In article <·············@beta.franz.com>, Duane Rettig
>> <·····@franz.com> wrote: >······@best.com (Chuck Fry) writes: >
>> >> And with 64 bits to play with, who's to say we can't spare a
>> couple for >> CDR-coding? > >Let's not make this mistake
>> again. When IBM (and Amdahl) created their XA >architectures
>> for the 370 compatibles (extending the address range from 24
>> >bits to 32 bits), several programs suddenly broke.
>> Can anyone come up with a plausible program in the next two
>> decades, to run on conventional architectures, that will require an
>> address space between 60 and 64 bits? If you say "let's map
>> everything into a flat space" you will need more than 64 bits. If
>> you stick to conventional addressing, including memory mapping
>> individual objects, you use a lot less than 64 bits.
Paul> Take a plastic ball and cover the bottom half with microwave
Paul> detecters and A/Ds. Hand wave that some horrid reasearcher will
Paul> come up with a way to get 8 bits per sample, even at, say,
Paul> 500GHz. Hand wave that the packages is about 5cm. So that is
Paul> about 200 per ball, with out getting too thing about cramming
Paul> them all in. So that comes to 1TB per sec.
Paul> That per 'ball'. You have a square Km of them, plus extras. Say
Paul> a spacing of 2M, thats 500x500, 250,000. So that comes to 250
Paul> EB/sec... Now run it 24 hrs/days for a decade... Or, just take
Paul> 10 hrs data for analysis. 64 bits does not go far here, just in
Paul> data size.
Paul> Plus, the accounting software may need to be 64+ bits as
Paul> well :)
The jump from 16 bit architectures to 32 was a _pretty_ big deal, as
the typical "dynamic scope" of 32 bits, +/-2^31-1, is a whole lot
bigger than +/-32767, and suffices to characterize a lot more
problems.
Jumping from there to 64 bits is a _whopping_ big jump; while 32 bits
isn't quite enough to represent national debts, it's not _far_ off,
and whether we speak of 36 bits, 48, 60, or 64, it's all "quite large
enough" to represent values that required compound data structures
with only 32 bits to play with.
Long and short being:
"No, the accounting software probably does _not_ need 64+ bits;
they'll probably revalue the currency before that becomes
relevant..."
There will indeed always be some "bleeding edge" applications that
could use the maximum capabilities that any technology can ever offer;
that doesn't mean that such technologies will be economically
feasible, or that they will offer enough to "more mundane"
applications to economically justify their development.
But where 8 bits sufficed for some applications, and having more is
gravy, the same is true for successive generations of microprocessors,
and 60 or 64 bits will "suffice" for many classes of applications
regardless of whether we are thinking of numeric ranges or the sizes
of memory addresses.
It should be eminently useful to take 64 bit architectures and waste a
few bits on tagging; cutting ranges from 64 to 60 still leaves a
useful range for _any_ application that has been deployable on a 32
bit architecture.
--
(concatenate 'string "cbbrowne" ·@hex.net")
<http://www.ntlug.org/~cbbrowne/>
Referring to undocumented private communications allows one to claim
virtually anything: "we discussed this idea in our working group last
year, and concluded that it was totally brain-damaged".
-- from the Symbolics Guidelines for Sending Mail
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <915f4n$3eu@web.nmti.com>
In article <·············@beta.franz.com>,
Duane Rettig <·····@franz.com> wrote:
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke.
And Microsoft made the same mistake with their Basic, *twice* on the Macintosh
and then *again* on the Amiga.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Duane Rettig <·····@franz.com> writes:
> ······@best.com (Chuck Fry) writes:
> > And with 64 bits to play with, who's to say we can't spare a couple for
> > CDR-coding?
>
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke.
As I recall (although I had nothing to do with Macs back then), early
Macs also used a 24-bit address space, and the same problems arose
when 32-bit addresses were introduced.
> 64-bit architectures tend to define their
> entire virtual space. For example, the PA-RISC with the W bit set defines
> text segments starting at 0x4000000000000000, and data segments starting at
> 0x8000000000000000. Which bits would you steal? Some bits in the middle?
Sure. The allocator needs to know about it, but as long as there's
plenty of virtual space, it's not that hard to do. If there isn't --
and I do believe it would be rash to assume somebody won't find a use
for all that space -- you need another version of your system with no
cdr-coding. It's wasteful not to put all that virtual space to
creative use, if the application doesn't need it all.
> > >Note that one such test that becomes nontrivial on General Purpose
> > >hardware (i.e. not lisp machines) is EQ. Instead of one instruction,
> > >those extra bits must be masked off before the comparison. I don't
> > >know if anyone has ever considered placing those extra bits "elsewhere",
> > >e.g. in a parallel location to the actual pointer/tag word.
I can't figure out what kind of an implementation you envision that
would have a special problem with EQ. I would place the extra bits in
the CAR cell to code the representation of the CDR, and mask them out
when reading the CAR; if you don't mask them out then, there's lots of
other operations that would have to know about them as well, not just
EQ.
--
Pekka P. Pirinen, Adaptive Memory Management Group, Harlequin Limited
The Memory Management Reference: articles, bibliography, glossary, news
<URL:http://www.xanalys.com/software_tools/mm/>
·····@harlequin.co.uk (Pekka P. Pirinen) writes:
> Duane Rettig <·····@franz.com> writes:
> > 64-bit architectures tend to define their
> > entire virtual space. For example, the PA-RISC with the W bit set defines
> > text segments starting at 0x4000000000000000, and data segments starting at
> > 0x8000000000000000. Which bits would you steal? Some bits in the middle?
>
> Sure. The allocator needs to know about it, but as long as there's
> plenty of virtual space, it's not that hard to do. If there isn't --
> and I do believe it would be rash to assume somebody won't find a use
> for all that space -- you need another version of your system with no
> cdr-coding. It's wasteful not to put all that virtual space to
> creative use, if the application doesn't need it all.
Right, but at Franz, we are not making such assumptions of non-use; this
is precisely the kind of assumption leading to a rewrite that I think
we should try to avoid.
> > > >Note that one such test that becomes nontrivial on General Purpose
> > > >hardware (i.e. not lisp machines) is EQ. Instead of one instruction,
> > > >those extra bits must be masked off before the comparison. I don't
> > > >know if anyone has ever considered placing those extra bits "elsewhere",
> > > >e.g. in a parallel location to the actual pointer/tag word.
>
> I can't figure out what kind of an implementation you envision that
> would have a special problem with EQ. I would place the extra bits in
> the CAR cell to code the representation of the CDR, and mask them out
> when reading the CAR; if you don't mask them out then, there's lots of
> other operations that would have to know about them as well, not just
> EQ.
Yes, exactly! Without cdr-coding, EQ objects have identical tagged
pointer values (which I call LispVals) and thus no masking is necessary
at all to compare identity. And my example was by no means
exhaustive, but was instead only the simplest example.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In comp.arch Duane Rettig <·····@franz.com> wrote:
> ······@best.com (Chuck Fry) writes:
>> And with 64 bits to play with, who's to say we can't spare a couple for
>> CDR-coding?
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke. These programs, which
> used the upper 8 bits of address words for tags and flags, assumed (correctly,
> but without foresight) that these bits were available for use.
This mistake hit several architectures based on the Motorola 680x0 also.
32bit address registers, but only 24 physical address lines on the 68000
and 68EC020 so the top 8 bits were ignored. Enter the 68030, and a lot
of software broke. It was especially bad on the Macintosh, which made Apple
add a "24bit compatibility mode" by using the MMU.
--
LarsG
In article <············@oslo-nntp.eunet.no>,
Lars Gaarden <·····@trustix.com> wrote:
>In comp.arch Duane Rettig <·····@franz.com> wrote:
>> ······@best.com (Chuck Fry) writes:
>
>>> And with 64 bits to play with, who's to say we can't spare a couple for
>>> CDR-coding?
>
>> Let's not make this mistake again. When IBM (and Amdahl) created their XA
>> architectures for the 370 compatibles (extending the address range from 24
>> bits to 32 bits), several programs suddenly broke. These programs, which
>> used the upper 8 bits of address words for tags and flags, assumed (correctly,
>> but without foresight) that these bits were available for use.
>
>This mistake hit several architectures based on the Motorola 680x0 also.
>32bit address registers, but only 24 physical address lines on the 68000
>and 68EC020 so the top 8 bits were ignored. Enter the 68030, and a lot
>of software broke. It was especially bad on the Macintosh, which made Apple
>add a "24bit compatibility mode" by using the MMU.
but there is a fundamental difference here: the examples you cite
started with entities (24 bit addresses) that were not big enough to
fill the space they were stored in. when those entities grew to fill
the available space they collided with situations where this space was
used for other purposes. in our case you deal with a conscious
decision to use a coarser granularity, which is not likely to change
(i really cannot imagine a memory subsystem allocating single bits or
bytes)
hs
"Duane Rettig" <·····@franz.com> wrote in message
··················@beta.franz.com...
> ······@best.com (Chuck Fry) writes:
> > And with 64 bits to play with, who's to say we can't spare a couple for
> > CDR-coding?
>
> Let's not make this mistake again. When IBM (and Amdahl) created their XA
> architectures for the 370 compatibles (extending the address range from 24
> bits to 32 bits), several programs suddenly broke.
The only user-mode programs that I can remember suddenly
breaking were programs to trawl OS control blocks to print
info about other address spaces which suddenly found that some
blocks had gone "above the line". And it was amazing those
control blocks stayed stable enough from one release to the next
that you could write programs which did sensible things with them.
> These programs, which
> used the upper 8 bits of address words for tags and flags, assumed
(correctly,
> but without foresight) that these bits were available for use.
Perhaps they did have foresight but different priorities. Perhaps
they thought that by the time 31-bit mode came along, it would be
time to rewrite the program in a HLL and remove dependencies on
ancient I/O interfaces. That often proved to be the case.
The alternative to stuffing things in the top 8 bits was where you
had a 32-bit word and 8-bit tag, forming a 5-byte entry in a packed
array. (I think this was used for exit lists on VTAM/VSAM ACBs.
Or something. Not very nice, anyway.)
And of course we still carried on using the top bit for a flag, even
in 31-bit mode. Was that lack of foresight?
> However,
> whether these programs were justified in the original designs or not, it
> would be safe for me to venture a guess that _none_ of them exist in their
> previous design form, and that all have been either rewritten or scrapped.
That's quite possibly true, but it's not because they didn't work.
The question is not whether you should or shouldn't use the top
bits of an address for flags, but whether you program in a language
that lets you change that design decision without pain when the
constraints change. Using spare bits in addresses (whether high
bits, due to small address space, or low bits, due to alignment) is
a perfectly valid technique.
At LMI, we abandoned CDR-coding for our last machine for several
reasons:
1. It took bits out of the pointers, which, being word pointers,
meant a reduction in address space.
2. It would complicate the hardware. We had hardware that
`understood' lists and could do `prefetches' on them.
3. Only about 10% of the allocated storage in a running Lisp
machine was compressed cdr-coded lists.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
"Frank A. Adrian" <·······@uswest.net> writes:
> So, again we see that the world simply re-invents Lisp but, more
> importantly, one of the neatest hacks from the olden days might be
> reasonable again. Opinions?
I have an even more efficient and radical idea: Don't use lists.
They are almost always the wrong data structure. Use arrays.
Arrays are available and efficient in Lisp, Scheme, Java, C++,
Fortran, etc etc.
If you need variable-sized data structures, you should still use
arrays. Just double the size of the array when it becomes too small.
You double the array to make sure that amortized (average) cost of
insertions is constant. Even just after you've doubled the array and
so almost half the space is unused, you are not using any more space
than a linked list would take.
If it is important to be able to insert/delete in the middle of
a sequence, you might consider using a list. But consider a
buffer-gap array first. Instead of leaving the unused space at
teh end of the array, allow it to be in the middle, whereever
insertions/deletions are done. You need to "move the gap" (i.e.
copy things around in the data structure) when the gap is not where
you need to insert/delete, but most of the time insertions/deletions
will tend to be clustered (near each other).
A arguemnt could be made that lists are easy to use in a functional
(side-effect-free) programs - just use recursion. But I don't think
that is a strong argument. You can write clean effecient side-effect-free
code using arrays: Just look at APL and its successors. Modern
functional languages have "list comprehensions" - these can just as
easily be used for "array comprehensions".
I am talking about standard Lisp-style lists implemented using pairs.
There is a better case to be made for chaining objects together using
link slot in a more complex objects. At least in that case you're not
allocating a separate cons cell for each link.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
* Per Bothner <···@bothner.com>
| I have an even more efficient and radical idea: Don't use lists.
Oh, Christ! We've been there, done that, about a million times.
| They are almost always the wrong data structure. Use arrays. Arrays
| are available and efficient in Lisp, Scheme, Java, C++, Fortran, etc
| etc.
Yes, so they are. Available. Arrays are also extremely inefficient
where lists are not the wrong data structure. Hence, use lists _and_
arrays, as appropriate. Lisp programmers are not the idiots you think
they are, Per. They figure out when it makes sense to use arrays even
if you don't and need to reduce the problem to silliness.
| If it is important to be able to insert/delete in the middle of
| a sequence, you might consider using a list.
If your data structures nest, use lists.
If your data structures have unpredictable structure, use lists.
If you need to maintain uniformity of type for the "rest" after
processing an initial portion of the data structure, use lists. (If
you think this only applies to recursion, which it seems you do, you
really must think more carefully about algorithm design with arrays
and why displaced arrays or passing _pairs_ (sorry, data structures in
small arrays, of course) of the array and the index won't cut it.)
If you need to manipulate the structure of the data structure in any
way, not just insert/delete, use lists. (Flattening, pairing, etc.)
If after much headache and painfully idiotic programming mistakes, you
discover that your array implementation really must use two elements
per "array", one data and one "next", you have reinvented the cons.
Congratulations! Welcome to the wisdom of 40 years of language design!
| I am talking about standard Lisp-style lists implemented using pairs.
| There is a better case to be made for chaining objects together using
| link slot in a more complex objects. At least in that case you're not
| allocating a separate cons cell for each link.
It is just insane to add "link" slots to more complex objects when you
can separate and abstract out the container concept from the containee.
Who cares about allocating separate cons cells? Simple arrays have to
be more than four elements long to take less space than the equivalent
list using cons cells. Specialized arrays may have to be much longer.
The allocation for cons cells is extremely efficiently implemented in
all Lisp implementations. Using your stupid arrays-for-lists would of
course force some super-clever implementation of exponent-of-2-sized
arrays that double in size as they are used _and_ move around all the
time (destroying object identity unless you waste even _more_ space on
their already inefficient implementation), but people have been there,
many times over, and they have _returned_ to lists, implemented via
cons cells, when that has been the most intelligent solution.
What we have left after so many years of using lists _and_ arrays
_and_ structures _and_ clos objects _and_ databases, etc, is that
there is not a single argument left for using anything _but_ lists
where lists are used. If you want to teach novices to use arrays, at
least don't make the misguided Scheme-style argument that alternatives
to the obvious and the most natural must be chosen because it somehow
more divine to use recursion, oops, I meant arrays.
You've done so much good work. How come you don't know simple things?
#:Erik
--
"When you are having a bad day and it seems like everybody is trying
to piss you off, remember that it takes 42 muscles to produce a
frown, but only 4 muscles to work the trigger of a good sniper rifle."
-- Unknown
Erik Naggum <····@naggum.net> writes:
> Arrays are also extremely inefficient
> where lists are not the wrong data structure.
Yes - but I think that is rare.
> Hence, use lists _and_ arrays, as appropriate.
Of course.
> Lisp programmers are not the idiots you think they are, Per. They
> figure out when it makes sense to use arrays even if you don't and
> need to reduce the problem to silliness.
Did I accuse anyone of being idiots? (I leave that to you.) Even
when arrays are *in principle* a better data structure, it may be
perfectly reasonable to use lists. For example it may be easier in
Lisp to program with lists than array. While Lisps do support arrays,
they are traditionally based on lists as the fundamental data
structure (for example &rest binds to a list, not an array), so it
just might be easier to use lists, even if they are less efficient.
Note the context of this discussion: Is cdr-coding a good idea?
My answer is that it might be a good useful for existing code
that uses lists heavily. However, it is reasonable to believe that
any code that would gain signficantly from cdr-coding would gain
as much or more from using arrays instead of lists. This
takes away much of the point of implementing cdr-coding.
> If your data structures nest, use lists.
Or nested arrays. Or you can use more clever data structures, such
as the "document table model" that the Xalan XSL processor uses
to represent XML documents.
> If your data structures have unpredictable structure, use lists.
If your data structures have unpredictable structure, you may be
in trouble ...
> If you need to maintain uniformity of type for the "rest" after
> processing an initial portion of the data structure, use lists.
Or use a more abstract sequence model, which would have arrays and
sub-ranges of arrsys *and* lists as special cases.
Or use a pair of an array and an integer.
> (If you think this only applies to recursion, which it seems you do, you
> really must think more carefully about algorithm design with arrays
> and why displaced arrays or passing _pairs_ (sorry, data structures in
> small arrays, of course) of the array and the index won't cut it.)
This is too vague for me to answer.
> If you need to manipulate the structure of the data structure in any
> way, not just insert/delete, use lists. (Flattening, pairing, etc.)
It is of course quite reasonable and efficient to do flattening
into an array.
> If after much headache and painfully idiotic programming mistakes, you
> discover that your array implementation really must use two elements
> per "array", one data and one "next", you have reinvented the cons.
> Congratulations! Welcome to the wisdom of 40 years of language design!
Erik, you know that I am older than you, and have probably been
programming at least as long as you. Don't you imagine I might have
quite a bit of experience with both arrays *and* lists? (I have less
experience with Common Lisp than you, I'm sure though.) I think my
share of "painfully idiotic programming mistakes" is modest, yet I
have not found that my "implementation really must use two elements
per "array"".
> Who cares about allocating separate cons cells? Simple arrays have to
> be more than four elements long to take less space than the equivalent
> list using cons cells.
Assume a modest one word of overhead per object. Then an extensible
array with three elements takes 3 words for the array header (header +
current length + pointer to data array) plus 4 words for the data
array (1 word of header plus three words of data) or 7 words total. A
non-extensible array would only need 4 words. If you need a separate
header word for the array allocated length, add one more word yielding
8 or 5 words, respectively. A list of three cons cells take 9 words.
Already for length three, the array wins.
If you use a memory allocation scheme not requiring any header words,
then the extensible 3-element array takes 6 words, the non-extensible
array takes 4 words, and the list version takes 6 words. The array
is equal to the list in space.
> The allocation for cons cells is extremely efficiently implemented in
> all Lisp implementations.
Unless you use cdr-coding, you will need 2 pointers plus any per-object
overhead (which may be zero) for each list element. This is twice as
much space as using an array. This only wins for short lists, or
for extensible size arrays whose size are just past the doubling threshhold.
> Using your stupid arrays-for-lists would of
> course force some super-clever implementation of exponent-of-2-sized
> arrays that double in size as they are used _and_ move around all the
> time (destroying object identity unless you waste even _more_ space on
> their already inefficient implementation),
One obvious way to solve the problem of unused space in extensible arrays
(while will still almost always takes less spaces than using lists
expect for very short sequences) is to have the garbage collector trim
off the excess space. (You double the space allocation to deal with
arrays that are actively growling, to prevent quadratic effects.
If they have survived to garbage collection, they are probably not
actively growing.)
An alternative to doubling the size may be to use the Fibonacci sequence
for the arrays sizees: 3 words, 5 words, 8 words, etc. This reduces the
maximum wastes space waste to asymptoticaly about 38%. I think this
preserves linear complexity (i.e. each array element has been copied
an *average* of a small constant number of times), but I'm not sure
enough of my math to say for sure. Does anyone know? (Knuth mentions
Fibonacci memory allocation, but I don't know if he says anything
relevant to this problem.)
> but people have been there,
> many times over, and they have _returned_ to lists, implemented via
> cons cells, when that has been the most intelligent solution.
Because it is convenient, not necessarily because it is efficient.
> You've done so much good work. How come you don't know simple things?
Likewise, I'm sure.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
* Per Bothner <···@bothner.com>
| > If your data structures have unpredictable structure, use lists.
|
| If your data structures have unpredictable structure, you may be
| in trouble ...
Really? Reading Lisp data from input streams is well-defined today,
despite an absence of predictability of the objects you read in. It
works for both lists and arrays, but arrays are very inefficient
because you don't know their size a priori, and copying elements like
crazy as the list grows is much less efficient than grabbing cons
cells from a pool of them. The way Lisp readers usually deal with
arrays is in fact to build a list from the input and then convert the
list to an array. You could process the data twice, once to count
elements and once to actually store them in memory. You could rely on
someone else not lying about the number of elements in the array,
which would make the data format impossible to edit by hand. Etc.
I recommend programmers in non-list-supporting languages to implement
the lists in the protocol syntaxes I have defined as arrays for one
good reason: Such languages seldom have automatic memory management,
either, so abandoning a reference causes memory leaks. If they keep
track of the array, they usually avoid the issue of keeping track of
all the individual elements. I have been kind to them by specifying
that no list will be longer than 32 elements, so allocating all arrays
that big remains a possibility. Much of the implementor feedback has
been the difficulty in reading lists of objects of unpredictable types
and my recommendation not to convert to an internal type until you
need it, but instead keep the string form with a prefix character (the
equivalent of a type tag) have resulted in some pretty weird code.
| Erik, you know that I am older than you, and have probably been
| programming at least as long as you. Don't you imagine I might have
| quite a bit of experience with both arrays *and* lists? (I have less
| experience with Common Lisp than you, I'm sure though.) I think my
| share of "painfully idiotic programming mistakes" is modest, yet I
| have not found that my "implementation really must use two elements
| per "array"".
You recommend arrays to other people, not just to yourself, unless I'm
_gravely_ mistaken about the purpose of your article. Other people
who have much less experience than you and rely on your experience in
order to form their own, with the least possible pain. My experience
with people who try to implement things with arrays is that they end
up with very cons-like concepts once they outgrow the C "memory is
really an array of bytes" idea. The reason is simple: Arrays with the
properties we discuss are very, very hard to implement correctly.
If we _are_ talking about what a compiler designer would do for a
language that natively supports these things in such a way that
programmers would never _see_ the array, the kind of reasoning you
have provided is completely misplaced. I have therefore assumed that
you do _not_ argue about primarily what a compiler writer would do,
although CDR-coding _is_ at that level.
| Assume a modest one word of overhead per object. Then an extensible
| array with three elements takes 3 words for the array header (header +
| current length + pointer to data array) plus 4 words for the data
| array (1 word of header plus three words of data) or 7 words total. A
| non-extensible array would only need 4 words. If you need a separate
| header word for the array allocated length, add one more word yielding
| 8 or 5 words, respectively. A list of three cons cells take 9 words.
| Already for length three, the array wins.
I think you need to explain how you ended up with 9 words for the
list. There is of course no header word for cons cells if you are
going to implement a very basic idea in a language with this. Type
information is stored much more efficiently if you care about these
things, or you allocate them in blocks that only hold cons cells.
That is, arrays of cons cells, if you want. Likewise, your arrays
would have type information in a similarly more efficient way.
Here's how I count. First, the data array. It has one word per
element, but it is aligned at and always allocate in some useful
chunk, like 4 words. This is to make allocation significantly
cheaper. Since the data array may need to be moved when grown, there
is a meta-object that contains the allocation size, the displacement
offset, the active size, and a pointer to the data array, or 4 words,
one allocation unit. The pointer to an allocation unit would of
course have four spare bits that could be used for type information,
just as pointers to cons cells have three spare bits for type
information, avoiding the header word overhead completely for the most
important builtin types. I need 4 cons cells to match an array of 4
elements.
| If you use a memory allocation scheme not requiring any header words,
| then the extensible 3-element array takes 6 words, the non-extensible
| array takes 4 words, and the list version takes 6 words. The array
| is equal to the list in space.
Yes, but only at enormous expense in allocation costs with such a fine
granularity. Also, you really do not want your array to move around
if it grows, or you lose the idea of identity of lists. Holding the
size without a pointer to data that can move means you always have to
copy the array if you do anything at all to it. This means that you
cannot perform any operations on the list implemented this way, only
on the elements of the fixed-size list. One of the charming issues
about lists is that they can grow so easily. If we are to give them a
_more_ efficient implementation, you can't take that away from them.
| Unless you use cdr-coding, you will need 2 pointers plus any
| per-object overhead (which may be zero) for each list element. This
| is twice as much space as using an array. This only wins for short
| lists, or for extensible size arrays whose size are just past the
| doubling threshhold.
The overhead is twice as much _after_ the overhead of arrays have been
dealt with, including that copying-when-doubling scheme, which out of
necessity must allocate memory for the same element many times over as
the list grows. By cleverly allocating allocation units for the data
array from a single pool that is not used for anything else, the same
way cons cells are allocated, you can grow your list painlessly and
linearly without copying until you allocate another array. You would
typically inform the allocator that you are allocating for a growing
list if you did this, howeer, to arrange for a large section of that
space to be temporarily reserved for the growing array. It would
revert to normal allocation once you inform the allocator that you
hare through growing that particular list. This cuts down on copying,
on reallocating, and speeds up just about everything while growing the
lists linearly, at the cost of a little bit more explicit code and
knowledge sharing between allocator and user. In a language with
support for these things you would want that, even if it would mostly
be used by system functions that build lists as return values.
If you get the impression that I have implemented this scheme for
real, that's exactly right.
| Because it is convenient, not necessarily because it is efficient.
When processing data that produces values of unknown lengths, and I
find myself doing that with an alarmingly high frequency compared to
values of known lengths, the growing pain of the array approach has
costs that do not win in the long run. If you usually process data
that produces values of known lengths with a similarly much higher
frequency than unknown lengths, you still have much higher overhad
than if you avoided the cleverness in the array implementation that
would take care of the growing. Hence, if you actually implement this
and compare approaches and test efficiency at a product-quality level,
lists made up of cons cells actually do win the efficiency contest,
especialy with a really fast cons cell allocator, and this not the
least because of the internal proximity of the data, which beats that
of the array that has been moved around several times while the data
it pointed to stayed where it was first stored (unless you implement
different memory pools for array allocation units and data, in which
case you don't get that added cost).
The interesting thing is, however, what the languages look like after
you decide to represent lists as arrays that way. E.g., cdr-ing down
a list can easily be emulated with displaced arrays, i.e., arrays that
really use another array's data for its elements, but starting from an
offset for elemen t0. (Hence the importance of storing type bits in
the pointer to the allocation unit for meta-object and data array, but
keeping them of the same size and from the same allocation pool.) You
may find that cdr-ing down an array allocates meta-objects, while
cdring down a list does not, because the _rest_ of an array is no
longer just a pointer to a cons cell, but a new displaced array. If
you want to push and pop elements on lists, you find that you want
lists that grow an shrink on different ends, leading to different
coding of many algorithsm, or you can be really clever and build
the list using displaced arrays that fill the allocation unit from
right to left and move them around instead. Many cool things that
affect what is efficient and convenient in surprising ways once you
start using it.
I did most of this work for a large automotive distributor back in the
late 80's when I had one of my many "pity I can't find a Lisp I can
use" periods and had major use for things Lisp makes easy. The system
was in use until they went from a dial-in terminal-based system over
both X.25 and regular phone lines to a web-based thing. I was greatly
amused when the new system cost 50 times more money to build (adjusted
for inflation and computer geek costs), required 30 times more memory,
processing power, and bandwidth to handle the same load, and almost 20
times more code to get the same work done. My system paid for itself
in 6 months of operation. I doubt this new system ever will pay for
itself before it is replaced or run into maintenance costs that will
keep it in the red for the rest of its life, but it's hot and flashy
and the users are very happy with the nicely laid-out web pages. The
users are also less efficient using it, though. Modern technology!
But I digress nostalgically.
#:Erik
--
"When you are having a bad day and it seems like everybody is trying
to piss you off, remember that it takes 42 muscles to produce a
frown, but only 4 muscles to work the trigger of a good sniper rifle."
-- Unknown
Erik Naggum <····@naggum.net> writes:
> Really? Reading Lisp data from input streams is well-defined today,
Perhaps, but does it make sense? Presumably you application has some
idea of the structure of the data it is expecting? (Even trees have
structure.)
> but arrays are very inefficient because you don't know their size
> a priori, and copying elements like crazy as the list grows is
> much less efficient than grabbing cons cells from a pool of them.
But that is just my argument - it is *not* much less efficient. If
you double the array as it grows, the "average" element is only copied
*a single time*. (Some elements may be copied n*log(n) times, but at
least half the elements have been copied at most once, at least
three-quarter of the elements have been copied at most twice, etc.)
> The way Lisp readers usually deal with arrays is in fact to build a
> list from the input and then convert the list to an array.
A reasonable thing to do, at least if you have a good GC.
> You recommend arrays to other people, not just to yourself, unless I'm
> _gravely_ mistaken about the purpose of your article. Other people
> who have much less experience than you and rely on your experience in
> order to form their own, with the least possible pain. My experience
> with people who try to implement things with arrays is that they end
> up with very cons-like concepts once they outgrow the C "memory is
> really an array of bytes" idea. The reason is simple: Arrays with the
> properties we discuss are very, very hard to implement correctly.
It is not that hard. Still, I agree it is is best if arrays are well
supported by the programming language or at least a good library.
> If we _are_ talking about what a compiler designer would do for a
> language that natively supports these things in such a way that
> programmers would never _see_ the array, the kind of reasoning you
> have provided is completely misplaced. I have therefore assumed that
> you do _not_ argue about primarily what a compiler writer would do,
> although CDR-coding _is_ at that level.
My argument is a little of all of:
(1) I think high-level programming languages should be built around
array support rather list support. (Actually, I prefer language
primitives to work on generalized sequences, with arrays just being
the default implementation.)
(2) I think programmers should (in general) use arrays rather than
lists, even in Lisp-like lanuages, at least when the number of
elements is large and performance is important.
(3) As a consequence of (1) and (2), I don't see a lot of justification
for cdr-coding, at least as something one would do in hardware.
> I think you need to explain how you ended up with 9 words for the
> list. There is of course no header word for cons cells if you are
> going to implement a very basic idea in a language with this.
I discussed this case in the following paragraph.
> Type information is stored much more efficiently if you care about
> these things, or you allocate them in blocks that only hold cons
> cells. That is, arrays of cons cells, if you want. Likewise,
> your arrays would have type information in a similarly more
> efficient way.
Agreed, at least for a Lisp-like language. For a object-oriented
language like Java, with many different types where fewer of them
predominate, it may make more sense to always have a header word. (Of
course CL is object-oriented. However, it differs from Java in that
certain pre-defined types are used very heavily so it makes sense to
optimize for those types in a way that makes less sense for Java.)
> Here's how I count. First, the data array. It has one word per
> element, but it is aligned at and always allocate in some useful
> chunk, like 4 words. This is to make allocation significantly
> cheaper. Since the data array may need to be moved when grown, there
> is a meta-object that contains the allocation size, the displacement
> offset, the active size, and a pointer to the data array, or 4 words,
> one allocation unit.
The displacement offset is not needed for adjustable arrays. It is only
needed for displaced arrays. And you can place the allocation size
at the start of the data array. (It would be needed there anyway if
memory management code needs to scan memory consequtively.) That gives
only 2 words for the header meta-object.
> | If you use a memory allocation scheme not requiring any header words,
> | then the extensible 3-element array takes 6 words, the non-extensible
> | array takes 4 words, and the list version takes 6 words. The array
> | is equal to the list in space.
>
> Yes, but only at enormous expense in allocation costs with such a fine
> granularity. Also, you really do not want your array to move around
> if it grows, or you lose the idea of identity of lists.
But the array object doesn't move if you use a header meta-word, only
the data array moves.
Of course if you want to preserve identity of lists for linked links,
you will also need an initial header cons.
> The overhead is twice as much _after_ the overhead of arrays have been
> dealt with, including that copying-when-doubling scheme, which out of
> necessity must allocate memory for the same element many times over as
> the list grows.
As I said: Only twice total, on average.
> By cleverly allocating allocation units for the data
> array from a single pool that is not used for anything else, the same
> way cons cells are allocated, you can grow your list painlessly and
> linearly without copying until you allocate another array.
No such tricks need to painlessly and *linearly* grow the array,
even *with* copying. Not to say that your scheme is a bad one
- I agree it a very reasonable thing to do.
> The interesting thing is, however, what the languages look like after
> you decide to represent lists as arrays that way. E.g., cdr-ing down
> a list can easily be emulated with displaced arrays, i.e., arrays that
> really use another array's data for its elements, but starting from an
> offset for elemen t0.
Agreed. However, my preference is for languages where the programmer
does not usually explicitly "cdr" down a list - or even index them.
I think languages should encourage programmers to work with arrays
as a unit, and language primitive should encourage that. I'm in favor
of "array languages" or what we might loosely call APL-style languages.
In the context of Common Lisp, I prefer the "series" design rather than
the "loop" design.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
"Per Bothner" <···@bothner.com> wrote in message
···················@kelso.bothner.com...
> (2) I think programmers should (in general) use arrays rather than
> lists, even in Lisp-like lanuages, at least when the number of
> elements is large and performance is important.
I think you are correct if (a) the language supports variable length arrays
or
(b) the user knows how many elements he will have or (c) random access of
the sequence data is important and (d) no large amountof sharing of sequence
data is needed and (e) no insertion of elements into the sequence happens.
In other
words, it depends. Also, note that what is supported by the language is
often what
gets coded by a user. If every sequence looks like an array, the user will
avoid
insertion and sharing of substructure and start assuming that O(1) access of
any
element is present. This, of course, makes some code very ugly, but since
the
user doesn't see any obvious alternative, he's stuck.
> (3) As a consequence of (1) and (2), I don't see a lot of justification
> for cdr-coding, at least as something one would do in hardware.
If you read my post carefully, you would have seen that nowhere did I ask
whether
or not there should be hardware support for such a feature. In fact,
history has shown
that language specific features for hardware are usually a bad idea. What I
asked was
given the mismatch of cache speed and CPU speed, would a software
implementation
of CDR-coding be a good implementation technique. I was hoping that someone
might
pop up with some discussion that actually involved hardware and cycle count.
Instead
I get straw men about hardware support for the feature.
To me the only valid argument I have seen so far is the one that says that
there are
better uses for the tag bits. This I can believe. Show me cycle counts
that demon-
strate that CDR-coding would add time to the "straight-line" path for list
traversal and
I will believe that CDR-coding is still a bad idea. Arguments that simply
assert that
arrays are "generally" better than lists are not convincing. Lists AND
arrays have been
around since the advent of computers. They are both useful under different
circumstances
and I don't think that you've given enough evidence to show that an array is
the best choice
for the default sequence type for a language. Certainly, it seems to be the
"most natural"
choice for Fortran, C, and Java language users, given that construction,
traversal, and
manipulation of lists is clunky in those languages (Strangely enough C++
with the STL
and operator overloading sidesteps this objection to a certain extent, but I
attribute that
to luck more than any conscious design effort) but that's hardly a
convincing argument
overall.
faa
"Frank A. Adrian" <·······@uswest.net> writes:
> What I asked was given the mismatch of cache speed and CPU speed,
> would a software implementation of CDR-coding be a good
> implementation technique.
This is a reasonable question. CDR-coding could conceivably be a
useful implementation techique in certain environments optimized for
large Lisp code bases. That assumes the effort to implement it is
reasonable; since I'm not convinced you could not get a better payoff
by using arrays in place of lists, at least in those place where
cdr-coding would make difference. But that might require re-writing
working code, which I can see people would rather not do!
> [List and arrays] are both useful under different circumstances
Agreed.
> and I don't think that you've given enough evidence to show that an
> array is the best choice for the default sequence type for a language.
Well, reasonable people can differ on this one.
> Certainly, it seems to be the "most natural" choice for Fortran, C,
> and Java language users, given that construction, traversal, and
> manipulation of lists is clunky in those languages
Using Lisp-style lists is quite straight-forward in Java. What it lacks
is a standard pre-defined list class based on conses. Using lists in
languages that lack a garbage collector is likely to be more tedious.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
* Per Bothner <···@bothner.com>
| Perhaps, but does it make sense?
Excuse me? Yes, it makes sense to read Lisp data that the application
does not know the structure of a priori. For one thing, source _code_
matches that description. Data whose structure is only evident after
having been read in as a complete expression also matches very well.
| Presumably you application has some idea of the structure of the data
| it is expecting? (Even trees have structure.)
Yes, it knows they will all be Lisp objects. That's it. This is part
of the significant charm with Lisp's lists.
| But that is just my argument - it is *not* much less efficient. If
| you double the array as it grows, the "average" element is only copied
| *a single time*. (Some elements may be copied n*log(n) times, but at
| least half the elements have been copied at most once, at least
| three-quarter of the elements have been copied at most twice, etc.)
There's something wrong with your math. The average number of times
an element is copied is 1.0 if you have (- (1+ (expt 2 n)) m) elements
in the final array with m elements in the initial allocation and
approaches 2.0 asymptotically for (1+ (expt 2 n)) elements in the
final array, since adding the last element will copy everything and
end up with half the allocated array empty (short that one element).
| My argument is a little of all of:
|
| (1) I think high-level programming languages should ...
| (2) I think programmers should ...
| (3) As a consequence of (1) and (2), ...
I'd be interested in how you _arrived_ at your preferences, instead of
just having something follow from some random personal favorites.
| (Of course CL is object-oriented. However, it differs from Java in
| that certain pre-defined types are used very heavily so it makes sense
| to optimize for those types in a way that makes less sense for Java.)
This is where you're losing me. I don't see a huge need for making a
basic type more efficient in languages that _don't_ show a predominant
use of that basic type. And why are we discussing Java? I thought it
was already as array-friendly as you wanted languages to be? And Lisp
already has adjustable arrays, so what's the point in redesigning them?
| The displacement offset is not needed for adjustable arrays. It is only
| needed for displaced arrays.
Sigh. Why am I wasting any time on this? The displacement is needed
for the "rest" (cdr) function that people who work on lists, however
they are actually implemented, need very often. If you want to offer
Lisp people something in terms of efficiency, you can't take away
their primitives without changing the entire language under them.
That's what I was trying to get at in the last parapraph I wrote.
| And you can place the allocation size at the start of the data array.
Yes, but that is a truly boneheaded decision as it means that the data
array can no longer be specialized to contain a uniform data type --
say double-floats instead of pointers to double-floats (and you really
want that if you go the way of arrays in the first place -- people
even ask for specialized list types, and they won't stop if they know
the underlying implementation is array-based), but must somehow have a
special initial element or consist of a single word that destroys
alignment of the rest of the array. This is just stupid. It's just
like the old string representation that used the first byte to hold
the length and couldn't hold more than 255 characters per string.
| (It would be needed there anyway if memory management code needs to
| scan memory consequtively.)
Really? I thought memory management code followed pointers and chains
from a root set if it were any smart. Randomly walking over memory it
has no idea what contains just seems like a recipe for major disaster.
Some conservative garbage collectors that are bolted onto languages
that were never properly designed to deal with abandoned pointers seem
to make do with interpreting machine words as pointers on a hunch, but
I have always been exceedingly skeptical of the entire approach. If
you keep the size of a vector in a well-known structure with the
pointer to the data array, there is only one way to get at that data
array, anyway.
| That gives only 2 words for the header meta-object.
And much more overhead and increased complexity and general madness.
You have obviously never actually experimented with this in _Lisp_-
like languages. I don't care about optimizing other languages for
adjustable arrays, as we already have efficient adjustable arrays in
Common Lisp, so there's no point in reinventing the wheel, and using
adjustable arrays to represent lists requires concern for the usage of
lists so optimized. I'm sure you can optimize this stuff to death in
Java, but Java doesn't have any need for cdr-coding or more efficient
storage of lists in particular, either, so I must admit to completely
failing to see the whole point of this exercise, except that you get
to argue for your preference for array-based languages, which I do not
share, and am unlikely to begin to share just because of this.
| Of course if you want to preserve identity of lists for linked links,
| you will also need an initial header cons.
I'm sorry, but is this a real argument? You're talking about data
arrays that move when they grow at the tail end, right? Lists can
grow at the tail end without any "header cons" simply because we just
overwrite the cdr that is nil in the final cons with a pointer to the
cons cell that holds the new element (and a new nil). Lists thus
grown retain their first cons cell, which does not move, and thus
retains identity at the pointer level. You do need a meta-object for
the header of arrays because the data array moves the address of the
first element when it is moved.
| As I said: Only twice total, on average.
Well, you actually said *a single time*, but if you keep doubling it
every time you say it, I'm happy. (Sorry, but I found that fuuny. :)
| However, my preference is for languages where the programmer does not
| usually explicitly "cdr" down a list - or even index them.
Well, I think it is preferable to ask people who prefer the languages
they are trying to optimize for their ideas on how to optimize them
instead of optimizing the programmers to use some language _they_ do
not prefer simply because the optimizers do.
I'm confused about your purposes, I guess. Promote APL if you like,
but then I don't see why you're even offering suggestions for how to
implement lists efficiently in Lisp. This is mystifying to me.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
Erik Naggum <····@naggum.net> writes:
> Excuse me? Yes, it makes sense to read Lisp data that the application
> does not know the structure of a priori. For one thing, source _code_
> matches that description.
Your point seems to be that it is a bad idea to use array to represent
complex nested structure, such as Lisp-style source code. I agree
this arrays are at a disadvantage here, since Lisp source code uses
many variable-sized and short sequences. However, the disadvantage is
minor (at most a small constant factor), and it doesn't really matter,
since source code is relatively small and only kept around during input
and compilation. It is more interesting what happens with large files
and large sequences. That is where arrays win and (I think) lists
have the disadvantage.
> Yes, [the application] knows they will all be Lisp objects.
> That's it. This is part of the significant charm with Lisp's lists.
An application has to know more. Presumably it intends to *do*
something with those Lisp objects.
> There's something wrong with your math. The average number of times
> an element is copied is 1.0 if you have (- (1+ (expt 2 n)) m) elements
> in the final array with m elements in the initial allocation and
> approaches 2.0 asymptotically for (1+ (expt 2 n)) elements in the
> final array, since adding the last element will copy everything and
> end up with half the allocated array empty (short that one element).
I believe your're right.
> | My argument is a little of all of:
> |
> | (1) I think high-level programming languages should ...
> | (2) I think programmers should ...
> | (3) As a consequence of (1) and (2), ...
>
> I'd be interested in how you _arrived_ at your preferences, instead of
> just having something follow from some random personal favorites.
First, I've tried to argue that arrays are a better general-purpose
data structure than lists: You can do things (indexing) that you can't
to efficiently with lists. Sequential traversal (including merging
and appending to the end) is perhaps most common use of sequences, and
boths arrays and lists handle that well, but arrays make it easy
to traverse from either end. Arrays have better locality of reference,
and they take less space (in general - short arrays and extensible
arrays just after resizing might lose out spacewise to lists, but not
by much). Lists have some advantages. Some of the advantages are
historic (i.e. Lisp is based on lists); other may be inherent. For
example some algorithms (tree-sort? - merge-sort? I forget the name)
need linked lists. Other applications may be more convenient to
code using lists, especially in languages like Lisp which historically
are oriented towards Lisp. So, generally arrays make a lots of sense,
and if you are going to represent a sequences of values in your
program, in general arrays are more likely to be the better answer
than lists. Now if you are going to be using Lisp, this may be a
little different, just because of Lisp's support for lists. But
from an abstract data-structure point of view (and note this is comp.arch,
not just comp.lang.lisp), arrays usually are to be preferred over lists.
Secondly, if I was designing a programming langauge from scratch, I
would use arrays rather than lists as the fundamental data type for
implementing sequences, partly for the reasons above. The other
reason is that I like languages that work on collections as a unit,
and provide operations than work on collections (sequences, sets,
relations, trees). Such languages can be very powerful and compact.
I also think they may be easier to use than languages that encourage
recursion to work on lists. I like side-effect free languages, or at
least languages that encourage a side-effect free style. In such a
language, the benefit of using lists instead of arrays is that it is
easy to use recursion to define your algorithms. But using recursion to
scan through a list is, I think, a bad idea, because it is harder to
understand than iteration. Recursion should be used for recursive
data structures. But defining a sequence as a recursive data
structure (a chain of conses) is wrong - you're creating recursion
where you don't need it.
> This is where you're losing me. I don't see a huge need for making a
> basic type more efficient in languages that _don't_ show a predominant
> use of that basic type.
Neither do I. My point was the different trade-off of Java and Lisp.
>. And why are we discussing Java? I thought it
> was already as array-friendly as you wanted languages to be?
Absolutely not. But that is a different discussion.
> And Lisp
> already has adjustable arrays, so what's the point in redesigning them?
I'm not. You're the one who insists on adding a displacement pointer
to all adjustable arrays.
> Sigh. Why am I wasting any time on this? The displacement is needed
> for the "rest" (cdr) function that people who work on lists, however
> they are actually implemented, need very often.
I guess the problem is we're discussing two different things here:
Arrays vs lists in abstract, and how they are used in Lisp. When
I work on arrays, I don't find the need for a rest function a lot.
If you need the rest/cdr function a lot, then perhaps you shouldn't
be using arrays. Or re-think your algorithms.
> Yes, but that is a truly boneheaded decision as it means that the data
> array can no longer be specialized to contain a uniform data type --
Of course it can. This is what Java primitive arrays are: A non-adjustable
array of some primtive type (for exampe double-floats). These are
comonly implemented as an object header followed by a length
followed by the data (words, bytes, or whatever).
> This is just stupid.
There are all sorts of tradeoffs, in memory management especially.
> | (It would be needed there anyway if memory management code needs to
> | scan memory consequtively.)
>
> Really? I thought memory management code followed pointers and chains
> from a root set if it were any smart.
Well, for example a compacting phase might be easier if you could
scan memory in order.
> Randomly walking over memory it has no idea what contains
If you have the appropriate header words, you would know what the memory
contains.
> | That gives only 2 words for the header meta-object.
>
> And much more overhead and increased complexity and general madness.
> You have obviously never actually experimented with this in _Lisp_-
> like languages.
Of course I have. However, I have done most of my work in the context
of the Java VM, which has other tradeoffs than if you can custom-design
your own memory-management. There are other Lisp implementations
that use the Boehm conservative collector (or similar), perhaps
because interoperability with C is desired, or because they would
rather used an existing well-engineered GC instead of writing their own.
> Lists can grow at the tail end without any "header cons" simply
> because we just overwrite the cdr that is nil in the final cons
> with a pointer to the cons cell that holds the new element (and a
> new nil).
Not if the list can start out empty.
> | As I said: Only twice total, on average.
>
> Well, you actually said *a single time*, but if you keep doubling it
> every time you say it, I'm happy. (Sorry, but I found that fuuny. :)
Twice referred to the number of memory accesses to write the array
(move once on average plus initialization). I'll bump that up to thrice,
if you like.
> I'm confused about your purposes, I guess. Promote APL if you like,
> but then I don't see why you're even offering suggestions for how to
> implement lists efficiently in Lisp. This is mystifying to me.
I am not talking about how to implement lists efficiently in Lisp, if
by "lists" you mean the "list" type of Common Lisp, except that I
don't think there is much point in implementing cdr-coding, at least
in hardware. What I am talking about how to implement sequences or
lists in the more generic sense.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjvgsn8e0z.fsf@tfeb.org>
Per Bothner <···@bothner.com> writes:
>
> Your point seems to be that it is a bad idea to use array to represent
> complex nested structure, such as Lisp-style source code. I agree
> this arrays are at a disadvantage here, since Lisp source code uses
> many variable-sized and short sequences. However, the disadvantage is
> minor (at most a small constant factor), and it doesn't really matter,
> since source code is relatively small and only kept around during input
> and compilation. It is more interesting what happens with large files
> and large sequences. That is where arrays win and (I think) lists
> have the disadvantage.
I think this paragraph sums up what is wrong with your argument for
Lisp. You seem to be assuming that people typicially use huge,
shallowly-nested lists which would `obviously' be better represented
in some array form. Well, I just don't think that's true for
well-written Lisp programs. I don't have general evidence, but in
programs I've written I've done some measurement of this kind of
thing, because I was concerned about things like search times, and I
found mean lengths of about 6 elements and mean nesting depths about
the same.
I suspect this isn't the case for Java, but Java programs perhaps
don't deal with the same kind of data that Lisp ones do.
--tim
Tim Bradshaw <···@tfeb.org> writes:
> You seem to be assuming that people typicially use huge,
> shallowly-nested lists which would `obviously' be better represented
> in some array form. Well, I just don't think that's true for
> well-written Lisp programs. I don't have general evidence, but in
> programs I've written I've done some measurement of this kind of
> thing, because I was concerned about things like search times, and I
> found mean lengths of about 6 elements and mean nesting depths about
> the same.
I think this actually supports my point, since arrays of length 6
are more space-efficient than lists of length 6. (With some quibbles
about implementation techniques for adjustable vs non-adjustable arrays.)
I agree it wouldn't make that much difference either way for short lists.
One might also use data structures differently in an array language.
The classic example is a matrix, which is more efficiently represented
using a single vector in row-major order, rather than a nested vector
of vectors. Another example is using two arrays to represent a tree.
> I suspect this isn't the case for Java, but Java programs perhaps
> don't deal with the same kind of data that Lisp ones do.
Probably not - and good Java programmers would be likely to structure
a solution differently than good Lisp programmers would.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjg0jqhtej.fsf@tfeb.org>
Per Bothner <···@bothner.com> writes:
>
> I think this actually supports my point, since arrays of length 6
> are more space-efficient than lists of length 6. (With some quibbles
> about implementation techniques for adjustable vs non-adjustable arrays.)
> I agree it wouldn't make that much difference either way for short lists.
Are they really? Remember you need both adjustability *and*
displacability (adjustability so you can do push &c, and
displacability so you can share tails, both of which are
significantly important features).
>
> One might also use data structures differently in an array language.
> The classic example is a matrix, which is more efficiently represented
> using a single vector in row-major order, rather than a nested vector
> of vectors. Another example is using two arrays to represent a tree.
Of course a matrix *would* be represented as a single entity in
row-major order in Lisp -- not a vector, as it knows its dimensions,
but the storage is the same, and indeed there is an accessor --
ROW-MAJOR-AREF which just treats arrays as vectors. Remember Lisp
*is* an array language.
--tim
Tim Bradshaw <···@tfeb.org> writes:
> Are they really? Remember you need both adjustability *and*
> displacability (adjustability so you can do push &c, and
> displacability so you can share tails, both of which are
> significantly important features).
You need displacability is you are doing straight-forward
re-implementation of list code to use arrays instead. But, otherwise
no, sharing tails is not a significantly important features, not
enough so that you want to include a displacement in every array, or
even every adjustable array. If you are using displaced arrays as a
replacement for cdr-ing down a list you're almost certainly doing
something wrong.
Note I am not talking running existing Lisp code or even necessaily
using the (a) Lisp language, but about solving Lisp-style problems
using a language based on arrays instead of lists.
If you have a language with type inheritance, it is useful to have
at least three array implementations:
(1) Constant arrays. Simple and compact. The equivalent of the Lisp
reader and the "list" function should return these, as should "arrays
comprehensions" (from a user's, not implementor's, point of view).
(2) Adjustable arrays.
(3) Slice arrays: arrays that are displaced to a sub-sequence of
another array, perhaps with a stride.
A fourth type:
(4) Fixed-size arrays with mutable elements.
- is useful for library implementors but most programs should
have little use for it. (I'm assuming a "mostly functional"
language.)
> Of course a matrix *would* be represented as a single entity in
> row-major order in Lisp -- not a vector, as it knows its dimensions,
> but the storage is the same, and indeed there is an accessor --
> ROW-MAJOR-AREF which just treats arrays as vectors. Remember Lisp
> *is* an array language.
Agreed. My point is that you have a data structure implemented as
a nested list, alternative data structures may be appropriate. I used
matrix as an example that is already available in Common Lisp.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
From: Tim Bradshaw
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <nkjhf45fsrl.fsf@tfeb.org>
Per Bothner <···@bothner.com> writes:
> You need displacability is you are doing straight-forward
> re-implementation of list code to use arrays instead. But, otherwise
> no, sharing tails is not a significantly important features, not
> enough so that you want to include a displacement in every array, or
> even every adjustable array. If you are using displaced arrays as a
> replacement for cdr-ing down a list you're almost certainly doing
> something wrong.
Sharing substructure is unfortunately completely crucial to a lot of
the code I'm talking about. Without it space requirements would be
hideous, and equality tests would change from something which is
almsot always a check for pointer equality to something which could
walk half the tree.
Indeed, if I was doing it in a language other than Lisp which didn't
have a huge array bias like, say, fortran, I'd use the same technique
I used in the Lisp code before I optimized it -- objects (in the sense
of things defined with DEFSTRUCT or DEFCLASS rather than in the better
Lisp sense of `everything is an object') with a couple or three of
slots. I definitely wouldn't use arrays.
What I'm trying to get at is that you seem to be assuming that lists
(really, conses, Lisp doesn't actually have lists as a distinct
representational type) are used as a kind of crap array substitute.
That was perhaps once true and might still be true in badly written
Lisp code (there is a lot of this). It *isn't* true in well-written
code.
But I don't think this argument is going anywhere unfortunately, so
I'll avoid wasting further bandwidth.
--tim
Tim Bradshaw <···@tfeb.org> writes:
> What I'm trying to get at is that you seem to be assuming that lists
> (really, conses, Lisp doesn't actually have lists as a distinct
> representational type) are used as a kind of crap array substitute.
> That was perhaps once true and might still be true in badly written
> Lisp code (there is a lot of this). It *isn't* true in well-written
> code.
Beeing able to use pointer equality is certainly valuable. Being able
to use pointer equality for tails of lists certainly seem less obvious
- and I suspect rarely useful. You may be right, but I am unconvinced
The only way to decide something like this is to examine the problems
where you say you need to be able to share list tails, and consider
alternative solutions. That is probably more time than either of us
want to spend. Might be a valueable paper in it, though.
--
--Per Bothner
···@bothner.com http://www.bothner.com/~per/
* Per Bothner <···@bothner.com>
| I am not talking about how to implement lists efficiently in Lisp, if
| by "lists" you mean the "list" type of Common Lisp, except that I
| don't think there is much point in implementing cdr-coding, at least
| in hardware. What I am talking about how to implement sequences or
| lists in the more generic sense.
Well, OK. cdr-coding is all about efficient implementation of lists in
Lisp, and the reason I insist on that displacement offset that you reject
out of hand even with the explanation is that the context of the whole
thread has been how to store lists more efficiently, while obviously
keeping all the listness qualities. I'm sorry you have missed this.
I don't have a problem with your desire to use arrays. I use them all
the time myself, and so do other Lispers. I don't see a conflict, and I
don't see a need to optimize Lisp any more towards arrays, as the support
for the abstract type "sequence" already there, too.
It seems to me that you think "Lisp" is the Scheme branch of things,
which I barely think is a Lisp at all -- I think Scheme is an Algol with
Lispy syntax and some functional properties. There are many reasons I
loath Scheme, but one of them is certainly that it exposes the choice of
underlying types so much in the interest of giving you that Algol feel of
efficiency and typeness.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
From: Paul DeMone
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <3A396FF2.A1301C34@igs.net>
Per Bothner wrote:
[...]
> Secondly, if I was designing a programming langauge from scratch, I
> would use arrays rather than lists as the fundamental data type for
> implementing sequences, partly for the reasons above. The other
> reason is that I like languages that work on collections as a unit,
> and provide operations than work on collections (sequences, sets,
> relations, trees). Such languages can be very powerful and compact.
You might be interested in NIAL (Nested Interactive Array Language)
http://www.nial.com
--
Paul W. DeMone The 801 experiment SPARCed an ARMs race of EPIC
Kanata, Ontario proportions to put more PRECISION and POWER into
······@mosaid.com architectures with MIPSed results but ALPHA's well
·······@igs.net that ends well.
Erik Naggum <····@naggum.net> writes:
> * Per Bothner <···@bothner.com>
> | (It would be needed there anyway if memory management code needs to
> | scan memory consequtively.)
>
> Really? I thought memory management code followed pointers and chains
> from a root set if it were any smart. Randomly walking over memory it
> has no idea what contains just seems like a recipe for major disaster.
No, not at all. There are many reasons why a collector might scan
memory sequentially. And if it's not a conservative collector, it
does know the layout of the objects. The most important sequential
technique is the Cheney scan <URL:http://www.xanalys.com/software_too
ls/mm/glossary/c.html#cheney.scan>, which most copying collectors use:
scanning over the newly copied objects eliminates the need for a
scanning stack. Non-moving collectors often scan over memory in the
reclaim phase to build the new freelist.
> If you keep the size of a vector in a well-known structure with the
> pointer to the data array, there is only one way to get at that data
> array, anyway.
As long as the array code is not interrupted by GC, leaving a pointer
to the data array on the stack/registers. Furthermore, having a data
array that the collector cannot parse without making reference to an
array header makes some barrier techniques harder, for example those
based on VM page protection. I wouldn't recommend it.
--
Pekka P. Pirinen, Adaptive Memory Management Group, Harlequin Limited
Hell is other posters.
- Nancy Lebovitz (nancyL_universe.digex.net)
From: Anne & Lynn Wheeler
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <ud7et4f7s.fsf@earthlink.net>
·····@harlequin.co.uk (Pekka P. Pirinen) writes:
> As long as the array code is not interrupted by GC, leaving a pointer
> to the data array on the stack/registers. Furthermore, having a data
> array that the collector cannot parse without making reference to an
> array header makes some barrier techniques harder, for example those
> based on VM page protection. I wouldn't recommend it.
for the port of apl/360 to cms/apl ... this was one of the problems
that we had run into. going from essentially a swapping
32kbyte-64kbyte workspace environment to a demand paging 16mbyte
workspace environment ... the original apl had the propensity that
every assignment allocated new space (and prior location
ignored). when end of workspace was reached, garbage collection would
compact all allocated space and the process started over.
one of the other projects at csc was vs/repack ... which included page
reference monitoring and drawing "pictures" with special TN train of
the operation. At times the halls of 4th floor, 545 tech. sq were
papered with these reference pictures. basically five foot long
section on reversed green-bar paper with storage accesses along the
length and time along the width. A whole series of these were taped to
the wall (length running vertical) giving several seconds of program
execution as you moved down the hall.
The bottom of the wall tended to be solid pattern of use ... but there
was a very strong sawtooth pattern where pages were touched, used for
very short periods and then storage allocation moved up until it
reached top of storage ... and then there was a solid line as all
allocated data was compacted back down to low storage. In virtual
memory environment, this tended to result in an application using
maximum amount (all) of available virtual memory regardless of the
application size or complexity ... with bimodel reference pattern
... partially LRU and partially MRU (i.e. the least recently used page
wasn't likely to be used again for awhile).
Part of the less obvious things that we had to do in the '71
time-frame for the port of apl/360 to cms/apl was a dynamically
adaptive memory collector.
--
Anne & Lynn Wheeler | ····@garlic.com - http://www.garlic.com/~lynn/
Per Bothner wrote:
>
> Erik Naggum <····@naggum.net> writes:
> > You recommend arrays to other people, not just to yourself, unless I'm
> > _gravely_ mistaken about the purpose of your article. Other people
> > who have much less experience than you and rely on your experience in
> > order to form their own, with the least possible pain. My experience
> > with people who try to implement things with arrays is that they end
> > up with very cons-like concepts once they outgrow the C "memory is
> > really an array of bytes" idea. The reason is simple: Arrays with the
> > properties we discuss are very, very hard to implement correctly.
>
> It is not that hard. Still, I agree it is is best if arrays are well
> supported by the programming language or at least a good library.
What's wrong with STL?
If you have to use C++ anyway, STL has a lot of very useful ideas.
> My argument is a little of all of:
>
> (1) I think high-level programming languages should be built around
> array support rather list support. (Actually, I prefer language
> primitives to work on generalized sequences, with arrays just being
> the default implementation.)
I.e. STL <vector> etc.
Terje
--
- <··············@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
>>>>> "Per" == Per Bothner <···@bothner.com> writes:
Per> Erik Naggum <····@naggum.net> writes:
>> Really? Reading Lisp data from input streams is well-defined
>> today,
Per> Perhaps, but does it make sense? Presumably you application has
Per> some idea of the structure of the data it is expecting? (Even
Per> trees have structure.)
Obviously loading in nonsense will have nonsensical results; the flip
side is that the reader can cope with a variety of sorts of data
regardless of whether you have _specifically_ coded to expect it or
not...
>> but arrays are very inefficient because you don't know their size a
>> priori, and copying elements like crazy as the list grows is much
>> less efficient than grabbing cons cells from a pool of them.
Per> But that is just my argument - it is *not* much less efficient.
Per> If you double the array as it grows, the "average" element is
Per> only copied *a single time*. (Some elements may be copied
Per> n*log(n) times, but at least half the elements have been copied
Per> at most once, at least three-quarter of the elements have been
Per> copied at most twice, etc.)
Interesting... Yes, that seems right. Note that what the array
consists of is an array _of references_; it should only need to be a
set of pointers that gets copied multiple times, and not the data
element themselves...
>> The way Lisp readers usually deal with arrays is in fact to build a
>> list from the input and then convert the list to an array.
Per> A reasonable thing to do, at least if you have a good GC.
It's not an _outrageous_ thing to do, regardless.
A Lisp reader can grab elements and shove them into a list as a
buffer, requiring O(1) memory and O(1) time for each request, and
maintain a counter, also an O(1) operation. That adds up to O(n) for
an array of size n. Once the size is determined, it's perfectly
reasonable to, at that point, establish an array of size n, and shove
the n references in. Again, O(n) time.
Using an array for "everything" is liable to make this a bit slower.
Not dramatically, and your analysis demonstrates that yes, indeed,
copying arrays ought not to be _spectacularly_ slower than the
list-based equivalents.
>> You recommend arrays to other people, not just to yourself, unless
>> I'm _gravely_ mistaken about the purpose of your article. Other
>> people who have much less experience than you and rely on your
>> experience in order to form their own, with the least possible
>> pain. My experience with people who try to implement things with
>> arrays is that they end up with very cons-like concepts once they
>> outgrow the C "memory is really an array of bytes" idea. The
>> reason is simple: Arrays with the properties we discuss are very,
>> very hard to implement correctly.
Per> It is not that hard. Still, I agree it is is best if arrays are
Per> well supported by the programming language or at least a good
Per> library.
"It's hard." "No, it's not." Hmm... May be tough to reconcile.
>> If we _are_ talking about what a compiler designer would do for a
>> language that natively supports these things in such a way that
>> programmers would never _see_ the array, the kind of reasoning you
>> have provided is completely misplaced. I have therefore assumed
>> that you do _not_ argue about primarily what a compiler writer
>> would do, although CDR-coding _is_ at that level.
Per> My argument is a little of all of:
Per> (1) I think high-level programming languages should be built
Per> around array support rather list support. (Actually, I prefer
Per> language primitives to work on generalized sequences, with arrays
Per> just being the default implementation.)
Per> (2) I think programmers should (in general) use arrays rather
Per> than lists, even in Lisp-like lanuages, at least when the number
Per> of elements is large and performance is important.
Per> (3) As a consequence of (1) and (2), I don't see a lot of
Per> justification for cdr-coding, at least as something one would do
Per> in hardware.
This sounds rather like the two forms of noneuclidean algebra, having
slightly different "shapes of space."
I'm not sure that CL is _terribly_ far away from your notion of (1);
the SEQUENCES dictionary works with both arrays _and_ lists. And as
for (2), _certainly_ the use of something other than lists will be
considered "fairly wise" for large numbers of elements. It may be
"violent agreement," but there should at least be _some_ agreement.
The thing that is _less_ clear is how to best cope with the bits of
"syntactic glue" that are needed to fiddle around with basic bits of
data.
For instance, in:
(defun discriminant (a b c)
"compute roots of ax^2 + bx + c = 0"
[do bunch of stuff])
What should (a b c) be? Should it be an array? Or a list?
Then comes the question of what to do about CAR/CDR/CA*D*R, and what
to do about things like DESTRUCTURING-BIND. Those naturally work with
lists, not arrays.
Changing Lisp to use arrays _really_ changes the structure that sort
of stuff; I'm not sure how to twist my mind around the implications.
APL probably provides a lot of the guidance...
Per> Agreed. However, my preference is for languages where the
Per> programmer does not usually explicitly "cdr" down a list - or
Per> even index them. I think languages should encourage programmers
Per> to work with arrays as a unit, and language primitive should
Per> encourage that. I'm in favor of "array languages" or what we
Per> might loosely call APL-style languages.
Per> In the context of Common Lisp, I prefer the "series" design
Per> rather than the "loop" design.
.. And both have their place.
Series offers some (potential) opportunities for doing static analysis
that might make runtime evaluations go away altogether, which is
pretty cool.
Looking at the "APL world" is of some significant benefit, and typical
APL developers would certainly agree with your preference of "SERIES"
over "LOOP"; one of my brothers used to do APL work, and the True APL
Way certainly involves using "series-like" operators to the exclusion
of the "diamond" loop operator.
Unfortunately, APL programmers would go to _extremes_ just to avoid
looping, often being happy to create unreadable code in the interests
of Keeping That Diamond Out.
Diamond, LOOP, and GOTO strike me as all being much the same thing:
powerful tools that, if used "for evil," can assortedly produce slow
and unreadable code. But if these are excluded _as evil_, their
alternatives may _also_ become competitive in obfuscated code
contests.
--
(reverse (concatenate 'string ····················@" "454aa"))
<http://www.ntlug.org/~cbbrowne/>
If you have nothing to say on a subject, replying with a line such as,
"I agree with this." puts you in the TO:'s for all future messages, and
establishes you as "one who really cares", if not an actual expert, on
the topic at hand.
-- from the Symbolics Guidelines for Sending Mail
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <917vho$dv0@web.nmti.com>
In article <··············@kelso.bothner.com>,
Per Bothner <···@bothner.com> wrote:
> Note the context of this discussion: Is cdr-coding a good idea?
CDR-coding is an implementation technique for hiding an array structure
behind a list structure.
There are other techniques you can use, for example you can simply partition
memory into "dotted pair territory" and "array territory", and start all
arrays on a known page boundary so that one comparison can tell if you're
looking at an array or a list and a masking operation will get you the
start of the array. This would limit you to a maximum array size (the
granularity of the pages) so you'd still want lists of arrays, but that
still gives you a factor of (page size / array element size) speedup in
indexing, and the memory overhead would be (1 - pointer size / page size).
Or you can use a mixture, and steal one bit to distinguish between conses
and arrays.
But it's all an issue of implementation techniques. You can pretty much hide
the whole thing from the programmer, with a little cleverness.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
In article <··········@web.nmti.com>, ·····@abbnm.com (Peter da Silva)
wrote:
> In article <··············@kelso.bothner.com>,
> Per Bothner <···@bothner.com> wrote:
> > Note the context of this discussion: Is cdr-coding a good idea?
>
> CDR-coding is an implementation technique for hiding an array structure
> behind a list structure.
CDR-coding is an implementation technique for optimizing
the space usage and the locality of lists.
--
Rainer Joswig, Hamburg, Germany
Email: ·············@corporate-world.lisp.de
Web: http://corporate-world.lisp.de/
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91919i$4b8@web.nmti.com>
In article <····························@news.is-europe.net>,
Rainer Joswig <······@corporate-world.lisp.de> wrote:
> In article <··········@web.nmti.com>, ·····@abbnm.com (Peter da Silva)
> wrote:
> > In article <··············@kelso.bothner.com>,
> > Per Bothner <···@bothner.com> wrote:
> > > Note the context of this discussion: Is cdr-coding a good idea?
> > CDR-coding is an implementation technique for hiding an array structure
> > behind a list structure.
> CDR-coding is an implementation technique for optimizing
> the space usage and the locality of lists.
I see these two statements as being equivalent in the context of this
discussion.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
·····@abbnm.com (Peter da Silva) writes:
> In article <····························@news.is-europe.net>,
> Rainer Joswig <······@corporate-world.lisp.de> wrote:
> > In article <··········@web.nmti.com>, ·····@abbnm.com (Peter da Silva)
> > wrote:
> > > In article <··············@kelso.bothner.com>,
> > > Per Bothner <···@bothner.com> wrote:
> > > > Note the context of this discussion: Is cdr-coding a good idea?
>
> > > CDR-coding is an implementation technique for hiding an array structure
> > > behind a list structure.
>
> > CDR-coding is an implementation technique for optimizing
> > the space usage and the locality of lists.
>
> I see these two statements as being equivalent in the context of this
> discussion.
The post points out that the discussion may be fundamentally flawed:
lists and vectors are abstract data types, linked lists and arrays are
implementations. Up until that post, the data type and implementation
were being confused.
--
Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
Department of Computer Science FAX -- (505) 646-1002
New Mexico State University http://www.cs.nmsu.edu/~pfeiffer
VL 2000 Homepage: http://www.cs.orst.edu/~burnett/vl2000/
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <91ava5$454@web.nmti.com>
In article <··············@viper.cs.nmsu.edu>,
Joe Pfeiffer <········@cs.nmsu.edu> wrote:
> The post points out that the discussion may be fundamentally flawed:
> lists and vectors are abstract data types, linked lists and arrays are
> implementations. Up until that post, the data type and implementation
> were being confused.
Good call, that man. Thanks for putting what I meant into clearer terms.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
From: ······@corporate-world.lisp.de
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <9193g3$j2j$1@nnrp1.deja.com>
In article <··········@web.nmti.com>,
·····@abbnm.com (Peter da Silva) wrote:
> In article <····························@news.is-europe.net>,
> Rainer Joswig <······@corporate-world.lisp.de> wrote:
> > In article <··········@web.nmti.com>, ·····@abbnm.com (Peter da Silva)
> > wrote:
> > > In article <··············@kelso.bothner.com>,
> > > Per Bothner <···@bothner.com> wrote:
> > > > Note the context of this discussion: Is cdr-coding a good idea?
>
> > > CDR-coding is an implementation technique for hiding an array structure
> > > behind a list structure.
>
> > CDR-coding is an implementation technique for optimizing
> > the space usage and the locality of lists.
>
> I see these two statements as being equivalent in the context of this
> discussion.
Symbolics was concerned with space usage. You don't get
constant time access of arrays. Accessing the nth
element of a cdr-coded list is not equivalent in terms of speed
to accessing the nth element of an array.
Sent via Deja.com
http://www.deja.com/
From: ···@larva.flyingcroc.NO.SPAM.PLEASE.net
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <90v4b90o77@news1.newsguy.com>
>
> Re: Could CDR-coding be on the way back?
>
> From: Per Bothner <···@bothner.com>
> Date: 09 Dec 2000 11:53:01 -0800
>
>"Frank A. Adrian" <·······@uswest.net> writes:
>
>> So, again we see that the world simply re-invents Lisp but, more
>> importantly, one of the neatest hacks from the olden days might be
>> reasonable again. Opinions?
>
>I have an even more efficient and radical idea: Don't use lists.
>They are almost always the wrong data structure. Use arrays.
>Arrays are available and efficient in Lisp, Scheme, Java, C++,
>Fortran, etc etc.
>
>If you need variable-sized data structures, you should still use
>arrays. Just double the size of the array when it becomes too small.
>You double the array to make sure that amortized (average) cost of
>insertions is constant. Even just after you've doubled the array and
>so almost half the space is unused, you are not using any more space
>than a linked list would take.
You mean like this?:
http://www.andansbutt.org/ttk/extant
There's a statement about efficiency of doubling array sizes
in the documentation that could have almost been a quote from
your paragraph above :-)
-- TTK
--
"Live and Learn; Die and Learn Faster"
(motto of the Lynbrook Historical Simulations club)
Technical and RKBA documents at: http://www.ciar.org/~ttk/
···@larva.flyingcroc.NO.SPAM.PLEASE.net wrote:
>
> >
> > Re: Could CDR-coding be on the way back?
> >
> > From: Per Bothner <···@bothner.com>
> > Date: 09 Dec 2000 11:53:01 -0800
> >
> >"Frank A. Adrian" <·······@uswest.net> writes:
> >If you need variable-sized data structures, you should still use
> >arrays. Just double the size of the array when it becomes too small.
> >You double the array to make sure that amortized (average) cost of
> >insertions is constant. Even just after you've doubled the array and
> >so almost half the space is unused, you are not using any more space
> >than a linked list would take.
>
> You mean like this?:
> http://www.andansbutt.org/ttk/extant
> There's a statement about efficiency of doubling array sizes
> in the documentation that could have almost been a quote from
> your paragraph above :-)
Except for a few CPU intensive applications, the flexibility of
lists far compensates for the loss of speed. STL vectors, for
example, are implemented with this doubling strategy, but they
also provide the resize() member function: if you have a
2,000,000 element vector you're going to need 8Mb just to add
one element to it!
And who would sacrifice the elegance of object orientation and
the expedience and breadth of STL to program by hand in plain
vanilla C using arrays ? Besides running benchmarks and the odd
go-for-broke scientific app, I see no reason why.
At some point in the past we evolved from writing in assembler
to writing in block-structured languages; we're in the process
of leaving BS for Object Orientation. Dynamic memory allocation
is of fundamental importance now, and we use it even when we
must give away some performance.
Alberto.
In article <·················@moreira.mv.com>, Alberto Moreira wrote:
>···@larva.flyingcroc.NO.SPAM.PLEASE.net wrote:
>
> [...] if you have a
>2,000,000 element vector you're going to need 8Mb just to add
>one element to it!
I implemented a similar idea for dynamic arrays but with the addition of
a "bump-size" parameter. The idea is to double the size of the array
until it hits the bump-size, say 4Kb, and thereafter only grow the array
in multiples of bump-size.
Of course the downside is to grow an 8Mb array still requires *another*
8Mb plus 4Kb to grow, assuming that realloc() will be unable to simply
resize the original array. But the original 8Mb will be freed.
But note that if the data is stored in a singly-linked list, it still
eats up 16Mb of memory - 8Mb for the data plus another 8Mb for each
pointer.
For really large arrays such as the 8Mb example, a combined approach of
a linked-list of smaller (say 16Kb) arrays would seem to be a more
manageable solution, for a modest increase in code-size.
But I digress, and no doubt bore to tears.
-- Parag Patel
Parag Patel wrote:
> For really large arrays such as the 8Mb example, a combined approach of
> a linked-list of smaller (say 16Kb) arrays would seem to be a more
> manageable solution, for a modest increase in code-size.
You know, Lisp lists aren't really lists but more like trees.
And a tree can be represented with an array, so if push comes to
shove it is probably possible to represent lists as arrays too,
and no pointers. But then, the memory allocation granularty
issue comes back.
Alberto.
"Alberto Moreira" <·······@moreira.mv.com> wrote in message
······················@moreira.mv.com...
> Parag Patel wrote:
>
> > For really large arrays such as the 8Mb example, a combined approach of
> > a linked-list of smaller (say 16Kb) arrays would seem to be a more
> > manageable solution, for a modest increase in code-size.
>
> You know, Lisp lists aren't really lists but more like trees.
> And a tree can be represented with an array, so if push comes to
> shove it is probably possible to represent lists as arrays too,
> and no pointers.
But if push comes to pop, then it is probably a stack :-) (I'm sorry,
I couldn't resist. I'll go back in my hole now.)
--
- Stephen Fuld
But then, the memory allocation granularty
> issue comes back.
>
>
> Alberto.
From: Peter da Silva
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <9113hn$mat@web.nmti.com>
In article <·····················@pinhead.parag.codegen.com>,
Parag Patel <·····@pinhead.parag.codegen.com> wrote:
> For really large arrays such as the 8Mb example, a combined approach of
> a linked-list of smaller (say 16Kb) arrays would seem to be a more
> manageable solution, for a modest increase in code-size.
Sounds like UNIX 'clists'.
--
`-_-' In hoc signo hack, Peter da Silva.
'U` "Milloin halasit viimeksi suttasi?"
Disclaimer: WWFD?
Peter da Silva wrote:
>
> In article <·····················@pinhead.parag.codegen.com>,
> Parag Patel <·····@pinhead.parag.codegen.com> wrote:
> > For really large arrays such as the 8Mb example, a combined approach of
> > a linked-list of smaller (say 16Kb) arrays would seem to be a more
> > manageable solution, for a modest increase in code-size.
>
> Sounds like UNIX 'clists'.
The fun part is that this is exactly the approach which x86 forced us to
use back in the 16-bit days:
A list (or array) of arrays, with each subarray less than 64K in size.
I usually set the number of elements in each subarray such as to make it
between 32 and 64K in total size, and still a power of two, to simplify
addressing.
Since all accesses within the same subarray would be to the same
segment, the overhead of segment reloads could often/mostly be avoided,
at least for (semi-)sequential accesses.
Terje
--
- <··············@hda.hydro.com>
Using self-discipline, see http://www.eiffel.com/discipline
"almost all programming can be viewed as an exercise in caching"
In article <··············@kelso.bothner.com>,
Per Bothner <···@bothner.com> wrote:
>I have an even more efficient and radical idea: Don't use lists.
>They are almost always the wrong data structure. Use arrays.
>Arrays are available and efficient in Lisp, Scheme, Java, C++,
>Fortran, etc etc.
>
>If you need variable-sized data structures, you should still use
>arrays. Just double the size of the array when it becomes too small.
>You double the array to make sure that amortized (average) cost of
>insertions is constant. Even just after you've doubled the array and
>so almost half the space is unused, you are not using any more space
>than a linked list would take.
An interesting case study is symbolic algebra systems.
Of the 4 major systems:
Reduce = based on lisp ==> uses lists
Macsyma = based on lisp ==> uses lists
Mathematica = ?? written in its own list-based programming language ??
Maple = written in its own array-based programming language
The 3 "M" systems all offer roughly comparable functionality (Reduce
is a bit behind). But Maple is generally regarded as a factor of 3-5
faster, and anywhere from a factor of 2 to a factor of 10 smaller in
memory usage than the other two.
Maple uses contiguous arrays, or trees whose nodes are contiguous arrays,
for all its data structures. The authors state that this is one reason
(though certainly not the only one) for their performance advantages.
--
-- Jonathan Thornburg <······@thp.univie.ac.at>
http://www.thp.univie.ac.at/~jthorn/home.html
Universitaet Wien (Vienna, Austria) / Institut fuer Theoretische Physik
Q: Only 7 countries have the death penalty for children. Which are they?
A: Congo, Iran, Nigeria, Pakistan[*], Saudi Arabia, United States, Yemen
[*] Pakistan moved to end this in July 2000. -- Amnesty International,
http://www.web.amnesty.org/ai.nsf/index/AMR511392000
From: ········@cc.hut.fi
Subject: Re: Could CDR-coding be on the way back?
Date:
Message-ID: <m3n1e1zfjy.fsf@mu.tky.hut.fi>
······@mach.thp.univie.ac.at (Jonathan Thornburg) writes:
> Mathematica = ?? written in its own list-based programming language ??
IIRC what Mathematica calls lists are really arrays. I would expect its
(lack of) performance to be more related to the weird evaluation model.
Hannu Rummukainen
On 09 Dec 2000 11:53:01 -0800, Per Bothner <···@bothner.com>
wrote:
>"Frank A. Adrian" <·······@uswest.net> writes:
>
>> So, again we see that the world simply re-invents Lisp but, more
>> importantly, one of the neatest hacks from the olden days might be
>> reasonable again. Opinions?
>
>I have an even more efficient and radical idea: Don't use lists.
>They are almost always the wrong data structure. Use arrays.
>Arrays are available and efficient in Lisp, Scheme, Java, C++,
>Fortran, etc etc.
>
>If you need variable-sized data structures, you should still use
>arrays. Just double the size of the array when it becomes too small.
>You double the array to make sure that amortized (average) cost of
>insertions is constant. Even just after you've doubled the array and
>so almost half the space is unused, you are not using any more space
>than a linked list would take.
>
>If it is important to be able to insert/delete in the middle of
>a sequence, you might consider using a list. But consider a
>buffer-gap array first. Instead of leaving the unused space at
>teh end of the array, allow it to be in the middle, whereever
>insertions/deletions are done. You need to "move the gap" (i.e.
>copy things around in the data structure) when the gap is not where
>you need to insert/delete, but most of the time insertions/deletions
>will tend to be clustered (near each other).
>
>A arguemnt could be made that lists are easy to use in a functional
>(side-effect-free) programs - just use recursion. But I don't think
>that is a strong argument. You can write clean effecient side-effect-free
>code using arrays: Just look at APL and its successors. Modern
>functional languages have "list comprehensions" - these can just as
>easily be used for "array comprehensions".
>
>I am talking about standard Lisp-style lists implemented using pairs.
>There is a better case to be made for chaining objects together using
>link slot in a more complex objects. At least in that case you're not
>allocating a separate cons cell for each link.
One technique I've used for threaded transactions with variable
numbers of entries of the same fixed length is to use dynamic
array sizing (with doubling) for allocation, with the list
entries as array entries. You can then use indices into the array
instead of pointers, and reduce the allocation overhead from
thousands of small chunks of memory to a dozen or so
reallocations of a single chunk, and a very simple custom subpool
allocator, that never reuses memory, until it can all be freed as
one chunk.
Of course, it's always better if you can compute an upper bound
on all the memory that may be required at the very start, and
allocate it all at one time, but that can only be done where you
know all the parameter values required to size the space at the
start of fairly simple transactions.
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada
--
············@CSi.com (Brian dot Inglis at SystematicSw dot ab dot ca)
use address above to reply