From: Dave Roberts
Subject: Data structure for raw byte manipulation
Date: 
Message-ID: <XEiQc.228991$JR4.91435@attbi_s54>
Okay, so say I want to interface to the outside world with an off-the-shelf,
non-ASCII protocol. (Like through sockets, say.) What would be the best
data structure to store things in on the Lisp side? A specialized array of
(unsigned-byte 32) ? Is there any way outside of an FFI to describe a
structure that can be overlayed on a buffer such that I don't have to
manually cobble up muti-byte words using shifts and ORs?

Ideally, I'd like something C-like where I can decode a bit of something,
then basically cast a pointer+offset to a structure overlay and use that to
do efficient load/stores of multi-byte quantities.

I doubt I'll be so lucky as to find I can do this in an ANSI standard
fashion, but I'd be interested in whether it's even possible with
proprietary extensions. I'm using SBCL/CMUCL for those that know those
well.

Thanks for any assistance.

-- 
Dave Roberts, ·············@re-move.droberts.com
Slowly but surely, the programming world is finding Lisp...
http://www.findinglisp.com/blog

From: Carl Shapiro
Subject: Re: Data structure for raw byte manipulation
Date: 
Message-ID: <ouyhdriqct7.fsf@panix3.panix.com>
Dave Roberts <·············@re-move.droberts.com> writes:

> Ideally, I'd like something C-like where I can decode a bit of something,
> then basically cast a pointer+offset to a structure overlay and use that to
> do efficient load/stores of multi-byte quantities.

See my post from the "Favorite C technique to Lisp" thread on this
subject.  You can read it on Google Groups through the URL below.

http://groups.google.com/groups?safe=images&ie=UTF-8&·························@panix3.panix.com&lr=&hl=en   

> I doubt I'll be so lucky as to find I can do this in an ANSI standard
> fashion, but I'd be interested in whether it's even possible with
> proprietary extensions. I'm using SBCL/CMUCL for those that know those
> well.

Well, the ANSI standard gives implementors enough latitude to
implement displaced arrays in the way I had described, but in practice
few do.  This should come as no surprise as the most sophisticated
array optimizations you will find in a contemporary Lisp system are
those to minimize spurious CONSing with AREF.  For example, you will
be hard pressed to come up with a Lisp system that performs strength
reduction of array references!  Granted, a sagacious human optimizer
may be able recast his loops using ROW-MAJOR-AREF to recover some lost
performance, but many Lisps, such as MCL and its ilk, have a totally
brain-damaged and counterintuitive implementation of this essential
primitive.  At the end of the day I consider this a non-solution
anyway because mangling code to appease a naive compiler invariably
results in the complete occlusion of the physics or linear algebra
that you originally set out to codify.  (Dusty-deck Lisp code,
anyone?)
     
Our compiler people need to do more for application codes that only
eat things which come in neat rectangular shapes.
From: Dave Roberts
Subject: Re: Data structure for raw byte manipulation
Date: 
Message-ID: <l_rQc.207700$a24.166550@attbi_s03>
Carl Shapiro wrote:

> Dave Roberts <·············@re-move.droberts.com> writes:
> 
>> Ideally, I'd like something C-like where I can decode a bit of something,
>> then basically cast a pointer+offset to a structure overlay and use that
>> to do efficient load/stores of multi-byte quantities.
> 
> See my post from the "Favorite C technique to Lisp" thread on this
> subject.  You can read it on Google Groups through the URL below.
> 
>
http://groups.google.com/groups?safe=images&ie=UTF-8&·························@panix3.panix.com&lr=&hl=en

Okay, that was somewhat helpful in that at least I feel like there may be
some way to do it. The problem the OP of that thread had is definitely the
same thing, approximately, as I'm asking. Can you supply any example code
that demonstrates this technique, hopefully that works in something like
CMUCL/SBCL (but I'll take anything)?

My alternative may be to do everything through the FFI. That is, allocate a
buffer outside Lisp to hold the raw data, manipulate the socket calls to
dump the data there, then use the FFI to declare structure overlays and
cast pointers about.

That would not be bad, but it leaves open the possibility of leaking memory,
etc., which I would rather avoid if possible.

Thanks,

-- 
Dave Roberts, ·············@re-move.droberts.com
Slowly but surely, the programming world is finding Lisp...
http://www.findinglisp.com/blog
From: Julian Stecklina
Subject: Re: Data structure for raw byte manipulation
Date: 
Message-ID: <86k6wdpri6.fsf@goldenaxe.localnet>
Carl Shapiro <·············@panix.com> writes:

> Dave Roberts <·············@re-move.droberts.com> writes:
>
>> Ideally, I'd like something C-like where I can decode a bit of something,
>> then basically cast a pointer+offset to a structure overlay and use that to
>> do efficient load/stores of multi-byte quantities.
>
> See my post from the "Favorite C technique to Lisp" thread on this
> subject.  You can read it on Google Groups through the URL below.
>
> http://groups.google.com/groups?safe=images&ie=UTF-8&·························@panix3.panix.com&lr=&hl=en   

This thread is not very helpful in itself. It just gives history
lessons and some vague guess that you could hack you Lisp compiler to
do it...

To stay with the topic: Say I got a IP packet. How would I efficiently
decode it?

Regards,
-- 
                    ____________________________
 Julian Stecklina  /  _________________________/
  ________________/  /
  \_________________/  LISP - truly beautiful
From: Christian Pietsch
Subject: Re: Data structure for raw byte manipulation
Date: 
Message-ID: <cetogc$73pql$1@hades.rz.uni-saarland.de>
Julian Stecklina wrote in article <··············@goldenaxe.localnet>:
> To stay with the topic: Say I got a IP packet. How would I efficiently
> decode it?

You may want to look at the nicely documented CL code that got posted to
gmane.lisp.sources.code:
Subject: packet.lisp, version 2
Date: Mon, 12 Jul 2004 12:11:54 +0200
Message-ID: <··············@dodo.bluetail.com>

Web viewing available:
http://article.gmane.org/gmane.lisp.sources.code/31

Hope this helps.
Christian

-- 
  Christian Pietsch
  http://www.interling.de