From: rif
Subject: How can I read raw floating point data?
Date: 
Message-ID: <wj0lm5cz13c.fsf@five-percent-nation.mit.edu>
In Common Lisp (I'm using CMUCL, if there's some CMUCL-specific way),
how can I read a file which contains raw 32-bit IEEE little-endian
floats?  I'm imagining doing something like read-byte where I set the
:element-type to 'float, but that doesn't work because the
:element-type for a binary stream has to be a subtype of integer.
I've poked around the hyperspec for a bit, but I can't figure out how
to do it.

In case my writing isn't clear, I'd like to mimic the effect of this
C++ program (sorry for posting C++ here) which reads and prints a
single float from a file:

#include <fstream>

int main(int argc, char **argv) {
  ifstream f;
  float fl;

  f.open(argv[1], ios::binary);
  f.read((char *)&fl, sizeof(float));
  cout << fl << endl;

  f.close();
}

Thanks in advance for any help or advice you can offer.

Cheers,

rif

From: Vassil Nikolov
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <u4rc0f819.fsf@poboxes.com>
    On 05 Oct 2002 23:40:39 -0400, rif <···@mit.edu> said:

     > In Common Lisp (I'm using CMUCL, if there's some CMUCL-specific way),
     > how can I read a file which contains raw 32-bit IEEE little-endian
     > floats?

Using the FFI, as Bulent Murtezaoglu suggested, will certainly be
faster, but you can also do it all in Lisp by reading 32-bit
unsigned integers from the input stream by supplying :ELEMENT-TYPE
'(UNSIGNED-BYTE 32), using LDB to extract the parts of the float as
integers, and then computing the float in Lisp from those.

In this way you wouldn't have to rely on the C implementation using
little-endian IEEE as the format of floats.  (But maybe all C
implementations these days use this format anyway?)

---Vassil.

-- 
Garbage collection is charged at 0.19e-9 cents a cons.  Bulk rates
are also available: please contact memory management for details.
From: Raymond Toy
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <4n4rbycpo3.fsf@rtp.ericsson.se>
>>>>> "Vassil" == Vassil Nikolov <········@poboxes.com> writes:

    Vassil>     On 05 Oct 2002 23:40:39 -0400, rif <···@mit.edu> said:
    >> In Common Lisp (I'm using CMUCL, if there's some CMUCL-specific way),
    >> how can I read a file which contains raw 32-bit IEEE little-endian
    >> floats?

    Vassil> Using the FFI, as Bulent Murtezaoglu suggested, will certainly be
    Vassil> faster, but you can also do it all in Lisp by reading 32-bit
    Vassil> unsigned integers from the input stream by supplying :ELEMENT-TYPE
    Vassil> '(UNSIGNED-BYTE 32), using LDB to extract the parts of the float as
    Vassil> integers, and then computing the float in Lisp from those.

    Vassil> In this way you wouldn't have to rely on the C implementation using
    Vassil> little-endian IEEE as the format of floats.  (But maybe all C
    Vassil> implementations these days use this format anyway?)

Well that might be true if your world consists of only x86.
Certainly not true on my big-endian Sparc.

Ray
From: Bulent Murtezaoglu
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <873crkgq0u.fsf@acm.org>
>>>>> "rif" == rif  <···@mit.edu> writes:

    rif> In Common Lisp (I'm using CMUCL, if there's some
    rif> CMUCL-specific way), 

I'd too would be interested in a better answer.  But for something
that works right now, you can punt to C from CMUCL, read using C/Unix
'read' (or whatever you want to use that calls it) and return the
float to Lisp.  The CMUCL manual has worked out examples on how to
call C functions and it is not hard (I am looking right now).  You
need the at least the chapter on Alien Objects (Chp 8 in my dated
copy).  If you want to access the file from both Lisp and C you also
need to read the chapter on the Unix interface (Chp 6 in mine) to 
create a Lisp stream using your file descriptor.

cheers,

BM
From: Paul Foley
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <m27kgwyt8b.fsf@mycroft.actrix.gen.nz>
On Sun, 06 Oct 2002 04:17:18 GMT, Bulent Murtezaoglu wrote:

>>>>>> "rif" == rif  <···@mit.edu> writes:
rif> In Common Lisp (I'm using CMUCL, if there's some
rif> CMUCL-specific way), 

> I'd too would be interested in a better answer.  But for something

(KERNEL:MAKE-SINGLE-FLOAT n) makes a float using the bit pattern given
by the 32 bit integer n.

(KERNEL:MAKE-DOUBLE-FLOAT n m) does the same for a double-float, with
n being the high order bits (sign bit, exponent, and 20 bits of the
significand) and m the low order bits (the remaining 32 bits of the
significand)


[You probably also want to know about KERNEL:SINGLE-FLOAT-BITS,
KERNEL:DOUBLE-FLOAT-HIGH-BITS and KERNEL:DOUBLE-FLOAT-LOW-BITS which
do the opposite, making integers out of floats]

-- 
And ælc þara þe gehierð þas min word, and þa ne wyrcþ, se bið gelic þæm
dysigan menn...

(setq reply-to
  (concatenate 'string "Paul Foley " "<mycroft" '(··@) "actrix.gen.nz>"))
From: Rolf Wester
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <3DA13B26.71DB1DD6@ilt.fhg.de>
Paul Foley wrote:

> [You probably also want to know about KERNEL:SINGLE-FLOAT-BITS,
> KERNEL:DOUBLE-FLOAT-HIGH-BITS and KERNEL:DOUBLE-FLOAT-LOW-BITS which
> do the opposite, making integers out of floats]
>

Hi,

is there any documentation of all the CMUCL functions other than the
sources or "describe"?

Rolf Wester
From: Raymond Toy
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <4n8z1acpqc.fsf@rtp.ericsson.se>
>>>>> "Rolf" == Rolf Wester <······@ilt.fhg.de> writes:

    Rolf> Paul Foley wrote:

    >> [You probably also want to know about KERNEL:SINGLE-FLOAT-BITS,
    >> KERNEL:DOUBLE-FLOAT-HIGH-BITS and KERNEL:DOUBLE-FLOAT-LOW-BITS which
    >> do the opposite, making integers out of floats]
    >> 

    Rolf> Hi,

    Rolf> is there any documentation of all the CMUCL functions other than the
    Rolf> sources or "describe"?

No.  "Use the source"

Ray
From: Thomas F. Burdick
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <xcvr8f29lvl.fsf@monsoon.OCF.Berkeley.EDU>
Raymond Toy <···@rtp.ericsson.se> writes:

> >>>>> "Rolf" == Rolf Wester <······@ilt.fhg.de> writes:
> 
>     Rolf> Paul Foley wrote:
> 
>     >> [You probably also want to know about KERNEL:SINGLE-FLOAT-BITS,
>     >> KERNEL:DOUBLE-FLOAT-HIGH-BITS and KERNEL:DOUBLE-FLOAT-LOW-BITS which
>     >> do the opposite, making integers out of floats]
>     >> 
> 
>     Rolf> Hi,
> 
>     Rolf> is there any documentation of all the CMUCL functions other than the
>     Rolf> sources or "describe"?
> 
> No.  "Use the source"

Or, even better, ask here or on the cmucl-help list to find out what
functions you want to know about, and then use the source.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: Lennart Staflin
Subject: Re: How can I read raw floating point data?
Date: 
Message-ID: <m2adlqshw4.fsf@lysator.liu.se>
rif <···@mit.edu> writes:

> In Common Lisp (I'm using CMUCL, if there's some CMUCL-specific way),
> how can I read a file which contains raw 32-bit IEEE little-endian
> floats?

I have a similar problem. I need to read and write floats using IEEE
binary format when doing IIOP (CORBA) marshaling. I haven't really
solved it in a portable way. The scale-float and integer-decode-float
would seem to be usable for this, but they are implementation
dependent. First I read the float as an integer (this takes care of
endian issues). Then use the following code

(defun ieee-integer-to-float (raw float-type-zero
                                     sign-bit expn-bits fraction-bits bias)
  (if (zerop raw)
    float-type-zero
    (let ((fraction (+ (ldb (byte fraction-bits 0) raw)
                       (ash 1 fraction-bits)))
          (exponent (ldb (byte expn-bits fraction-bits) raw))
          (sign  (logbitp sign-bit raw)))
      (* (if sign -1 1)
         (scale-float (float fraction float-type-zero)
                      (+ exponent 
                         (- bias)
                         (- fraction-bits)))))))

For 32bit float: (ieee-integer-to-float x 0s0 31 8 23 127)
For 64bit float: (ieee-integer-to-float x 0d0 63 11 52 1023)

This seems to work on MCL (PowerPC) and ACL5 (x86). 

There is obviously some assumptions in the above code. I can't quite
characterize what they are (presumably the implementation has to use
IEEE floats internally).

Is there a portable way of doing this?


//Lennart Staflin