From: ··········@gmail.com
Subject: floating data I/O
Date: 
Message-ID: <1126934968.810527.285930@g44g2000cwa.googlegroups.com>
Hello, all.  im new to lisp and am having trouble finding info on
reading and writing floating point data to files.  From what I have
found, there is no statndard in CL for this, or am I mistaken?

From: Barry Margolin
Subject: Re: floating data I/O
Date: 
Message-ID: <barmar-3E80B3.01503817092005@comcast.dca.giganews.com>
In article <························@g44g2000cwa.googlegroups.com>,
 ··········@gmail.com wrote:

> Hello, all.  im new to lisp and am having trouble finding info on
> reading and writing floating point data to files.  From what I have
> found, there is no statndard in CL for this, or am I mistaken?

READ and WRITE will read and write most types of data, including 
floating point.

If you're looking for ways to write them in a binary format, you're 
correct that there's no standard functions for this.  You could use 
INTEGER-DECODE-FLOAT to break it up into integers, and then use 
WRITE-BYTES to write these to a file.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Pisin Bootvong
Subject: Re: floating data I/O
Date: 
Message-ID: <1126946535.828278.162730@g14g2000cwa.googlegroups.com>
Does this work if I wanted to use Lisp to read binary floating point
value written by other language, say C?
From: Christopher C. Stacy
Subject: Re: floating data I/O
Date: 
Message-ID: <u64t0xbuq.fsf@news.dtpq.com>
"Pisin Bootvong" <··········@gmail.com> writes:
> Does this work if I wanted to use Lisp to read binary floating point
> value written by other language, say C?

C does not define a portable binary floating point file format.
Even if you restrict your files to IEEE-754 machines, 
you still have an "endian" problem.  (754 doesn't specify
a byte order, and neither does the C language.)

Also, not all systems (machines, languages, compilers, etc.) 
have full IEEE-754 implementations.   
In particular, Common Lisp does not have NaNs.

I wonder if someone has written the (10 line or so)
library that reads and writes floats (in a given order).
From: Pascal Bourguignon
Subject: Re: floating data I/O
Date: 
Message-ID: <877jdfwwpr.fsf@thalassa.informatimago.com>
······@news.dtpq.com (Christopher C. Stacy) writes:

> "Pisin Bootvong" <··········@gmail.com> writes:
>> Does this work if I wanted to use Lisp to read binary floating point
>> value written by other language, say C?
>
> C does not define a portable binary floating point file format.
> Even if you restrict your files to IEEE-754 machines, 
> you still have an "endian" problem.  (754 doesn't specify
> a byte order, and neither does the C language.)
>
> Also, not all systems (machines, languages, compilers, etc.) 
> have full IEEE-754 implementations.   
> In particular, Common Lisp does not have NaNs.
>
> I wonder if someone has written the (10 line or so)
> library that reads and writes floats (in a given order).

Yes, several of us have.  I've posted mine once or twice here.

Here it is again:

extracted from: 
cvs -z3 -d ··················@cvs.informatimago.com:/usr/local/cvs/public/chrooted-cvs/cvs co common/common-lisp/heap.lisp

(defmacro gen-ieee-encoding (name type exponent-bits mantissa-bits)
  ;; Thanks to ivan4th (········@nat-msk-01.ti.ru) for correcting an off-by-1
  (wsiosbp
   `(progn
      (defun ,(intern (format nil "~A-TO-IEEE-754" name))  (float)
        (multiple-value-bind (mantissa exponent sign) 
            (integer-decode-float float)
          (dpb (if (minusp sign) 1 0)
               (byte 1 ,(1- (+ exponent-bits mantissa-bits)))
               (dpb (+ ,(+ (- (expt 2 (1- exponent-bits)) 2) mantissa-bits)
                       exponent)
                    (byte ,exponent-bits ,(1- mantissa-bits))
                    (ldb (byte ,(1- mantissa-bits) 0) mantissa)))))
      (defun ,(intern (format nil "IEEE-754-TO-~A" name))  (ieee)
        (let ((aval (scale-float
                     (coerce
                      (dpb 1 (byte 1 ,(1- mantissa-bits))
                           (ldb (byte ,(1- mantissa-bits) 0) ieee))
                      ,type)
                     (- (ldb (byte ,exponent-bits ,(1- mantissa-bits))
                             ieee) 
                        ,(1- (expt 2 (1- exponent-bits)))
                        ,(1- mantissa-bits)))))
          (if (zerop (ldb (byte 1 ,(1- (+ exponent-bits mantissa-bits))) ieee))
            aval
            (- aval)))))))


(gen-ieee-encoding float-32 'single-float  8 24)
(gen-ieee-encoding float-64 'double-float 11 53)


(defun test-ieee-read-double ()
  (with-open-file (in "value.ieee-754-double" 
                      :direction :input :element-type '(unsigned-byte 8))
    (loop while (< (file-position in) (file-length in))
          do (loop repeat 8 for i = 1 then (* i 256)
                   for v = (read-byte in) then (+ v (* i (read-byte in))) 
                   finally (progn
                             (let ((*print-base* 16)) (princ v))
                             (princ " ")
                             (princ (IEEE-754-TO-FLOAT-64 v))
                             (terpri))))))

(defun test-ieee-read-single ()
  (with-open-file (in "value.ieee-754-single" 
                      :direction :input :element-type '(unsigned-byte 8))
    (loop while (< (file-position in) (file-length in))
          do (loop repeat 4 for i = 1 then (* i 256)
                   for v = (read-byte in) then (+ v (* i (read-byte in))) 
                   finally (progn
                             (let ((*print-base* 16)) (princ v))
                             (princ " ")
                             (princ (IEEE-754-TO-FLOAT-32 v))
                             (terpri))))))

(defun test-single-to-ieee (&rest args)
  (dolist (arg args)
    (format t "~16,8R ~A~%" 
            (float-32-to-ieee-754 (coerce arg 'single-float)) arg)))

(defun test-double-to-ieee (&rest args)
  (dolist (arg args)
    (format t "~16,16R ~A~%" 
            (float-64-to-ieee-754 (coerce arg 'double-float)) arg)))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You never feed me.
Perhaps I'll sleep on your face.
That will sure show you.
From: rif
Subject: Re: floating data I/O
Date: 
Message-ID: <wj07jdfocvp.fsf@five-percent-nation.mit.edu>
For my own personal use, I've found an effective solution is to ignore
CL streams (for this task), to use UFFI to wrap C io operations
(fopen, fclose, fread, fwrite), and to leverage the fact that the CL
I use (mostly CMUCL) stores numerical arrays in a raw format, so I can
write functions like:

(defun save-df-array (array file-handle)
  (declare (type (simple-array double-float (*)) array))
  (%fwrite (sys:vector-sap array)
           8
           (length array)
           file-handle))

(defun read-df-array (array file-handle)
  (declare (type (simple-array double-float (*)) array))
  (%fread (sys:vector-sap array)
          8
          (length array)
          file-handle))

I generally wrap my calls to fopen in:

(defmacro with-fopen-file ((var path mode) &body body)
    `(let ((,var (fopen ,path ,mode)))
      (unwind-protect 
           (if (= (%fp-is-null-p ,var) 1)
               (error "Error with file ~A (mode ~A): ~A~%" ,path ,mode (%get-err
no))
               (multiple-value-prog1
                   (progn ,@body)))
        (%fclose ,var))))

Disadvantages: cannot mix binary saved arrays and other data (Lisp
forms) in a single file.  Only works for CMUCL (maybe SBCL, haven't
checked?), although implementations could be made fairly easily for
any CL that supports UFFI, with efficiency loss if the arrays aren't
represented directly (you'd have to alloc a C array, read that, then
convert).

Advantages: Fast, easy to use. 

Cheers,

rif
From: ··········@gmail.com
Subject: Re: floating data I/O
Date: 
Message-ID: <1127000018.796117.85510@o13g2000cwo.googlegroups.com>
Thanks for all the responses they are helping alot.  Is this probem
regarded as a signifcant 'shortcoming' of the CL, specifically IEEE
standards?
From: Barry Margolin
Subject: Re: floating data I/O
Date: 
Message-ID: <barmar-9A620E.20232017092005@comcast.dca.giganews.com>
In article <·······················@o13g2000cwo.googlegroups.com>,
 ··········@gmail.com wrote:

> Thanks for all the responses they are helping alot.  Is this probem
> regarded as a signifcant 'shortcoming' of the CL, specifically IEEE
> standards?

Not really, since most other languages don't have any built-in way to do 
portable binary I/O of floating point, either.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Geoffrey Summerhayes
Subject: Re: floating data I/O
Date: 
Message-ID: <_1DXe.1267$0u2.569672@news20.bellglobal.com>
<··········@gmail.com> wrote in message 
····························@o13g2000cwo.googlegroups.com...
> Thanks for all the responses they are helping alot.  Is this probem
> regarded as a signifcant 'shortcoming' of the CL, specifically IEEE
> standards?
>

Well, I remeber getting the source and data files for a game,
code compiled and ran ok but the game wouldn't run because
the data file used a 16-bit int, and the newer compiler used
32-bit.

Standards change, if the program is a one-shot deal it's not
that much of a problem. For the long term, you either have to
lock down the data format or save data in text format, both
of which have problems, but text is usually easier to deal with.

--
Geoff 
From: Christopher C. Stacy
Subject: Re: floating data I/O
Date: 
Message-ID: <ud5n4ric4.fsf@news.dtpq.com>
"Geoffrey Summerhayes" <·············@hotmail.com> writes:

> <··········@gmail.com> wrote in message 
> ····························@o13g2000cwo.googlegroups.com...
> > Thanks for all the responses they are helping alot.  Is this probem
> > regarded as a signifcant 'shortcoming' of the CL, specifically IEEE
> > standards?
> >
> 
> Well, I remeber getting the source and data files for a game,
> code compiled and ran ok but the game wouldn't run because
> the data file used a 16-bit int, and the newer compiler used
> 32-bit.

This was an ANSI legal Common Lisp program?

> Standards change, if the program is a one-shot deal it's not
> that much of a problem. For the long term, you either have to
> lock down the data format or save data in text format, both
> of which have problems, but text is usually easier to deal with.

I'm lost here -- what does the input data format of a program
have to do with the ANSI standard for Common Lisp, which has
never changed?
From: Steven M. Haflich
Subject: Re: floating data I/O
Date: 
Message-ID: <7R4Ze.6091$6e1.1523@newssvr14.news.prodigy.com>
Geoffrey Summerhayes wrote:

> Standards change

Standards do _not_ change.  Standards are revised, or replaced, or
ignored, but they do not change.