Hi: How would you read in binary floating point numbers from binary
files? I cannot really find useful lisp code or examples which would
teach me somehting. In Bigloo I do it as follows: Bigloo has some
undocumented functions (ieee-string->double string), where 'string'
here denotes a 4byte.
The Common Lisp version should be fast and it should have the option
to chose between little and big endian. I am not saying I am going to
use Lisp now, but my Fortran code spends most if its time reading in
files (from global climate and chemistriy and satellite simulations
and observations). A day worths of data has typically 10 to 15 arrays
of dimension 7x7x50000 times 365 days + the binary from the climate
models. The code walks more than once through the years until the
solution converges (typically after 5 iterations and a couple of
hours).
For example Bigloo would take nearly 17 seconds to decode a binary
file of dimension 7x7x50000. Nonsensly slow. However, a dramatic boost
can be obtained by using (mmap-get) instead of simple ports. mmap
reads in the binary file as a string and the time to decode 64bit
floats into a 7x7x60000 array takes 0.8 seconds (instead of 17seconds
with ports).
I am not requesting other people to work things out here for me.
However, I would like to learn.
Thanks for any pointers (I am off to the states to SF for a
conference, so likeley cannot quickly answer any further questions),
KingKong
==
Bigloo code:
The binary files consist of 3 integer dimensions (dim1,dim2,dim3), and
the 64bit float numbers
==
;; converts 8 bytes string to ieee-64bit float either little or big
endian
(define-inline
(ieee-str->f8::double string::bstring endian)
(if (eq? endian 'swap-bytes)
(ieee-string->double
(list->string (reverse (string->list string))))
(ieee-string->double string)))
;; reading in 3D array: first 3 numbers are dimensions
;; string->uint is nicked from srfi-56: it converts a string to an
integer number
(define
(rc-3d-bin::class-f64array file-name::string endian::obj)
(let* ((buf::mmap (open-mmap f-avk read: #t))
(dim1::int (string->uint (mmap-get-string buf 4) endian))
(dim2::int (string->uint (mmap-get-string buf 4) endian))
(dim3::int (string->uint (mmap-get-string buf 4) endian)))
(print "dims " dim1 " " dim2 " " dim3)
;(exit)
(let ((erg::class-f64array (make-f64array dim3 dim1 dim2 fun:
(lambda (x) (make-vector x 0.0)))))
(do ((i 0 (+fx i 1)))
((=fx i dim3)
erg)
(do ((j 0 (+fx j 1)))
((=fx j dim1))
(do ((z 0 (+fx z 1)))
((=fx z dim2))
(let ((val::double (ieee-str->f8 (mmap-get-string buf 8)
endian)))
(aset! erg val i j z))))))))
==
···········@yahoo.de wrote:
> Hi: How would you read in binary floating point numbers from binary
> files? I cannot really find useful lisp code or examples which would
> teach me somehting. In Bigloo I do it as follows: Bigloo has some
> undocumented functions (ieee-string->double string), where 'string'
> here denotes a 4byte.
Try the following
http://common-lisp.net/project/ieee-floats/
- Daniel
D Herring <········@at.tentpost.dot.com> wrote:
+---------------
| ···········@yahoo.de wrote:
| > Hi: How would you read in binary floating point numbers from binary
| > files? I cannot really find useful lisp code or examples which would
| > teach me somehting. In Bigloo I do it as follows: Bigloo has some
| > undocumented functions (ieee-string->double string), where 'string'
| > here denotes a 4byte.
|
| Try the following
| http://common-lisp.net/project/ieee-floats/
+---------------
Also see:
http://gigamonkeys.com/book/practical-parsing-binary-files.html
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607