I have a binary stream with entries that can be ints, doubles, unsigned
longs, chars, whatever. I'm trying to read them from lisp, but all I
see is read-byte and read-char, that don't do what I want at all.
I think I want read-double, read-int, etc. But there don't seem to be
such functions.
Okay, so how do I do it? :)
Sent via Deja.com
http://www.deja.com/
David Morse <···········@my-deja.com> wrote:
> I think I want read-double, read-int, etc. But there don't seem to be
> such functions.
>
> Okay, so how do I do it? :)
Check <http://www.cs.uit.no/~frodef/sw/binary-types/>.
SY, Uwe
--
···@ptc.spbu.ru | Zu Grunde kommen
http://www.ptc.spbu.ru/~uwe/ | Ist zu Grunde gehen
David Morse wrote:
> I have a binary stream with entries that can be ints, doubles, unsigned
> longs, chars, whatever. I'm trying to read them from lisp, but all I
> see is read-byte and read-char, that don't do what I want at all.
>
> I think I want read-double, read-int, etc. But there don't seem to be
> such functions.
Please don't understand me wrong (it's not a joke) but ... implement it if
you want it. It sounds like a good solution to your problem and should not
be so difficult. You can base it on read-byte (or read-char directly for
chars). Integers would be straight forward - only doubles would be a hairy
part, I think you'll need your own binary->double conversion function.
But this would also be only a matter of fiddling out mantiss and exponent
out of the double.
Regards,
Jochen
* David Morse
| I have a binary stream with entries that can be ints, doubles, unsigned
| longs, chars, whatever.
Do you know beforehand exactly what bit patterns the stream will
contain, or do you have special encoding of the data types that lets
you read them independent of high-level type expectations?
| I'm trying to read them from lisp, but all I see is read-byte and
| read-char, that don't do what I want at all.
The Lisp reader is built from read-char, and it can build any object.
You can write a reader for your own format that can build any object,
too, using read-byte if the stream is binary. This is not always
trivial, but it can certainly be done.
| I think I want read-double, read-int, etc. But there don't seem to be
| such functions.
No, because binary formats are not defined by the language. Such
formats are either much more complex than textual representation or
they are as machine-dependent as a raw memory dump, which they much
too often actually are. You may be able to dump data to be loaded
back into the same build of the same program on the same machine with
the same version compiler and same version libraries and everything
else being the same, but otherwise, it is highly risky business to
load binary data. Therefore, such complexification as you find in
ASN.1 BER and tagged data formats from .fasl (fast load) files in Lisp
systems are necessary.
| Okay, so how do I do it? :)
You don't write binary data to begin with. :)
If you really have no choice about the data you're reading, you need
to interpret the bit patterns yourself and build Lisp objects from
them. This is not as impossible as it sounds, but some Lisps have
functions that help you do this. E.g., Allegro CL 6.0 has functions
that both hack up floating-point types into several shorts (16-bit
integers) and reassemble them, for use in binary data formats. Do
(apropos :shorts) to find them. You build integers from bytes the
easy way, but you may also specify bytes that are bigger than the
trivial 8 bits. A Lisp "byte" is as large as you want it. Specify
the :element-type when you open the stream.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
In article <················@naggum.net>,
Erik Naggum <····@naggum.net> wrote:
> * David Morse
> | I have a binary stream with entries that can be ints, doubles,
unsigned
> | longs, chars, whatever.
>
> Do you know beforehand exactly what bit patterns the stream will
> contain, or do you have special encoding of the data types that lets
> you read them independent of high-level type expectations?
>
> | I'm trying to read them from lisp, but all I see is read-byte and
> | read-char, that don't do what I want at all.
>
> The Lisp reader is built from read-char, and it can build any
object.
> You can write a reader for your own format that can build any
object,
> too, using read-byte if the stream is binary. This is not always
> trivial, but it can certainly be done.
>
> | I think I want read-double, read-int, etc. But there don't seem to
be
> | such functions.
>
> No, because binary formats are not defined by the language. Such
> formats are either much more complex than textual representation or
> they are as machine-dependent as a raw memory dump, which they much
> too often actually are. You may be able to dump data to be loaded
> back into the same build of the same program on the same machine
with
> the same version compiler and same version libraries and everything
> else being the same, but otherwise, it is highly risky business to
> load binary data. Therefore, such complexification as you find in
> ASN.1 BER and tagged data formats from .fasl (fast load) files in
Lisp
> systems are necessary.
>
> | Okay, so how do I do it? :)
>
> You don't write binary data to begin with. :)
>
> If you really have no choice about the data you're reading, you need
> to interpret the bit patterns yourself and build Lisp objects from
> them. This is not as impossible as it sounds, but some Lisps have
> functions that help you do this. E.g., Allegro CL 6.0 has functions
> that both hack up floating-point types into several shorts (16-bit
> integers) and reassemble them, for use in binary data formats. Do
> (apropos :shorts) to find them. You build integers from bytes the
> easy way, but you may also specify bytes that are bigger than the
> trivial 8 bits. A Lisp "byte" is as large as you want it. Specify
> the :element-type when you open the stream.
>
> #:Erik
I agree with you in principle, and I would love to write a text file,
but I don't think its possible to write floats as text files without
roundoff errors creeping in. Or am I wrong?
Sent via Deja.com
http://www.deja.com/
David Morse <···········@my-deja.com> writes:
> I agree with you in principle, and I would love to write a text file,
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
You are wrong. (couldn't resist)
It is possible to write out a float such that when it is read back in
you get the same bits back. Essentially, you write out enough digits
so that when it is rounded you get the same number.
Guy Steele and Jon L. White wrote a paper on how to do that quickly
and efficiently (in Lisp).
>
> Sent via Deja.com
> http://www.deja.com/
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
In article <············@content-integrity.com>,
Joe Marshall <···@content-integrity.com> wrote:
><snip>
>It is possible to write out a float such that when it is read back in
>you get the same bits back. Essentially, you write out enough digits
>so that when it is rounded you get the same number.
>
>Guy Steele and Jon L. White wrote a paper on how to do that quickly
>and efficiently (in Lisp).
Anyone have a ref to this paper?
David
·······@panix.com (David Combs) writes:
> In article <············@content-integrity.com>,
> Joe Marshall <···@content-integrity.com> wrote:
> ><snip>
> >It is possible to write out a float such that when it is read back in
> >you get the same bits back. Essentially, you write out enough digits
> >so that when it is rounded you get the same number.
> >
> >Guy Steele and Jon L. White wrote a paper on how to do that quickly
> >and efficiently (in Lisp).
>
> Anyone have a ref to this paper?
>
> David
The MIT Scheme source code mentions the following:
The Dragon4 algorithm is described in "How to print floating point
numbers accurately" by Guy L Steele Jr and Jon L White in ACM SIGPLAN
Conference on Programming Language Design and Implementation 1990
(PLDI '90).
Burger & Dybvig ("Printing Floating-Point Numbers Quickly and
Accurately" by Robert G Burger and R Kent Dybvig, PLDI '96) describe a
variant of the Dragon4 algorithm that addresses some of the efficiency
issues. It is much faster for very large or very small numbers, but
not much different to numbers within a few orders of magnitude of 1.
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Joe Marshall <···@content-integrity.com> writes:
> The MIT Scheme source code mentions the following:
>
> The Dragon4 algorithm is described in "How to print floating point
> numbers accurately" by Guy L Steele Jr and Jon L White in ACM SIGPLAN
> Conference on Programming Language Design and Implementation 1990
> (PLDI '90).
>
> Burger & Dybvig ("Printing Floating-Point Numbers Quickly and
> Accurately" by Robert G Burger and R Kent Dybvig, PLDI '96) describe a
> variant of the Dragon4 algorithm that addresses some of the efficiency
> issues. It is much faster for very large or very small numbers, but
> not much different to numbers within a few orders of magnitude of 1.
I was able to retrieve both of these articles from the ACM Digital Library.
Does anyone happen to have a reference to an implementation of either of
these algorithms in C?
-- Kaelin
>>>>> "David" == David Combs <·······@panix.com> writes:
David> In article <············@content-integrity.com>, Joe Marshall
David> <···@content-integrity.com> wrote:
>> <snip> It is possible to write out a float such that when it is
>> read back in you get the same bits back. Essentially, you write
>> out enough digits so that when it is rounded you get the same
>> number.
>>
>> Guy Steele and Jon L. White wrote a paper on how to do that quickly
>> and efficiently (in Lisp).
David> Anyone have a ref to this paper?
I think this is probably what you are referring to:
G. L. Steele and J. L. White, "How to Print Floating-Point Numbers
Accurately," pp 112-126 in _Proceedings of the ACM SIGPLAN '90
Conference of Programming Language Design and Implementation_.
White Plains, NY, June 20-22, 1990.
It is a companion to another paper:
W. D. Clinger, "How to Read Floating Point Numbers Accurately,"
pp. 99-101 in _Proceedings of the ACM SIGPLAN '90 Conference of
Programming Language Design and Implementation_. White Plains, NY,
June 20-22, 1990.
A google search of "Steele White Clinger" turned up some related
material.
--
Russell Senior ``The two chiefs turned to each other.
·······@aracnet.com Bellison uncorked a flood of horrible
profanity, which, translated meant, `This is
extremely unusual.' ''
* David Morse <···········@my-deja.com>
| I agree with you in principle, and I would love to write a text file,
| but I don't think its possible to write floats as text files without
| roundoff errors creeping in. Or am I wrong?
Yes. We have an important principle in Common Lisp: Write-read
consistency. That means that whenever an object is printed out and
*print-readably* is true, reading it back in produces an object that
consistent with the object printed out. Achieving this is not always
cheap, but it is a very, very important property of a language that
defines the operations write and read for its objects through textual
representation. Inferior languages may not go the great lengths that
are required to maintain write-read consistency, however.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
> I agree with you in principle, and I would love to write a text file,
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
You can write a float as text to a file with exactly the same precision
as its binary representation has.
The only difference to the binary representation would be a bigger
amount of data per float-value on an average.
Regards
Jochen Schmidt
David Morse <···········@my-deja.com> writes:
>
> I agree with you in principle, and I would love to write a text file,
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
I believe that not only is this possible (this is kind of obvious if
you think -- you could just write out the bit string...) but that
Common Lisp systems should actually do this -- if you write a float
and then read it back you should get something which is the same
number, assuming you haven't done anything weird with printer control
variables.
However I might be wrong about this, though I don't think so.
There is a famous paper by Jon L White (and others?) on how to do
this reasonably efficiently.
--tim
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
| David Morse <···········@my-deja.com> writes:
[printing and reading bit-accurate floating point numbers]
| There is a famous paper by Jon L White (and others?) on how to do
| this reasonably efficiently.
See "Printing Floating Point Numbers Quickly and Accurately" by Robert G.
Burger and R. Kent Dybvig:
http://www.cs.indiana.edu/%7Eburger/FP-Printing-PLDI96.ps.gz
This method is used in many Lisp, Scheme, Haskell, and Smalltalk
implementations.
--
-- Tim Olson