Hello,
I'd like to implement a custom vector type, which stores fixnums and
double-floats only. It needs (amongst others) a fast SETF operation
that also coerces bignum values to double-floats. To speed things up,
it needs to be inlined, for which the possible solution is a macro. To
make it behave like a function rather than a macro, I am considering a
compiler-macro. For this type of admittedly unusual optimization it
looks like the right tool anyway.
(defun (setf varef) (value simple-vector index)
(setf (aref simple-vector index)
(typecase value
(bignum (coerce (the bignum value) 'double-float))
(t value))))
(define-compiler-macro (setf varef) (value simple-vector index)
`(setf (aref ,simple-vector ,index)
(typecase ,value
(bignum (coerce (the bignum ,value) 'double-float))
(t ,value))))
I have a few questions about it:
1. Isn't (SETF VAREF) valid as a compiler-macro name? I got errors from
some implementations. Without either this feature or inlining it does
not seem possible to avoid an extra function call inside my loops.
Maybe I'm not getting it right, maybe someone has a workaround.
2. To potentially spare TYPECASE, I could analyze a declaration provided
the function was called like (setf (varef a i) (the fixnum n)), but what
would be real great is to also access any prior declarations of the
value. I am thinking of the ability to use a kind of TYPECASE _outside_
the backquote (thus dispatching on any possible local or lexical
_declarations_), and returning code with TYPECASE in it only if no
declaration was found:
(define-compiler-macro (setf varef) (value simple-vector index)
(declared-typecase value
(bignum `(setf (aref ,simple-vector ,index)
(coerce (the bignum ,value) 'double-float)))
(fixnum `(setf (aref ,simple-vector ,index) ,value))
(t `(setf (aref ,simple-vector ,index)
(typecase ,value
(bignum (coerce (the bignum ,value)
'double-float))
(t ,value))))))
Any ideas on how to do it?
3. The Hyperspec says that it's not polite to implement compiler-macros
on CL functions (one reason for using VAREF instead of AREF). Would it
make sense to separate user-defined and implementor-defined
compiler-macros such a way that first the user-defined ones get
executed, then the implementor-defined ones on whatever the user-defined
macro returned? Maybe it works this way already? Perhaps I must create
a package, shadow cl:+ for example and define + as a dummy function
referring to cl:+ so that I can define a compiler-macro for + (which
would fall back to cl:+ if no optimization is possible)?
***
If anyone is interested in how it would be used: I've got large
vectors, which typically contain fixnums, but some of the values and
many of the aggregates contain larger integers (still fitting in 50-some
bits). I am planning to use double-floats as their representation is
very efficient in ACL, while operations like + on bignums tend to cons,
at least with on mixed fixnum - mednum data.
Thanks for any suggestions. Hopefully the relative lack of discussions
on compiler-macrolet is not an indication that it is an esoteric
facility that should be avoided. It looks wonderful to me, provided I
can make it work.
Robert
* Robert Monfera <·······@fisec.com>
| I'd like to implement a custom vector type, which stores fixnums and
| double-floats only.
from your problem description, it does not seem excessively rational to
want to store fixnums. I'd just go ahead and store double-floats.
| Hopefully the relative lack of discussions on compiler-macrolet is not an
| indication that it is an esoteric facility that should be avoided. It
| looks wonderful to me, provided I can make it work.
it is indeed a useful facility, but it helps a lot to be able to access
the declarations and (other) type information (inferences) in the
environment it is expanded.
#:Erik
Erik Naggum wrote:
> from your problem description, it does not seem excessively rational to
> want to store fixnums. I'd just go ahead and store double-floats.
It also does not seem excessively rational to me that sometimes I want
to use floats instead of integers to store currency amounts :-)
Operations on float vectors seem well optimized; I was surprised to see
them competitive with fixnums or bytes (let alone bignums). You gave me
some good and practical advice when suggesting a pure double-float
solution.
OTOH, double-floats take up double the space even when the extra bits
aren't needed (which is 95% of the time). There is only so much memory
I can use on a 32-bit address space (I didn't mention this in my
question), and it uses double the memory bandwidth.
Thanks,
Robert
Centuries ago, Nostradamus foresaw a time when Robert Monfera would say:
>Erik Naggum wrote:
>> from your problem description, it does not seem excessively rational to
>> want to store fixnums. I'd just go ahead and store double-floats.
>
>It also does not seem excessively rational to me that sometimes I want
>to use floats instead of integers to store currency amounts :-)
Using floats for something that is inherently rational seems to me to
be fundamentally irrational. (And there are too many puns there to
feel comfortable...)
>Operations on float vectors seem well optimized; I was surprised to see
>them competitive with fixnums or bytes (let alone bignums). You gave me
>some good and practical advice when suggesting a pure double-float
>solution.
>
>OTOH, double-floats take up double the space even when the extra bits
>aren't needed (which is 95% of the time). There is only so much memory
>I can use on a 32-bit address space (I didn't mention this in my
>question), and it uses double the memory bandwidth.
The *problem* with using floats for financial operations is that this
causes you to enter into all the numerical stability issues that are
rampant in FP math.
<ick-mode>
Common Lisp probably needs to have a BCD numeric representation for
this purpose.
</ick-mode>
--
Multics Emacs: a lifetime of convenience, a moment of regret.
········@ntlug.org- <http://www.hex.net/~cbbrowne/lisp.html>
Christopher Browne wrote:
> The *problem* with using floats for financial operations is that this
> causes you to enter into all the numerical stability issues that are
> rampant in FP math.
Of course. If you read my previous article, it becomes clear that I am
considering double-floats to store integers. It's just a little ugly,
wasteful and counter-intuitive, but as long as you are careful enough
not to go beyond the 50-some useful bits, you should be safe.
> <ick-mode>
> Common Lisp probably needs to have a BCD numeric representation for
> this purpose.
> </ick-mode>
Phew! Why? What's wrong with integers when you want to represent
currency amounts? Space-wise BCD is even more wasteful than using
floats. Next time you suggest Common Lisp needs that dates are
represented as mm/dd/yyyy?
Robert
* Christopher Browne
| Using floats for something that is inherently rational seems to me to
| be fundamentally irrational. (And there are too many puns there to
| feel comfortable...)
the machine representation of a floating point number _is_ a rational!
specifically, M*2^E, for Mantissa (an integer) and Exponent (an integer),
usually with a separate sign, but that's beside the point. contrary to
common beliefs, floating point numbers _are_ exact, they just aren't the
numbers we'd _like_ to be exact, because we stupidly cling to decimal
instead of hexadecimal or octal.
| The *problem* with using floats for financial operations is that this
| causes you to enter into all the numerical stability issues that are
| rampant in FP math.
this is sheer nonsense. the reason we run into numerical stability
issues in floating point arithmetic operations is that we're asking the
system to find the representable (exact) number that is closest to the
(exact, but unrepresentable) result of a computation. as long as the
result is representable, floating point arithmetic cannot but remain
exact and stable.
| <ick-mode>
| Common Lisp probably needs to have a BCD numeric representation for
| this purpose.
| </ick-mode>
supporting BCD numeric representation only makes sense when the hardware
also supports it. for all common, relevant purposes, arbitrary-precision
integers suffice.
#:Erik
In article <·················@fisec.com>,
Robert Monfera <·······@fisec.com> wrote:
>3. The Hyperspec says that it's not polite to implement compiler-macros
>on CL functions (one reason for using VAREF instead of AREF). Would it
>make sense to separate user-defined and implementor-defined
>compiler-macros such a way that first the user-defined ones get
>executed, then the implementor-defined ones on whatever the user-defined
>macro returned?
Collisions between the system and the user are not the only reason for the
prohibition. What if two applications loaded into the same environment
both tried to define a compiler macro on AREF? The second one would
override the first one.
This is why Common Lisp is very restrictive about things you can do with
symbols in the COMMON-LISP package -- they're shared by all applications
loaded into environment. Packages normally prevent applications from
conflicting with each other, but if they start modifying a shared package
like COMMON-LISP, all bets are off.
> Maybe it works this way already? Perhaps I must create
>a package, shadow cl:+ for example and define + as a dummy function
>referring to cl:+ so that I can define a compiler-macro for + (which
>would fall back to cl:+ if no optimization is possible)?
That's the approach that's normally recommended.
--
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin wrote:
> Collisions between the system and the user are not the only reason for the
> prohibition. What if two applications loaded into the same environment
> both tried to define a compiler macro on AREF? The second one would
> override the first one.
Thanks, you drove home the point. I was probably blinded by the
explanation in the standard that was specific about possible collisions
between a conforming implementation and a non-conforming user!
Robert
Robert Monfera <·······@fisec.com> writes:
> [...] I am thinking of the ability to use a kind of TYPECASE _outside_
> the backquote (thus dispatching on any possible local or lexical
> _declarations_), and returning code with TYPECASE in it only if no
> declaration was found:
>
> (define-compiler-macro (setf varef) (value simple-vector index)
> (declared-typecase value
> (bignum `(setf (aref ,simple-vector ,index)
> (coerce (the bignum ,value) 'double-float)))
> (fixnum `(setf (aref ,simple-vector ,index) ,value))
> (t `(setf (aref ,simple-vector ,index)
> (typecase ,value
> [...]
Yes, that would be nice. CLtL2 had a function called
VARIABLE-INFORMATION that could be used to find out about variable
declarations, but all the environment functions were dropped from the
ANSI standard, as people weren't happy with the design. However, that
wouldn't really be enough for your purposes, since VALUE most probably
wouldn't be a variable. Also, a good compiler can deduce much about
types that isn't directly stated by a declaration, so you'd really
want access to the compiler's information about the type -- that
doesn't fit into the compiler macro model.
Of course, a sufficiently clever compiler will use the type
information to simplify the TYPECASE your compiler macro generates,
i.e., if it sees(*)
(LET ((#:VALUE value))
(TYPECASE #:VALUE
(BIGNUM (SETF (AREF VEC I) (COERCE #:VALUE 'DOUBLE-FLOAT)))
...))
and knows the type of 'value' is BIGNUM, it would simplify that to
(LET ((#:VALUE value))
(SETF (AREF VEC I) (COERCE #:VALUE 'DOUBLE-FLOAT))
LispWorks would, when OPTIMIZE declarations permit, but YMMV.
(*) LET added on purpose; compiler macros are macros, you don't want
double evaluation.
--
Pekka P. Pirinen
Harlequin Limited/Xanalys Inc.
"Pekka P. Pirinen" wrote:
> Yes, that would be nice. CLtL2 had a function called
> VARIABLE-INFORMATION that could be used to find out about variable
> declarations, but all the environment functions were dropped from the
> ANSI standard, as people weren't happy with the design.
Hmmm. I'll try to look up any reference to the design or discussions in
the Hyperspec.
> Also, a good compiler can deduce much about
> types that isn't directly stated by a declaration, so you'd really
> want access to the compiler's information about the type -- that
> doesn't fit into the compiler macro model.
Good point, but your hint below made me think that I _don't even need_
access to compiler-inferred types in this case.
> Of course, a sufficiently clever compiler will use the type
> information to simplify the TYPECASE your compiler macro generates,
> i.e., if it sees(*)
> (LET ((#:VALUE value))
> (TYPECASE #:VALUE
> (BIGNUM (SETF (AREF VEC I) (COERCE #:VALUE 'DOUBLE-FLOAT)))
> ...))
> and knows the type of 'value' is BIGNUM, it would simplify that to
> (LET ((#:VALUE value))
> (SETF (AREF VEC I) (COERCE #:VALUE 'DOUBLE-FLOAT))
> LispWorks would, when OPTIMIZE declarations permit, but YMMV.
You got me, I wasn't sufficiently smart to figure it out - it perfectly
makes sense that something like
(typecase (the fixnum a)
(fixnum foo)
(bignum ...))
would be optimized by the compiler after the expansion (resulting just
foo), rather than me trying to dispatch expansion based on the possibly
second-guessed type. It's not easy to 'invent' something both useful
and different in the Common Lisp world :-) I cannot wait to run some
tests.
Thanks for the advice!
Regards
Robert