Hi,
I havn't posted here for a long, long time, and also havn't had the
opportunity to keep up with all the threads I've been missing for months
now. I hope that the following isn't re-hashing recent postings.
I have a Lisp program which now reads events of a socket stream. Each
event comes in, effectively, as (make-instance 'some-kind-of-event ...)
form. I have a simple event loop in the program, with the main GF being
HANDLE-EVENT.
The issue is that once events are handled, they're done with. No one
cares about them, and they're garbage. So, since this program does this
24/7, and events are streamed in continuously, and relatively fast, I
figured I'd reuse events.
Since my events are CLOS objects, I figured I could do something like
this:
(defmethod handle-event :around ((event event))
(unwind-protect (call-next-method)
;; protected:
(deallocate-instance event)))
where DEALLOCATE-INSTANCE is also a GF which, for events, stores the
events in an event cache, which is a hashtable keyed by the specific
class of the event, and the values being, say, a list of events of that
class.
In this case, one has to update ALLOCATE-INSTANCE for each subclass of
the common ancestor class EVENT (not a problem -- just define new events
types with a DEFINE-EVENT-TYPE macro that takes care of this).
What I want to know is what is necessary for me to put in the
ALLOCATE-INSTANCE (or maybe in DEALLOCATE-INSTANCE or in both) to
somehow make sure that when ALLOCATE-INSTANCE gets one of the cached
events, that it's just like a freshly allocated instance that would have
been returned by ALLOCATE-INSTANCE had I not bothered. I'd much rather
put this `cleanup' stuff in ALLOCATE-INSTANCE than DEALLOCATE-INSTANCE
for design reasons.
I'd also like to have know what others think about this general
strategy. I'm sure there may be other ways (e.g. by possibly using
CHANGE-CLASS ??), and I'd like to know about them before I write this
up.
thanks,
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> Hi,
>
> I havn't posted here for a long, long time, and also havn't had the
> opportunity to keep up with all the threads I've been missing for months
> now. I hope that the following isn't re-hashing recent postings.
>
> I have a Lisp program which now reads events of a socket stream. Each
> event comes in, effectively, as (make-instance 'some-kind-of-event ...)
> form. I have a simple event loop in the program, with the main GF being
> HANDLE-EVENT.
>
> The issue is that once events are handled, they're done with. No one
> cares about them, and they're garbage. So, since this program does this
> 24/7, and events are streamed in continuously, and relatively fast, I
> figured I'd reuse events.
Is there a measurable performance hit caused by garbage collection?
Have you tuned the GC?
From: Tim Moore
Subject: Re: efficient instance reallocation in CLOS
Date:
Message-ID: <9og0g7$5k3$0@216.39.145.192>
I think you want to take a look at REINITIALIZE-INSTANCE and
SHARED-INITIALIZE in the HyperSpec and MOP document. Do you really mean
the system ALLOCATE-INSTANCE, or your own resource management function?
I'd stay away from the former.
Tim
On 21 Sep 2001, Dave Bakhash wrote:
> Hi,
>
> I havn't posted here for a long, long time, and also havn't had the
> opportunity to keep up with all the threads I've been missing for months
> now. I hope that the following isn't re-hashing recent postings.
>
> I have a Lisp program which now reads events of a socket stream. Each
> event comes in, effectively, as (make-instance 'some-kind-of-event ...)
> form. I have a simple event loop in the program, with the main GF being
> HANDLE-EVENT.
>
> The issue is that once events are handled, they're done with. No one
> cares about them, and they're garbage. So, since this program does this
> 24/7, and events are streamed in continuously, and relatively fast, I
> figured I'd reuse events.
>
> Since my events are CLOS objects, I figured I could do something like
> this:
>
> (defmethod handle-event :around ((event event))
> (unwind-protect (call-next-method)
> ;; protected:
> (deallocate-instance event)))
>
> where DEALLOCATE-INSTANCE is also a GF which, for events, stores the
> events in an event cache, which is a hashtable keyed by the specific
> class of the event, and the values being, say, a list of events of that
> class.
>
> In this case, one has to update ALLOCATE-INSTANCE for each subclass of
> the common ancestor class EVENT (not a problem -- just define new events
> types with a DEFINE-EVENT-TYPE macro that takes care of this).
>
> What I want to know is what is necessary for me to put in the
> ALLOCATE-INSTANCE (or maybe in DEALLOCATE-INSTANCE or in both) to
> somehow make sure that when ALLOCATE-INSTANCE gets one of the cached
> events, that it's just like a freshly allocated instance that would have
> been returned by ALLOCATE-INSTANCE had I not bothered. I'd much rather
> put this `cleanup' stuff in ALLOCATE-INSTANCE than DEALLOCATE-INSTANCE
> for design reasons.
>
> I'd also like to have know what others think about this general
> strategy. I'm sure there may be other ways (e.g. by possibly using
> CHANGE-CLASS ??), and I'd like to know about them before I write this
> up.
>
> thanks,
> dave
>
>
>
>
Tim Moore <·····@herschel.bricoworks.com> writes:
> I think you want to take a look at REINITIALIZE-INSTANCE and
> SHARED-INITIALIZE
At first glance after reading this post, I'd thought that maybe I should
go back and revisit those (esp. ALLOCATE-INSTANCE), but as I have
reconsidered this, I actually believe that they are not the right
choice, and that my original instinct to go with ALLOCATE-INSTANCE was
the right way to go about it.
Note that the very _point_ of ALLOCATE-INSTANCE is to figure out how to
get the object in the first place. That behavior doesn't know about any
object cache by default, and so when you build one in, ALLOCATE-INSTANCE
instantly becomes the GF you want to play with to get the behavior I'm
after.
Also, as for the stuff that has to do with initializing the
instance...well, you might argue that if you got the object from the
cache, then at the stage where it's gonna get initialized (i.e. by
INITIALIZE-INSTANCE), now it's gonna be reinitialized instead. But my
gut says that ALLOCATE-INSTANCE is advertized to return a certain kind
of object, and in a particular state (uninitialized, slots all unbound,
etc.) I say: make ALLOCATE-INSTANCE cache-aware, but such that it
behaves the same as if there were no cache. If I put the work there,
and that's where it really belongs (IMO), then there's really no reason
for REINITIALIZE-INSTANCE.
Similar reasoning goes for SHARED-INITIALIZE, which I have almost never
had to touch, and hope I never do.
> Do you really mean the system ALLOCATE-INSTANCE, or your own resource
> management function? I'd stay away from the former.
I would *not* stay away from ALLOCATE-INSTANCE, which is a GF by design,
and an :around method on a simple metaclass,
e.g. `cached-standard-class', wouldn't be a bad thing.
This was a really useful post for me, because it had been so long since
I'd gone near ALLOCATE-INSTANCE, I forgot that it created an instance of
a metaclass, and not of the direct class you're trying to make an
instance of (i.e. some kind of event, in this case). But the nice thing
about it is that just by virtue of its name, it led me to believe that a
new metaclass is the ideal way to implement this feature, and will yeild
the least mess codewise. I just hope that it takes care of the mess
garbage-wise, which was my original intent. If it doesn't (which is
likel the case), then writing a custom resource management function is
the right way to go, and I guess that's what I'll do. But I really
wanted to avoid non-standardisms for creating these objects.
I'll give ALLOCATE-INSTANCE a shot, and see how it goes, and see if that
doesn't work, then I'll try a non-MOP solution. I just want to make
sure that if the system is allocating, about 60 CLOS instances/second,
and GC'ing them at about the same rate (steady-state kinda thing), that
this does better than using the standard allocation.
dave
From: Tim Moore
Subject: Re: efficient instance reallocation in CLOS
Date:
Message-ID: <9oh26s$r9k$0@216.39.145.192>
On 21 Sep 2001, Dave Bakhash wrote:
> Tim Moore <·····@herschel.bricoworks.com> writes:
>
> > I think you want to take a look at REINITIALIZE-INSTANCE and
> > SHARED-INITIALIZE
>
> At first glance after reading this post, I'd thought that maybe I should
> go back and revisit those (esp. ALLOCATE-INSTANCE), but as I have
> reconsidered this, I actually believe that they are not the right
> choice, and that my original instinct to go with ALLOCATE-INSTANCE was
> the right way to go about it.
Glad to be of service :P
> Note that the very _point_ of ALLOCATE-INSTANCE is to figure out how to
> get the object in the first place. That behavior doesn't know about any
> object cache by default, and so when you build one in, ALLOCATE-INSTANCE
> instantly becomes the GF you want to play with to get the behavior I'm
> after.
Assuming an object cache of completely unalllocated objects makes sense
for your application.
>
> Also, as for the stuff that has to do with initializing the
> instance...well, you might argue that if you got the object from the
> cache, then at the stage where it's gonna get initialized (i.e. by
> INITIALIZE-INSTANCE), now it's gonna be reinitialized instead. But my
> gut says that ALLOCATE-INSTANCE is advertized to return a certain kind
> of object, and in a particular state (uninitialized, slots all unbound,
> etc.) I say: make ALLOCATE-INSTANCE cache-aware, but such that it
> behaves the same as if there were no cache. If I put the work there,
> and that's where it really belongs (IMO), then there's really no reason
> for REINITIALIZE-INSTANCE.
Again, your approach makes sense if you're only worried about consing and
make every slot unbound when you're finished with the object. On the
other hand, a good reason to keep caches of objects around is to avoid
doing expensive initialization. In that case you wouldn't screw around
with ALLOCATE-INSTANCE and would instead look at SHARED-INITIALIZE and
friends.
> Similar reasoning goes for SHARED-INITIALIZE, which I have almost never
> had to touch, and hope I never do.
Pick your poison...
Tim
Tim Moore <·····@herschel.bricoworks.com> writes:
> Again, your approach makes sense if you're only worried about consing
> and make every slot unbound when you're finished with the object.
yes...I should have mentioned that initialization of these objects is
mostly trivial; most have zero or one slot, and whose values are simple
values like fixnums.
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> Tim Moore <·····@herschel.bricoworks.com> writes:
>
> > I think you want to take a look at REINITIALIZE-INSTANCE and
> > SHARED-INITIALIZE
>
> At first glance after reading this post, I'd thought that maybe I should
> go back and revisit those (esp. ALLOCATE-INSTANCE), but as I have
> reconsidered this, I actually believe that they are not the right
> choice, and that my original instinct to go with ALLOCATE-INSTANCE was
> the right way to go about it.
>
> Note that the very _point_ of ALLOCATE-INSTANCE is to figure out how to
> get the object in the first place. That behavior doesn't know about any
> object cache by default, and so when you build one in, ALLOCATE-INSTANCE
> instantly becomes the GF you want to play with to get the behavior I'm
> after.
Dave,
Is there any reason you can't just use a pool of objects? Something
like
(defmethod execute :after ((self event))
(return-to-pool self))
and
(defun make-event ()
(let ((event (get-event-from-pool-or-make-new-one)))
(initialize-event event)
event))
type thing? Delving into ALLOCATE-INSTANCE seems very heavyweight.
Of course, I don't know the entirety of your problem... :-)
--ap
--
It would be difficult to construe Larry Wall, in article
this as a feature. <·····················@netlabs.com>
Alain Picard <·······@optushome.com.au> writes:
> Is there any reason you can't just use a pool of objects? Something
> like
>
> (defmethod execute :after ((self event))
> (return-to-pool self))
>
> and
>
> (defun make-event ()
> (let ((event (get-event-from-pool-or-make-new-one)))
> (initialize-event event)
> event))
>
> type thing? Delving into ALLOCATE-INSTANCE seems very heavyweight.
Partly, you are right, and the problem is that I don't like creating
these kinds of solutions, because they're too localized. Over time,
these kinds of things pop up again and again, and I feel that the best
way to deal with it is to have a generalizable tool for dealing with
such things. I'm kinda spending the extra time here because I want to
get on the right track to making a more general solution.
Part of the issue with CLOS instances is that once they're initialized
(i.e. their slots are filled), reuse involves a bit of bookkeeping on
those slots, so that if a subsequent MAKE-INSTANCE gets one of these
resused instances, that these don't have the old slot values.
Part of the MAKE-INSTANCE deal is that it creates a fresh instance
(usually of STANDARD-CLASS), and then sets the slots with
INITIALIZE-INSTANCE, which of course calls SHARED-INITIALIZE.
The bottom line is that the system's instance allocator should not be
messed with except for the classes which one defines to have this
different, hopefully non-consing behavior. When a system has the
potential to cons up on the order of 100 objects/second, it's good to
start thinking about alternative mechanisms for instance allocation, and
I think that as this comes up more and more, it's nice to have a
metaclass and some additional class options to just write all the code
underneath to take care of it, rather than to manually define these
resourcing tools every time. I think I have some good ideas on how to
do it nicely, so it's worth a shot, and easy to benchmark how it
compares to the default.
dave
Dave Bakhash wrote:
> Alain Picard <·······@optushome.com.au> writes:
>
>> Is there any reason you can't just use a pool of objects? Something
>> like
>>
>> (defmethod execute :after ((self event))
>> (return-to-pool self))
>>
>> and
>>
>> (defun make-event ()
>> (let ((event (get-event-from-pool-or-make-new-one)))
>> (initialize-event event)
>> event))
>>
>> type thing? Delving into ALLOCATE-INSTANCE seems very heavyweight.
>
> Partly, you are right, and the problem is that I don't like creating
> these kinds of solutions, because they're too localized. Over time,
> these kinds of things pop up again and again, and I feel that the best
> way to deal with it is to have a generalizable tool for dealing with
> such things. I'm kinda spending the extra time here because I want to
> get on the right track to making a more general solution.
CLIM contains a resource-managing facility in the CLIM-SYS part.
AFAIK McCLIM contains a free implementation of that.
It may be a look worth - why reinventing another API for doing this things?
ciao,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> CLIM contains a resource-managing facility in the CLIM-SYS part.
> AFAIK McCLIM contains a free implementation of that. It may be a look
> worth - why reinventing another API for doing this things?
yes...I've never really messed with CLIM, so I didn't know this. I'll
check it out.
thanks,
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> Jochen Schmidt <···@dataheaven.de> writes:
>
> > CLIM contains a resource-managing facility in the CLIM-SYS part.
> > AFAIK McCLIM contains a free implementation of that. It may be a
> > look worth - why reinventing another API for doing this things?
>
> yes...I've never really messed with CLIM, so I didn't know this. I'll
> check it out.
I checked it out. It's something called DEFRESOURCE, and it looks very
applicable. I macroexpanded it once in LispWorks, and noticed that it
expanded to code which used another (probably non-published)
LispWorks-specific resource management facility.
thanks,
dave
> expanded to code which used another (probably non-published)
> LispWorks-specific resource management facility.
It's not documented in the current LW releases, but I wrote it to be
compatible with the Genera resource facility. You probably have some
Genera manuals left there at MIT, if you're interested.
--
Pekka P. Pirinen, Global Graphics Software
[Genera] gives you the feeling it is the One way Nature meant
programming to be, that was ignored, forgotten and/or corrupted by
other computer makers. from <http://fare.tunes.org/LispM.html>
Dave Bakhash <·····@alum.mit.edu> writes:
> I think that as this comes up more and more, it's nice to have a
> metaclass and some additional class options to just write all the
> code underneath to take care of it, rather than to manually define
> these resourcing tools every time. I think I have some good ideas
> on how to do it nicely, so it's worth a shot, and easy to benchmark
> how it compares to the default.
I took a little stab at this, and so here's a simple implementation
under LispWorks:
(defparameter *resources* (make-hash-table :test #'eq))
(defclass resourced-standard-class (standard-class) ())
(defmethod allocate-instance ((class resourced-standard-class)
&rest args &key &allow-other-keys)
(declare (ignore args))
(or (pop (gethash class *resources*))
(call-next-method)))
(defmethod deallocate-instance (instance)
(assert (typep (class-of instance) 'resourced-standard-class)
nil
"Only instances whose metaclasses are of type RESOURCED-STANDARD-CLASS can be deallocated.")
(loop
for slot-def in (clos:class-slots (class-of instance))
for slot-name = (clos:slot-definition-name slot-def)
do (slot-makunbound instance slot-name))
(pushnew instance (gethash (class-of instance) *resources*) :test #'eq)
instance)
The intention is that DEALLOCATE-INSTANCE is called by the programmer
when he/she knows that the instance in question is about to become
garbage. I know that DEALLOCATE-INSTANCE can be written somehow to
behave more like its well-defined CLOS analog, and I welcome ideas,
since I'm not 100% sure what the right way is. I especially would
like to know of any improvements as far as portability goes,
i.e. across different CL implementations, though I really only care
much about LispWorks.
Again, I hope for some comments. Again, note that this accomplishes
what I was going for, which was to be able to re-use instances of
certain classes. In the case above, ALLOCATE-INSTANCE does what it is
advertized to do, AFAIK, which is to return an un-initialized instance
of a class.
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> I think that as this comes up more and more, it's nice to have a
> metaclass and some additional class options to just write all the
> code underneath to take care of it, rather than to manually define
> these resourcing tools every time. I think I have some good ideas
> on how to do it nicely, so it's worth a shot, and easy to benchmark
> how it compares to the default.
I took a little stab at this, and so here's a simple implementation
under LispWorks:
(defparameter *resources* (make-hash-table :test #'eq))
(defclass resourced-standard-class (standard-class) ())
(defmethod allocate-instance ((class resourced-standard-class)
&rest args &key &allow-other-keys)
(declare (ignore args))
(or (pop (gethash class *resources*))
(call-next-method)))
(defmethod deallocate-instance (instance)
(assert (typep (class-of instance) 'resourced-standard-class)
nil
"Only instances whose metaclasses are of type RESOURCED-STANDARD-CLASS can be deallocated.")
(loop
for slot-def in (clos:class-slots (class-of instance))
for slot-name = (clos:slot-definition-name slot-def)
do (slot-makunbound instance slot-name))
(pushnew instance (gethash (class-of instance) *resources*) :test #'eq)
instance)
The intention is that DEALLOCATE-INSTANCE is called by the programmer
when he/she knows that the instance in question is about to become
garbage. I know that DEALLOCATE-INSTANCE can be written somehow to
behave more like its well-defined CLOS analog, and I welcome ideas,
since I'm not 100% sure what the right way is. I especially would
like to know of any improvements as far as portability goes,
i.e. across different CL implementations, though I really only care
much about LispWorks.
Again, I hope for some comments. Again, note that this accomplishes
what I was going for, which was to be able to re-use instances of
certain classes. In the case above, ALLOCATE-INSTANCE does what it is
advertized to do, AFAIK, which is to return an un-initialized instance
of a class.
dave
David Bakhash wrote:
>
> Dave Bakhash <·····@alum.mit.edu> writes:
> I took a little stab at this, and so here's a simple implementation
> under LispWorks:
<snip>
I am curious:
How much faster is resourced-standard-class at making instances? I mean
given the re-use level you anticipate in the event-processing problem.
What fraction of the total processing cost of an event went to
instantiation of the event, before and after?
ken
Kenny Tilton <·······@nyc.rr.com> writes:
> I am curious:
>
> How much faster is resourced-standard-class at making instances? I
> mean > given the re-use level you anticipate in the
> event-processing problem.
>
> What fraction of the total processing cost of an event went to >
> instantiation of the event, before and after?
I have no clue; I havn't tested it; I just gave the implementation a
little try. My event handling system isn't done yet...and won't be
for a bit of time...I was just thinking of this more for the cuteness
of it than anthing else...and showing that the DEFRESOURCE thing
actually could have been done more elegantly, and transparently, using
DEFCLASS. It's easy to add some some slots and behavior to the
RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
dave
Kenny Tilton <·······@nyc.rr.com> writes:
> I am curious:
>
> How much faster is resourced-standard-class at making instances? I
> mean > given the re-use level you anticipate in the
> event-processing problem.
>
> What fraction of the total processing cost of an event went to >
> instantiation of the event, before and after?
I have no clue; I havn't tested it; I just gave the implementation a
little try. My event handling system isn't done yet...and won't be
for a bit of time...I was just thinking of this more for the cuteness
of it than anthing else...and showing that the DEFRESOURCE thing
actually could have been done more elegantly, and transparently, using
DEFCLASS. It's easy to add some some slots and behavior to the
RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
I should add, as well, that the only extra step that this code adds is
the uninitializing of the slots of the old objects before they return
to the resource -- i.e. the thing that loops thru the slots and
unbinds them. So I think that as far as re-use goes, and the overhead
of creating and GC'ing evanescent objects, this will likely be an
improvement over the default behavior.
dave
David Bakhash wrote:
> Kenny Tilton <·······@nyc.rr.com> writes:
>
>> I am curious:
>>
>> How much faster is resourced-standard-class at making instances? I
>> mean > given the re-use level you anticipate in the
>> event-processing problem.
>>
>> What fraction of the total processing cost of an event went to >
>> instantiation of the event, before and after?
>
> I have no clue; I havn't tested it; I just gave the implementation a
> little try. My event handling system isn't done yet...and won't be
> for a bit of time...I was just thinking of this more for the cuteness
> of it than anthing else...and showing that the DEFRESOURCE thing
> actually could have been done more elegantly, and transparently, using
> DEFCLASS. It's easy to add some some slots and behavior to the
> RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
DEFRESOURCE is meant as a generic resource managment facility - not only
for CLOS Objects.
Have you looked at the free implementations of DEFRESOURCE in CL-HTTP and
the other in McCLIM?
IMHO this stuff is well designed and portable - I see no reason to replace
it if nothing is wrong with it...
ciao,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> David Bakhash wrote:
>
> > Kenny Tilton <·······@nyc.rr.com> writes:
> >
> >> I am curious:
> >>
> >> How much faster is resourced-standard-class at making instances? I
> >> mean > given the re-use level you anticipate in the
> >> event-processing problem.
> >>
> >> What fraction of the total processing cost of an event went to >
> >> instantiation of the event, before and after?
> >
> > I have no clue; I havn't tested it; I just gave the implementation a
> > little try. My event handling system isn't done yet...and won't be
> > for a bit of time...I was just thinking of this more for the cuteness
> > of it than anthing else...and showing that the DEFRESOURCE thing
> > actually could have been done more elegantly, and transparently, using
> > DEFCLASS. It's easy to add some some slots and behavior to the
> > RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
>
> DEFRESOURCE is meant as a generic resource managment facility - not only
> for CLOS Objects.
>
> Have you looked at the free implementations of DEFRESOURCE in CL-HTTP and
> the other in McCLIM?
>
> IMHO this stuff is well designed and portable - I see no reason to replace
> it if nothing is wrong with it...
Probably not. But it would be very nice to have a
(defpackage "RESOURCES" ....)
that can be used stand-alone, distributed *separatedly*, and with a
clear License agreement.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Marco Antoniotti wrote:
>
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> David Bakhash wrote:
>>
>> > Kenny Tilton <·······@nyc.rr.com> writes:
>> >
>> >> I am curious:
>> >>
>> >> How much faster is resourced-standard-class at making instances? I
>> >> mean > given the re-use level you anticipate in the
>> >> event-processing problem.
>> >>
>> >> What fraction of the total processing cost of an event went to >
>> >> instantiation of the event, before and after?
>> >
>> > I have no clue; I havn't tested it; I just gave the implementation a
>> > little try. My event handling system isn't done yet...and won't be
>> > for a bit of time...I was just thinking of this more for the cuteness
>> > of it than anthing else...and showing that the DEFRESOURCE thing
>> > actually could have been done more elegantly, and transparently, using
>> > DEFCLASS. It's easy to add some some slots and behavior to the
>> > RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
>>
>> DEFRESOURCE is meant as a generic resource managment facility - not only
>> for CLOS Objects.
>>
>> Have you looked at the free implementations of DEFRESOURCE in CL-HTTP and
>> the other in McCLIM?
>>
>> IMHO this stuff is well designed and portable - I see no reason to
>> replace it if nothing is wrong with it...
>
> Probably not. But it would be very nice to have a
>
> (defpackage "RESOURCES" ....)
>
> that can be used stand-alone, distributed *separatedly*, and with a
> clear License agreement.
Probably - but on the other side the CLIM-SYS package as a whole is not so
much code that it would be too big as a standalone package. I could imagine
this as part of a base of a real cross-implementation package for such
extensions.
In regard to your statement on making a clear License agreement: My copy of
the defresource.lisp file of McCLIM contains a header that says it is LGPL
so what is wrong with it?
bye,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> Marco Antoniotti wrote:
>
> >
> > Jochen Schmidt <···@dataheaven.de> writes:
> >
> >> David Bakhash wrote:
> >>
> >> > Kenny Tilton <·······@nyc.rr.com> writes:
> >> >
> >> >> I am curious:
> >> >>
> >> >> How much faster is resourced-standard-class at making instances? I
> >> >> mean > given the re-use level you anticipate in the
> >> >> event-processing problem.
> >> >>
> >> >> What fraction of the total processing cost of an event went to >
> >> >> instantiation of the event, before and after?
> >> >
> >> > I have no clue; I havn't tested it; I just gave the implementation a
> >> > little try. My event handling system isn't done yet...and won't be
> >> > for a bit of time...I was just thinking of this more for the cuteness
> >> > of it than anthing else...and showing that the DEFRESOURCE thing
> >> > actually could have been done more elegantly, and transparently, using
> >> > DEFCLASS. It's easy to add some some slots and behavior to the
> >> > RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
> >>
> >> DEFRESOURCE is meant as a generic resource managment facility - not only
> >> for CLOS Objects.
> >>
> >> Have you looked at the free implementations of DEFRESOURCE in CL-HTTP and
> >> the other in McCLIM?
> >>
> >> IMHO this stuff is well designed and portable - I see no reason to
> >> replace it if nothing is wrong with it...
> >
> > Probably not. But it would be very nice to have a
> >
> > (defpackage "RESOURCES" ....)
> >
> > that can be used stand-alone, distributed *separatedly*, and with a
> > clear License agreement.
>
> Probably - but on the other side the CLIM-SYS package as a whole is not so
> much code that it would be too big as a standalone package. I could imagine
> this as part of a base of a real cross-implementation package for such
> extensions.
I disagree. IMHO, the CLIM-SYS package contains many disparate things
which should logically go in separate packages. No matter how small.
RESOURCES is one of them.
> In regard to your statement on making a clear License agreement: My copy of
> the defresource.lisp file of McCLIM contains a header that says it is LGPL
> so what is wrong with it?
I have nothing against or for the LGPL.
I am personally much more concerned about modularity and portability
issues.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Marco Antoniotti wrote:
>
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> Marco Antoniotti wrote:
>>
>> >
>> > Jochen Schmidt <···@dataheaven.de> writes:
>> >
>> >> David Bakhash wrote:
>> >>
>> >> > Kenny Tilton <·······@nyc.rr.com> writes:
>> >> >
>> >> >> I am curious:
>> >> >>
>> >> >> How much faster is resourced-standard-class at making instances? I
>> >> >> mean > given the re-use level you anticipate in the
>> >> >> event-processing problem.
>> >> >>
>> >> >> What fraction of the total processing cost of an event went to >
>> >> >> instantiation of the event, before and after?
>> >> >
>> >> > I have no clue; I havn't tested it; I just gave the implementation a
>> >> > little try. My event handling system isn't done yet...and won't be
>> >> > for a bit of time...I was just thinking of this more for the
>> >> > cuteness of it than anthing else...and showing that the DEFRESOURCE
>> >> > thing actually could have been done more elegantly, and
>> >> > transparently, using
>> >> > DEFCLASS. It's easy to add some some slots and behavior to the
>> >> > RESOURCED-STANDARD-CLASS to make it do what DEFRESOURCE advertizes.
>> >>
>> >> DEFRESOURCE is meant as a generic resource managment facility - not
>> >> only for CLOS Objects.
>> >>
>> >> Have you looked at the free implementations of DEFRESOURCE in CL-HTTP
>> >> and the other in McCLIM?
>> >>
>> >> IMHO this stuff is well designed and portable - I see no reason to
>> >> replace it if nothing is wrong with it...
>> >
>> > Probably not. But it would be very nice to have a
>> >
>> > (defpackage "RESOURCES" ....)
>> >
>> > that can be used stand-alone, distributed *separatedly*, and with a
>> > clear License agreement.
>>
>> Probably - but on the other side the CLIM-SYS package as a whole is not
>> so much code that it would be too big as a standalone package. I could
>> imagine this as part of a base of a real cross-implementation package for
>> such extensions.
>
> I disagree. IMHO, the CLIM-SYS package contains many disparate things
> which should logically go in separate packages. No matter how small.
> RESOURCES is one of them.
>
>> In regard to your statement on making a clear License agreement: My copy
>> of the defresource.lisp file of McCLIM contains a header that says it is
>> LGPL so what is wrong with it?
>
> I have nothing against or for the LGPL.
>
> I am personally much more concerned about modularity and portability
> issues.
It depends on what you want to achieve - I still cannot really follow what
stands against using the file "defresource.lisp" from McCLIM as your
standalone "RESOURCES" thingy...
Where is the problem? Is there one? I don't see it...
bye,
Jochen
--
http://www.dataheaven.de
To make clear some things of what I've written:
My use of the word "package" in the earlier posts was more like
"library" and not like ANSI-CL Packages.
bye,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> IMHO this stuff [DEFRESOURCE, as in McCLIM] is well designed and
> portable - I see no reason to replace it if nothing is wrong with
> it...
I didn't, of course, say that there was anything _particularly_ wrong
with it, except that if I can manage to _not_ change the way I code,
then I'll take that solution over any other, provided that it is
sufficiently functional for my needs.
Now, I'm not at all talking specifically about this DEFRESOURCE vs. my
MOP-based implementation, but in the most general sense:
The goal if a nice macro library is ideally to change as few of the
accepted coding practices as possible, and to provide the sought
behavior with a minimal modification to standard programming style
and techniques.
This is, of course, just my opinion. DEFRESOURCE may indeed be
implemented underneath by extending STANDARD-CLASS and
STRUCTURE-CLASS, but who cares? What changes for the every-day
programmer is the overhead that must be incurred to using this
facility over not using it. If I question the DEFRESOURCE facility as
suspect to poor design with respect to unnecessary code-writing, then
I can attempt something better. Of course, whatever I might create
won't exactly do everything that DEFRESOURCE does, but I feel that in
this case, DEFRESOURCE fails to make direct use of a basic feature of
the MOP, which I consider to be mostly standard nowadays in the Lisps
I care about, except for clisp, in which building a MOP-enabled CLOS
system is not the default, but is an option.
If in a few lines of code, I can save myself the trouble of switching
to the whole DEFRESOURCE style, get the primary feature I want, and
only have to set the metaclass of the classes I care to be resourced,
then I'll stay as far away from DEFRESOURCE as possible, except to
ascertain the features I'd like to borrow from DEFRESOURCE. I do
realize that MOP-based programs are not portable, and I'm willing to
live with it. The MOP is too useful for me to try to avoid it. I'd
rather avoid Lisp systems that don't support it. That would be much
easier.
So, Jochen, I hope this answers your question. Of course, you phrased
it in an awkward way, since I never sought out to replace DEFRESOURCE;
only to make use of the concept of resourcing, which by no means
DEFRESOURCE is responsible for. Nor did I imply that CLIM should
change any.
Lastly, there's a huge advantage to using a MOP-based implementation
over DEFRESOURCE if you can. That advantage is simply that the
programmers who use the implementation don't need to wade through
heavy macroexpansions, or figure out how to use the DEFRESOURCE macro
and all of the other non-standard things that come with it. Instead,
you use the metaclass option, perhaps with some additional class
options, and that's it. You can call me short-sighted, but I don't
see a million-and-one uses for the MOP. I think that it is ideal and
unmatched for particular tasks, and I feel this is one of them.
Yes...this means that I think DEFRESOURCE was a mistake in many
ways...but I understand why it exists. For all I know it may even
have been developed before the MOP was around to provide an
alternative approach.
dave
David Bakhash wrote:
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> IMHO this stuff [DEFRESOURCE, as in McCLIM] is well designed and
>> portable - I see no reason to replace it if nothing is wrong with
>> it...
>
> I didn't, of course, say that there was anything _particularly_ wrong
> with it, except that if I can manage to _not_ change the way I code,
> then I'll take that solution over any other, provided that it is
> sufficiently functional for my needs.
>
> Now, I'm not at all talking specifically about this DEFRESOURCE vs. my
> MOP-based implementation, but in the most general sense:
>
> The goal if a nice macro library is ideally to change as few of the
> accepted coding practices as possible, and to provide the sought
> behavior with a minimal modification to standard programming style
> and techniques.
>
> This is, of course, just my opinion. DEFRESOURCE may indeed be
> implemented underneath by extending STANDARD-CLASS and
> STRUCTURE-CLASS, but who cares? What changes for the every-day
> programmer is the overhead that must be incurred to using this
> facility over not using it. If I question the DEFRESOURCE facility as
> suspect to poor design with respect to unnecessary code-writing, then
> I can attempt something better. Of course, whatever I might create
> won't exactly do everything that DEFRESOURCE does, but I feel that in
> this case, DEFRESOURCE fails to make direct use of a basic feature of
> the MOP, which I consider to be mostly standard nowadays in the Lisps
> I care about, except for clisp, in which building a MOP-enabled CLOS
> system is not the default, but is an option.
> If in a few lines of code, I can save myself the trouble of switching
> to the whole DEFRESOURCE style, get the primary feature I want, and
> only have to set the metaclass of the classes I care to be resourced,
> then I'll stay as far away from DEFRESOURCE as possible, except to
> ascertain the features I'd like to borrow from DEFRESOURCE. I do
> realize that MOP-based programs are not portable, and I'm willing to
> live with it. The MOP is too useful for me to try to avoid it. I'd
> rather avoid Lisp systems that don't support it. That would be much
> easier.
My objection was not because of the use of the MOP - but you mentioned in
an earlier article that you want to create something more generic for
managing resources. My point was only to make clear that the
ALLOCATE-INSTANCE approach is not more generic than DEFRESOURCE (which you
did not say - I only wanted to make it clear). DEFRESOURCE solves the
problem of managing resources in a wider way.
This does _not_ mean that you should use it - you'll have to decide by
yourself what fits best in your situation.
> So, Jochen, I hope this answers your question. Of course, you phrased
> it in an awkward way, since I never sought out to replace DEFRESOURCE;
> only to make use of the concept of resourcing, which by no means
> DEFRESOURCE is responsible for. Nor did I imply that CLIM should
> change any.
DEFRESOURCE not so much tangled with CLIM as it might seem: besides of the
fact that CLIM uses it there are no other dependencies. It is somewhat of a
shame that many people see this tools as only useful if used with the
packages they originated from. DEFRESOURCE is perfectly usable in non-CLIM
code and this was why I thought it would be a good idea to mention it.
(Particularily since this question is somewhat of a frequently asked one...)
> Lastly, there's a huge advantage to using a MOP-based implementation
> over DEFRESOURCE if you can. That advantage is simply that the
> programmers who use the implementation don't need to wade through
> heavy macroexpansions, or figure out how to use the DEFRESOURCE macro
> and all of the other non-standard things that come with it. Instead,
> you use the metaclass option, perhaps with some additional class
> options, and that's it. You can call me short-sighted, but I don't
> see a million-and-one uses for the MOP. I think that it is ideal and
> unmatched for particular tasks, and I feel this is one of them.
> Yes...this means that I think DEFRESOURCE was a mistake in many
> ways...but I understand why it exists. For all I know it may even
> have been developed before the MOP was around to provide an
> alternative approach.
The bad thing is that you _have_ to use metaclasses then - You will have to
change existing classes to use this metaclass and it gets ugly if your code
already uses metaclasses. What if you would want to manage UncommonSQL
view-class instances? Yes it is a nice hack - but If I had to choose
between having a nice DB-binding or some syntactic sugar for
resource-management I would opt for the first. Another nice thing with
DEFRESOURCE is that it scales with your design and is more consistent. If
you later need to manage resources other than CLOS instances or you want to
use other metaclasses, then you would have to redo all resource-managment.
I might have sounded a bit harsh - this was not intented sorry - I only
wanted to show that DEFRESOURCE is there and that it is thought for doing
stuff like that. Nobody forces you to do anything - it's simply my opinion
that using DEFRESOURCE might pay later...
P.S.
If you don't want to hear my opinion - run! ;-)
bye,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> The bad thing is that you _have_ to use metaclasses then - You will
> have to change existing classes to use this metaclass and it gets
> ugly if your code already uses metaclasses. What if you would want
> to manage UncommonSQL view-class instances? Yes it is a nice hack -
> but If I had to choose between having a nice DB-binding or some
> syntactic sugar for resource-management I would opt for the
> first. Another nice thing with DEFRESOURCE is that it scales with
> your design and is more consistent. If you later need to manage
> resources other than CLOS instances or you want to use other
> metaclasses, then you would have to redo all resource-managment.
I'm not sure I see this to be the case. The downside to the
DEF-VIEW-CLASS system is that it deviates from using DEFCLASS, and so
you lose control of defining the metaclass of the classes you define.
But I'd guess that you can still make the metaclass an option which
defaults to STANDARD-DB-CLASS or whatever it is, but why you couldn't
use a metaclass that inherited both from STANDARD-DB-CLASS and
RESOURCED-STANDARD-CLASS or whatever I called it...I don't know.
The point about RESOURCED-STANDARD-CLASS was that it deals only with
the transient objects, and not the persistent backend, as in the
relational backend you're using in your UncommonSQL. A friendly
implementation of any class definer like DEF-VIEW-CLASS that
supersedes DEFCLASS should maintain the ability to set the metaclass.
Compile-time assertions might be in order (i.e. making sure that the
metaclass you do choose be a subclass of STANDARD-DB-CLASS or
whatever).
> I might have sounded a bit harsh - this was not intented sorry - I
> only wanted to show that DEFRESOURCE is there and that it is thought
> for doing stuff like that. Nobody forces you to do anything - it's
> simply my opinion that using DEFRESOURCE might pay later...
You don't sound harsh...Just opinionated, which we all seem to be.
But the point that I'm going for here is to see perhaps how a
MOP-based implementation that leaves macros like DEFRESOURCE out of it
can be designed. I don't know much about how you can extend
STRUCTURE-CLASS, and use DEFSTRUCT in conjunction with a subclass of
STRUCTURE-CLASS, and so yeah...I realize that DEFRESOURCE buys you
that...again, only with the coding effort. My whole point here is to
prevent this coding effort using CLOS/MOP.
> If you don't want to hear my opinion - run! ;-)
I of course read your opinions, and appreciate your time, and care
about your point of view. Otherwise, how or why would I follow up to
them?
I think what you might then be saying is "If you don't _agree_ with my
opinions, then run", and of course, this is either some kind of joke
(not terribly funny humor, with or without the smiley), or it's some
kind of "I'm dictator here...agree or leave" thing...again, silly.
dave
David Bakhash wrote:
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> The bad thing is that you _have_ to use metaclasses then - You will
>> have to change existing classes to use this metaclass and it gets
>> ugly if your code already uses metaclasses. What if you would want
>> to manage UncommonSQL view-class instances? Yes it is a nice hack -
>> but If I had to choose between having a nice DB-binding or some
>> syntactic sugar for resource-management I would opt for the
>> first. Another nice thing with DEFRESOURCE is that it scales with
>> your design and is more consistent. If you later need to manage
>> resources other than CLOS instances or you want to use other
>> metaclasses, then you would have to redo all resource-managment.
>
> I'm not sure I see this to be the case. The downside to the
> DEF-VIEW-CLASS system is that it deviates from using DEFCLASS, and so
> you lose control of defining the metaclass of the classes you define.
> But I'd guess that you can still make the metaclass an option which
> defaults to STANDARD-DB-CLASS or whatever it is, but why you couldn't
> use a metaclass that inherited both from STANDARD-DB-CLASS and
> RESOURCED-STANDARD-CLASS or whatever I called it...I don't know.
As I said... it get's ugly.
IMHO it is much more difficult to read the sources and prove that
RESOURCE-STANDARD-CLASS does not interfere with e.g. VIEW-METACLASS.
IMHO the approach looses it's elegance if you have to inherit new classes
in each of this cases.
Btw: DEF-VIEW-CLASS is only syntactic sugar - there are even examples that
use DEFCLASS.
> The point about RESOURCED-STANDARD-CLASS was that it deals only with
> the transient objects, and not the persistent backend, as in the
> relational backend you're using in your UncommonSQL. A friendly
> implementation of any class definer like DEF-VIEW-CLASS that
> supersedes DEFCLASS should maintain the ability to set the metaclass.
> Compile-time assertions might be in order (i.e. making sure that the
> metaclass you do choose be a subclass of STANDARD-DB-CLASS or
> whatever).
DEF-VIEW-CLASS is only syntactic sugar - IMHO it is ok to use DEFCLASS in
the case you would create your own inherited version of VIEW-METACLASS (it
is not called STANDARD-DB-CLASS as you call it). You could break the
protocol in a way that makes DEF-VIEW-CLASS not work anymore - therefore it
is IMHO ok to let this option to people who _know_ what they are doing (and
therefore should have no problems using DEFCLASS instead).
>> I might have sounded a bit harsh - this was not intented sorry - I
>> only wanted to show that DEFRESOURCE is there and that it is thought
>> for doing stuff like that. Nobody forces you to do anything - it's
>> simply my opinion that using DEFRESOURCE might pay later...
>
> You don't sound harsh...Just opinionated, which we all seem to be.
> But the point that I'm going for here is to see perhaps how a
> MOP-based implementation that leaves macros like DEFRESOURCE out of it
> can be designed. I don't know much about how you can extend
> STRUCTURE-CLASS, and use DEFSTRUCT in conjunction with a subclass of
> STRUCTURE-CLASS, and so yeah...I realize that DEFRESOURCE buys you
> that...again, only with the coding effort. My whole point here is to
> prevent this coding effort using CLOS/MOP.
Yes - that's ok - but the use of DEFRESOURCE is IMHO not very code intensive
but you immediately can manage CLOS objects, buffers (arrays), open sockets
and every resourced construct you can imagine with the same facility. It
gives you a consistent and simple resource-facility. You could even let
your MOP approach use DEFRESOURCEs ALLOCATE-RESOURCE and
DEALLOCATE-RESOURCE to manage the instances.
But first you should try if pooling the objects will get you _really_ some
speed gain - since the only thing you really would gain is allocating of
memory for the instances - and that could be _very_ fast compared to
initializing the slots.
>> If you don't want to hear my opinion - run! ;-)
>
> I of course read your opinions, and appreciate your time, and care
> about your point of view. Otherwise, how or why would I follow up to
> them?
>
> I think what you might then be saying is "If you don't _agree_ with my
> opinions, then run", and of course, this is either some kind of joke
> (not terribly funny humor, with or without the smiley), or it's some
> kind of "I'm dictator here...agree or leave" thing...again, silly.
Yes it was a joke. No it has nothing to do with being dictator or having
you to agree or leave - it is an old quote with a similar meaning like
"IMHO" or expressing that I realized that I'm opinionated in this
discussion. If it was offending for you - sorry - this was not intented...
bye,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> > I'm not sure I see this to be the case. The downside to the
> > DEF-VIEW-CLASS system is that it deviates from using DEFCLASS, and
> > so you lose control of defining the metaclass of the classes you
> > define. But I'd guess that you can still make the metaclass an
> > option which defaults to STANDARD-DB-CLASS or whatever it is, but
> > why you couldn't use a metaclass that inherited both from
> > STANDARD-DB-CLASS and RESOURCED-STANDARD-CLASS or whatever I
> > called it...I don't know.
>
> As I said... it get's ugly. IMHO it is much more difficult to read
> the sources and prove that RESOURCE-STANDARD-CLASS does not
> interfere with e.g. VIEW-METACLASS.
Obviously, you have a bias here. It is a silly bias. You think:
1) very many people care about the whole DEF-VIEW-CLASS thing.
2) the possibility that DEF-VIEW-CLASS _might_ be used supersedes the
right for *anyone* to use the MOP, and to define new metaclasses.
(this is, of course, absurd).
The way I see it, you've worked on something called UncommonSQL, and
feel that it is a superior package, and because you've used the same
tools I'm suggesting, and in such a way so as to possibly preclude
others from using them, that *others* should find alternative
approaches. Nonsense. I'm starting to think you really are a
dictator.
> IMHO the approach looses it's elegance if you have to inherit new
> classes in each of this cases. Btw: DEF-VIEW-CLASS is only
> syntactic sugar - there are even examples that use DEFCLASS.
The purpose of ALLOCATE-INSTANCE, and the reason it was made to be a
generic function was exactly so that if you wanted to change the way
allocation takes place, that's the function you want to be using.
Defining all sorts of new macros is *not* the best way, and how you
can possibly try to argue that using ALLOCATE-INSTANCE is less elegant
is beyond me.
> DEF-VIEW-CLASS is only syntactic sugar - IMHO it is ok to use
> DEFCLASS in the case you would create your own inherited version of
> VIEW-METACLASS (it is not called STANDARD-DB-CLASS as you call
> it). You could break the protocol in a way that makes DEF-VIEW-CLASS
> not work anymore - therefore it is IMHO ok to let this option to
> people who _know_ what they are doing (and therefore should have no
> problems using DEFCLASS instead).
Again...there's no reason why anyone should define a DEFCLASS-like
macro that prevents people who know what they're doing from using a
metaclass other than the default one.
> Yes - that's ok - but the use of DEFRESOURCE is IMHO not very code
> intensive but you immediately can manage CLOS objects, buffers
> (arrays), open sockets and every resourced construct you can imagine
> with the same facility.
This is true, though I said since the first or second post that I was
only interested in CLOS objects, and they happen to be ones which I'm
defining. I do realize that DEFRESOURCE is very generic, and very
nice, and I do like it a lot. It solves a different problem, in a
sense, and a more general problem too.
> >> If you don't want to hear my opinion - run! ;-)
> >
> > I think what you might then be saying is "If you don't _agree_
> > with my opinions, then run", and of course, this is either some
> > kind of joke (not terribly funny humor, with or without the
> > smiley), or it's some kind of "I'm dictator here...agree or leave"
> > thing...again, silly.
>
> Yes it was a joke. No it has nothing to do with being dictator or
> having you to agree or leave - it is an old quote with a similar
> meaning like "IMHO" or expressing that I realized that I'm
> opinionated in this discussion. If it was offending for you - sorry
> - this was not intented...
No problem. But I get this aweful sense of militant superiority from
you. Like arguing against using metaclasses for pooling because you
have used metaclasses for persistence...I just see this as a
superiority issue you'll probably have to deal with. No one's running
because you say run. An no one's gonna be scared to ever use
metaclasses because it might pose some problems if they try later to
interface to UncommonSQL or any other such package, according to your
dictates.
dave
David Bakhash <·····@mit.edu> writes:
> Obviously, you have a bias here. It is a silly bias. You think:
>
> 1) very many people care about the whole DEF-VIEW-CLASS thing.
> 2) the possibility that DEF-VIEW-CLASS _might_ be used supersedes the
> right for *anyone* to use the MOP, and to define new metaclasses.
> (this is, of course, absurd).
>
> The way I see it, you've worked on something called UncommonSQL, and
> feel that it is a superior package, and because you've used the same
> tools I'm suggesting, and in such a way so as to possibly preclude
> others from using them, that *others* should find alternative
> approaches. Nonsense. I'm starting to think you really are a
> dictator.
>
Dave, Dave. Lighten up. I think Jochen is saying nothing of
the sort. I think the argument is simply:
* you have to use moppish stuff to do something like persistence
* you _don't_ have to use moppish stuff to do something like resourcing
(though, of course, you _can_)
therefore
* it's better to avoid moppish stuff for resourcing, since that leaves
you open to having your cake and eating it too [i.e. persistent, resourced
objects]
I don't think anyone is dreaming of telling you what _you_ may/may not write!
Heck -- just write your fancy moppish resourcing kit, and submit it to cClan.
The more the merrier.
BTW, regarding resourcing, have you considered "doing the simplest
thing that could possibly work"? :-)
--ap
--
It would be difficult to construe Larry Wall, in article
this as a feature. <·····················@netlabs.com>
Alain Picard <·······@optushome.com.au> writes:
> BTW, regarding resourcing, have you considered "doing the simplest
> thing that could possibly work"? :-)
IMHO, the MOP solution is the simplest, since you can control aspects
of allocate-instance whereas you can't do that for non-CLOS objects.
--
-> -/- - Rahul Jain - -\- <-
-> -\- http://linux.rice.edu/~rahul -=- ·················@usa.net -/- <-
-> -/- "I never could get the hang of Thursdays." - HHGTTG by DNA -\- <-
|--|--------|--------------|----|-------------|------|---------|-----|-|
Version 11.423.999.220020101.23.50110101.042
(c)1996-2000, All rights reserved. Disclaimer available upon request.
Rahul Jain <·····@rice.edu> writes:
> Alain Picard <·······@optushome.com.au> writes:
>
> > BTW, regarding resourcing, have you considered "doing the simplest
> > thing that could possibly work"? :-)
>
> IMHO, the MOP solution is the simplest, since you can control
> aspects of allocate-instance whereas you can't do that for non-CLOS
> objects.
Yes. It is...and the tiny amount of code, and negligible changes to
subsequent coding (i.e. adding the metaclass tag) is evidence...until
I see something better.
dave
Alain Picard <·······@optushome.com.au> writes:
> David Bakhash <·····@mit.edu> writes:
>
> > The way I see it, you've worked on something called UncommonSQL,
> > and feel that it is a superior package, and because you've used
> > the same tools I'm suggesting, and in such a way so as to possibly
> > preclude others from using them, that *others* should find
> > alternative approaches. Nonsense. I'm starting to think you
> > really are a dictator.
> [...]
> I don't think anyone is dreaming of telling you what _you_ may/may
> not write! Heck -- just write your fancy moppish resourcing kit,
> and submit it to cClan. The more the merrier.
Really? I could have sworn that that's what I read:
Jochen wrote:
> What if you would want to manage UncommonSQL view-class instances?
> Yes it is a nice hack - but If I had to choose between having a nice
> DB-binding or some syntactic sugar for resource-management I would
> opt for the first.
This was some argument against using the MOP for something other than
UncommonSQL.
There are a few people on this newsgroup to have this kind of
superiority complex, and sometimes, it's nice not to rebuke them, but
to remind them that they have a problem. Considering that the subject
heading of the very first message in this thread was
"efficient instance reallocation in CLOS"
it was obvious that I was not talking about the most general resource
management tool, nor was I talking about efficient re-use of arrays,
structs, etc, of course, which no matter how hard I tried, backfired,
as I tried to explain the direction.
I was merely trying to make the following points:
1) No MOP-based programs should preclude the possibility of using
other MOP-based programs unless there's a fundamental reason why
the two don't jive well. That's why there's multiple
inheritance.
2) As much as I love macros, there happens to be a generic functional
approach to the problem I brought up, and I think that arguing
that a macro-based solution is superior for the subclass of
problems (i.e. CLOS) I am talking about is a weak argument.
3) I would like to know of any neat ideas to add to the few LOC I
wrote, which unfortunately, I havn't yet seen...I really just care
about the code...This is more an exercise, and my personal goals
here are somewhat academic, so to speak.
On that last note, I did notice a use of LOAD-TIME-VALUE in the McClim
impl. of DEFRESOURCE. If someone wouldn't mind explaining that, I'd
definitely appreciate that too.
thanks,
dave
David Bakhash wrote:
> Alain Picard <·······@optushome.com.au> writes:
>
>> David Bakhash <·····@mit.edu> writes:
>>
>> > The way I see it, you've worked on something called UncommonSQL,
>> > and feel that it is a superior package, and because you've used
>> > the same tools I'm suggesting, and in such a way so as to possibly
>> > preclude others from using them, that *others* should find
>> > alternative approaches. Nonsense. I'm starting to think you
>> > really are a dictator.
>> [...]
>> I don't think anyone is dreaming of telling you what _you_ may/may
>> not write! Heck -- just write your fancy moppish resourcing kit,
>> and submit it to cClan. The more the merrier.
>
> Really? I could have sworn that that's what I read:
>
> Jochen wrote:
>
>> What if you would want to manage UncommonSQL view-class instances?
>> Yes it is a nice hack - but If I had to choose between having a nice
>> DB-binding or some syntactic sugar for resource-management I would
>> opt for the first.
>
> This was some argument against using the MOP for something other than
> UncommonSQL.
No it was not. I _explicitely_ mentioned _me_ . If *I* _had_ to choose!
Since using multiple inheritance can buy us both if both packages are well
written this might be a non-issue but I still think that it is a valid
argument _against_ the MOP approach that the MOP is not really needed as the
gain is somewhat minor (IMHO !!!).
> There are a few people on this newsgroup to have this kind of
> superiority complex, and sometimes, it's nice not to rebuke them, but
> to remind them that they have a problem. Considering that the subject
> heading of the very first message in this thread was
Mr. David Bakhash: What do you have against me personally?
You do _not_ know me nor do you know what I think. I would suggest you to
restrict insulting offenses like calling me a dictator with a superiority
complex to private mail since _I_ can then filter away your mails easily
and others do not get annoyed because of OT discussion relating to my
person. Much nicer for me, all other and probably even you would be to calm
down and find back to a technical tone of discussion.
To make the first step: Sorry if something I said was offending or
insulting to you - it was for sure _not_ meant this way.
> "efficient instance reallocation in CLOS"
>
> it was obvious that I was not talking about the most general resource
> management tool, nor was I talking about efficient re-use of arrays,
> structs, etc, of course, which no matter how hard I tried, backfired,
> as I tried to explain the direction.
I explained that handling resources in a consistent style may make code more
readable - IMHO this is a valid argument.
> I was merely trying to make the following points:
>
> 1) No MOP-based programs should preclude the possibility of using
> other MOP-based programs unless there's a fundamental reason why
> the two don't jive well. That's why there's multiple
> inheritance.
I fully agree - but this has nothing to do with the question of _when_ to
use MOP for a problem and when not...
> 2) As much as I love macros, there happens to be a generic functional
> approach to the problem I brought up, and I think that arguing
> that a macro-based solution is superior for the subclass of
> problems (i.e. CLOS) I am talking about is a weak argument.
Exactly the same way as arguing a generic function approach would be
superior to a macro-based solution per se. DEFRESOURCE IMHO (!) not better
because it uses macros but because it does not _need_ the MOP and yet is a
more general approach. IMHO (!) this is a valid argument and valid opinion
to have.
Regards,
Jochen Schmidt
From: Tim Bradshaw
Subject: Re: efficient instance reallocation in CLOS
Date:
Message-ID: <ey3wv2fr9kl.fsf@cley.com>
* Alain Picard wrote:
> * it's better to avoid moppish stuff for resourcing, since that leaves
> you open to having your cake and eating it too [i.e. persistent, resourced
> objects]
I think that if it is not possible to have a MOP-based resourcing
system and a MOP-based persistency system coexist then probably either
the design of one of these systems is broken, or the MOP is broken.
If the latter then this is interesting from the point of language
design & possible future standardisation of a MOP.
A third alternative is that having resourced, persistent objects is
nonsensical but I presume this whole discussion would not have arisen
then.
--tim
Tim Bradshaw wrote:
>
> * Alain Picard wrote:
>
> > * it's better to avoid moppish stuff for resourcing, since that leaves
> > you open to having your cake and eating it too [i.e. persistent, resourced
> > objects]
>
> I think that if it is not possible to have a MOP-based resourcing
> system and a MOP-based persistency system coexist then probably either
> the design of one of these systems is broken, or the MOP is broken.
I have been lurking similar thoughts, based on having myself two
elaborate metaclasses (one being AllegroStore) via multiple inheritance.
I did it for good reasons and it works great, but I still marvel that it
works. Praise to the MOP, AStore and praise also to Franz for fixing and
once even tweaking AStore as stuff dropped out.
ken
clinisys
Tim Bradshaw <···@cley.com> writes:
> * Alain Picard wrote:
>
> > * it's better to avoid moppish stuff for resourcing, since that
> > leaves you open to having your cake and eating it too
> > [i.e. persistent, resourced objects]
>
> I think that if it is not possible to have a MOP-based resourcing
> system and a MOP-based persistency system coexist then probably either
> the design of one of these systems is broken, or the MOP is broken.
> If the latter then this is interesting from the point of language
> design & possible future standardisation of a MOP.
As Kenny wrote in a followup post, multiple inheritance does work for
metaclass behavior. But...here's an issue.
[Note: to really get what follows, it would help to be able to refer
to code from a prior post:
http://groups.google.com/groups?q=group:comp.lang.lisp+bakhash+clos+resourced-standard-class+defmethod&hl=en&rnum=1&selm=c29vgi3mi4c.fsf%40no-knife.mit.edu
]
Suppose that the UncommonSQL package has defined a primary method for
ALLOCATE-INSTANCE for its VIEW-METACLASS (though I'm not sure why it
would...but just for argument's sake, let's say it did...)
Let's also note that RESOURCED-STANDARD-CLASS also defined a primary
method for ALLOCATE-INSTANCE. Well, here's effectively what you start
out with:
(defclass resourced-standard-class (standard-class) ())
(defclass view-metaclass (standard-class) (...))
(defmethod allocate-instance ((class resourced-standard-class)
&rest initargs &key &allow-other-keys)
...)
(defmethod allocate-instance ((class view-metaclass)
&rest initargs &key &allow-other-keys)
...)
Now, you wish to use both in your code, so to do that, you first define
the class that inherits from both:
(defclass resourced-persistent-class (resourced-standard-class view-metaclass) ())
The reason for this, of course, is that you must specify a metaclass as
a single class. Also, the reason I placed RESOURCED-STANDARD-CLASS
before VIEW-METACLASS is specifically because the ALLOCATE-INSTANCE rule
for RESOURCED-STANDARD-CLASS is more general here...i.e. that in the
sample implementation posted a few days ago, I really did intend that
this code code kick in first. I could have coded the ALLOCATE-INSTANCE
method on RESOURCED-STANDARD-CLASS to be an :around method instead of a
primary method to be yet more certain that the
RESOURCED-STANDARD-CLASS's method for ALLOCATE-INSTANCE would really
supersede all others. Such decisions are subtle, and I believe it takes
a while before programmers get a sense of how to make such decisions. I
struggle with such things, and for good reason. In the case above,
simply changing the order of superclasses for RESOURCED-PERSISTENT-CLASS
would probably cause ALLOCATE-INSTANCE to break, unless
RESOURCED-STANDARD-CLASS's method for ALLOCATE-INSTANCE was defined as
an :around method.
Not a big deal, but worth mentioning on this topic of multiple
inheritance and method dispatch. The conclusion being that YES such
metaclasses can coexist and be used simultaneously, but one care must be
taken both in their design and use.
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> Suppose that the UncommonSQL package has defined a primary method for
> ALLOCATE-INSTANCE for its VIEW-METACLASS (though I'm not sure why it
> would...but just for argument's sake, let's say it did...)
>
> Let's also note that RESOURCED-STANDARD-CLASS also defined a primary
> method for ALLOCATE-INSTANCE. Well, here's effectively what you start
> out with:
>
> (defclass resourced-standard-class (standard-class) ())
> (defclass view-metaclass (standard-class) (...))
>
> (defmethod allocate-instance ((class resourced-standard-class)
> &rest initargs &key &allow-other-keys)
> ...)
>
> (defmethod allocate-instance ((class view-metaclass)
> &rest initargs &key &allow-other-keys)
> ...)
>
> Now, you wish to use both in your code, so to do that, you first define
> the class that inherits from both:
>
> (defclass resourced-persistent-class (resourced-standard-class view-metaclass) ())
>
Well, I think that there are two possible cases here:
1. One or both of these classes genuinely allocates its own instances
- in the sense that it never calls STANDARD-CLASS's ALLOCATE-INSTANCE.
In that case there is a real problem because the objects that are
allocated are probably `alien' (not in the `foreign language' sense -
actually maybe in that sense, maybe they are pointers to objects in
C-space or something). So such a metaclass probably can not coexist
with others.
2. Really, these classes are defining wrappers, which at some point
will or may do a CALL-NEXT-METHOD to actually get an instance. In
*this* case, then (a) they might be better defined as around methods,
and (b) it ought to be possible for the classes to coexist, since both
methods will be called. There might, I agree, be order dependencies
(and the classes should document what these are), although I think
good design would try to minimise these. There might also be problems
if one or other classes has tangled up initialisation with allocation,
but that's just bad design, I think.
I think that there may be special considerations in the case of
ALLOCATE-INSTANCE: for instance a class might assume that calling
ALLOCATE-INSTANCE returned an object which is not EQ to any other
object, which your resourced-class obviously can't ensure. Whether
that assumption is legitimate, I'm not sure.
--tim
This is the kind of post I was waiting for...
Tim Bradshaw <···@tfeb.org> writes:
> > (defclass resourced-persistent-class (resourced-standard-class view-metaclass) ())
>
> Well, I think that there are two possible cases here:
>
> 1. One or both of these classes genuinely allocates its own instances
> - in the sense that it never calls STANDARD-CLASS's ALLOCATE-INSTANCE.
Well, the point is that this is 100% fine if any primary method's
implementation uses a CALL-NEXT-METHOD. I'd argue that a correct
understanding of what each metaclass does, as well as a correct
implementation of ALLOCATE-INSTANCE in each (i.e. that it does what
ALLOCATE-INSTANCE is advertized to do...no more and no less) will yield
perfect results.
It just so happens that RESOURCED-STANDARD-CLASS's method for
ALLOCATE-INSTANCE uses CALL-NEXT-METHOD, and rightfully so. If the user
switched the order by accident, then there would still be resourcing
done, but not at the right level (provided that the ALLOCATE-INSTANCE
method for VIEW-METACLASS also had a CALL-NEXT-METHOD. In fact, one
would probably argue that *no* implementation of ALLOCATE-INSTANCE would
lack the call to CALL-NEXT-METHOD at some point.
> In that case there is a real problem because the objects that are
> allocated are probably `alien' (not in the `foreign language' sense -
> actually maybe in that sense, maybe they are pointers to objects in
> C-space or something). So such a metaclass probably can not coexist
> with others.
Most MOP stuff won't affect allocation...only initialization. That's my
guess. If, however, the UncommonSQL person figured that his/her stuff
was so heavy on allocation that he/she did their own resourcing, then
the nice thing is that all this would still work, though two layers of
resourcing is dumb.
> 2. Really, these classes are defining wrappers, which at some point
> will or may do a CALL-NEXT-METHOD to actually get an instance. In
> *this* case, then (a) they might be better defined as around methods,
> and (b) it ought to be possible for the classes to coexist, since both
> methods will be called. There might, I agree, be order dependencies
> (and the classes should document what these are), although I think
> good design would try to minimise these. There might also be problems
> if one or other classes has tangled up initialisation with allocation,
> but that's just bad design, I think.
Yes. I think it's extremely important to make sure that when someone
implements a method on a gf, that they hold tightly to the specification
of what that GF is supposed to return. If not, then I think the door
opens for all sorts of bugs.
> I think that there may be special considerations in the case of
> ALLOCATE-INSTANCE: for instance a class might assume that calling
> ALLOCATE-INSTANCE returned an object which is not EQ to any other
> object, which your resourced-class obviously can't ensure. Whether
> that assumption is legitimate, I'm not sure.
I don't think this particular problem is a problem in the sense that:
the *only* way that an instance can be placed in the resource pool
according to the code in RESOURCED-STANDARD-CLASS is when
DEALLOCATE-INSTANCE is called on it, and that function should *only*
be called when the programmer knows for certain that the instance is
no longer referenced, and could be gc'd.
I think this brings up an interesting point about using weak pointers
for the cache (yeah..more implementation-dependency). But it's nice to
imagine that while GC can collect these objects, that they're still
re-usable. An interesting point. This would be nicer in the sense that
we don't retain long lists of unwanted objects that we never need
again...they will eventually go away, and only the ones whose
reallocation lifecycles are very short will be the ones that get reused.
dave
Tim Bradshaw <···@cley.com> writes:
> * Alain Picard wrote:
>
> > * it's better to avoid moppish stuff for resourcing, since that
> > leaves you open to having your cake and eating it too
> > [i.e. persistent, resourced objects]
>
I want to make it clean that the above was my _paraphrasing_ my
understanding of another poster [Jochen]. It is not a belief
I personally hold. This was clear from the original context,
but not from the lone extract which Tim posted.
--
It would be difficult to construe Larry Wall, in article
this as a feature. <·····················@netlabs.com>
Alain Picard <·······@optushome.com.au> writes:
> I want to make it clean that the above was my _paraphrasing_ my
> understanding of another poster [Jochen]. It is not a belief
> I personally hold. This was clear from the original context,
> but not from the lone extract which Tim posted.
My apologies. As far as I can see it was (what I understood to be)
Jochen's position I was arguing against, and not yours: you'd just
summarised my understanding of his position so concisely and well that
it was easier to sloppily quote you.
--tim
I was curious as to whether reusing instances would be any faster, got
some contrary results (but I am no Lisp expert so maybe I screwed
something up:
The results for 100,000 instances were (all code below):
Standard allocation:
; cpu time (non-gc) 20 msec user, 0 msec system
; cpu time (gc) 40 msec user, 0 msec system
; cpu time (total) 60 msec user, 0 msec system
; real time 60 msec
; space allocation:
; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes
That's the worst timing I got, just making standard, three-slot
instances in a tight loop.
Resourced way:
; cpu time (non-gc) 1,502 msec user, 0 msec system
; cpu time (gc) 30 msec user, 0 msec system
; cpu time (total) 1,532 msec user, 0 msec system
; real time 1,532 msec
; space allocation:
; 100,001 cons cells, 0 symbols, 48 other bytes, 0 static bytes
That's making resourced instances, where I have things rigged so one
actual instance gets made and then reused for all other "allocations".
The code below produced the results above. Note that I whacked an
assertion to go for a little more speed.
Not shown is code-mangling to suppress making the slots unbound (not
that that would be a Good Idea, just to see where the time was going):
; cpu time (non-gc) 1,082 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 1,082 msec user, 0 msec system
; real time 1,082 msec
; space allocation:
; 100,006 cons cells, 0 symbols, 48 other bytes, 0 static bytes
... and even defeating the whole scheme, the pop and the push:
; cpu time (non-gc) 1,122 msec user, 0 msec system
; cpu time (gc) 40 msec user, 0 msec system
; cpu time (total) 1,162 msec user, 0 msec system
; real time 1,162 msec
; space allocation:
; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes
OK that got rid of the consing, but no faster. Am I screwing things up?
I am using ACL5 on NT4 on a 1.4ghz Dell. Speed 2, debug 3, safety 1,
space 1.
kenny
clinisys
;------------------------------------------------------------------------------------------
(defparameter *resources* (make-hash-table :test #'eq))
(defclass resourced-standard-class (standard-class) ())
(defmethod allocate-instance ((class resourced-standard-class)
&rest args &key &allow-other-keys)
(declare (ignore args))
(or (pop (gethash class *resources*))
(call-next-method)))
(defmethod deallocate-instance (instance)
#+courage (assert (typep (class-of instance)
'resourced-standard-class)
nil
"Only instances whose metaclasses are of type
RESOURCED-STANDARD-CLASS can be deallocated.")
(loop
for slot-def in (clos:class-slots (class-of instance))
for slot-name = (clos:slot-definition-name slot-def)
do (slot-makunbound instance slot-name))
(pushnew instance (gethash (class-of instance) *resources*) :test
#'eq)
instance)
(defclass estd ()
((etime :initform nil :accessor etime :initarg etime)
(mpos :initform nil :accessor mpos :initarg mpos)
(kstroke :initform nil :accessor kstroke :initarg kstroke)))
(defclass eresourced ()
((etime :initform nil :accessor etime :initarg etime)
(mpos :initform nil :accessor mpos :initarg mpos)
(kstroke :initform nil :accessor kstroke :initarg kstroke))
(:metaclass resourced-standard-class))
(defun test-estd (count)
(dotimes (x count)
(make-instance 'estd)))
(defun test-eresourced (count)
(dotimes (x count)
(deallocate-instance
(make-instance 'eresourced))))
#+test
(let ((n 100000))
(time
(test-estd n))
(clrhash *resources*)
(time
(test-eresourced n)))
Tim Bradshaw wrote:
>
> Alain Picard <·······@optushome.com.au> writes:
>
> > I want to make it clean that the above was my _paraphrasing_ my
> > understanding of another poster [Jochen]. It is not a belief
> > I personally hold. This was clear from the original context,
> > but not from the lone extract which Tim posted.
>
> My apologies. As far as I can see it was (what I understood to be)
> Jochen's position I was arguing against, and not yours: you'd just
> summarised my understanding of his position so concisely and well that
> it was easier to sloppily quote you.
>
> --tim
From: Dave Bakhash
Subject: Re: puzzled Re: efficient instance reallocation in CLOS
Date:
Message-ID: <c29wv2domcr.fsf@no-knife.mit.edu>
Kenny Tilton <·······@nyc.rr.com> writes:
> I was curious as to whether reusing instances would be any faster, got
> some contrary results (but I am no Lisp expert so maybe I screwed
> something up:
I don't have time at this moment to write the results of my tests in LW,
but they were that standard and resourced allocations were mostly
similar. I didn't put much time into the test, however.
The test is strongly sensitive to how many GC's occur, I'd think, while
running the test. The whole point is that not as much garbage is being
created, so it's during the GC that you save your time when resourcing.
I'll also post test results for LW soon.
thanks for posting,
dave
Hey,
I just tried it on MS Windows, with LispWorks. I did the more pragmatic
test. I left out the assertion, and I changed the PUSHNEW to PUSH,
since the implication there is that programmers using this know what
they're doing. Of course, I left in the SLOT-MAKUNBOUND calls, since I
feel that they're necessary. I also believe that this is where the time
is going, since commenting out that part makes the two run in just about
the same exact amount of time, though the resourced version barely conses.
I also up'd the number of instances to 1,000,000...just to see the
differences more clearly. Lastly, I did a global GC before each run,
since state of Lisp is kinda important.
CL-USER 68 > (compile-file "c:/temp/resource" :load t)
;;; Compiling file c:/temp/resource ...
;;; Safety = 1, Speed = 3, Space = 0, Float = 1, Interruptible = 0
;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3, GC safety = 3
;;; Source level debugging is on
;;; Source file recording is on
;;; Cross referencing is on
; (TOP-LEVEL-FORM 1)
; (DEFPARAMETER *RESOURCES*)
; (DEFCLASS RESOURCED-STANDARD-CLASS)
; (METHOD ALLOCATE-INSTANCE (RESOURCED-STANDARD-CLASS))
; (METHOD DEALLOCATE-INSTANCE (T))
; (DEFCLASS ESTD)
; (DEFCLASS ERESOURCED)
; TEST-ESTD
; TEST-ERESOURCED
; (TOP-LEVEL-FORM 2)
; Loading fasl file c:\temp\resource.fsl
#P"c:/temp/resource.fsl"
NIL
NIL
CL-USER 69 > (mark-and-sweep 2)
17949030
CL-USER 70 > (time (TEST-ESTD 1000000))
11.0 seconds used.
Standard Allocation 40042248 bytes.
Fixlen Allocation 11000341 bytes.
NIL
CL-USER 71 > (mark-and-sweep 2)
17948294
CL-USER 72 > (time (TEST-ERESOURCED 1000000))
16.5 seconds used.
Standard Allocation 2552 bytes.
Fixlen Allocation 22000979 bytes.
NIL
----------
Again...if someone knows a better way to unbind the slots, please do
tell. That's where the time is going, since this test runs in 11.5
seconds with that line uncommented.
Also, note that in my case, most classes have no slots, and a few have
just one (so, I think for me, unbinding the slots won't be as
painful...it goes with the size of the slots. For example, changing
this code so that the classes only have 1 slot yields much closer
results: 10.3 and 13.7 in favor of *not* resourcing. It's that nasty
slot unbinding step that needs to be optimized, and I bet there's a
single way to do that without a LOOPing construct...Anyone know?
CL-USER 81 > (mark-and-sweep 2)
18133790
CL-USER 82 > (time (TEST-ESTD 1000000))
10.3 seconds used.
Standard Allocation 32032032 bytes.
Fixlen Allocation 11000440 bytes.
NIL
CL-USER 83 > (mark-and-sweep 2)
18133764
CL-USER 84 > (time (TEST-ERESOURCED 1000000))
13.7 seconds used.
Standard Allocation 2224 bytes.
Fixlen Allocation 22001034 bytes.
NIL
--dave
From: Tim Moore
Subject: Re: puzzled Re: efficient instance reallocation in CLOS
Date:
Message-ID: <9pd6cn$9tp$0@216.39.145.192>
Dunno if this is the cause of your results, but MAKE-INSTANCE is often
heavily optimized for standard-class and short-circuits the MOP if
possible. So, by using objects of non-standard metaclasses you may be
pissing away any performance gain you might get by avoiding consing.
Tim
In article <·················@nyc.rr.com>, "Kenny Tilton"
<·······@nyc.rr.com> wrote:
> I was curious as to whether reusing instances would be any faster, got
> some contrary results (but I am no Lisp expert so maybe I screwed
> something up:
> The results for 100,000 instances were (all code below): Standard
> allocation:
> ; cpu time (non-gc) 20 msec user, 0 msec system ; cpu time (gc) 40
> msec user, 0 msec system ; cpu time (total) 60 msec user, 0 msec system
> ; real time 60 msec
> ; space allocation:
> ; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes That's
> the worst timing I got, just making standard, three-slot instances in a
> tight loop.
> Resourced way:
> ; cpu time (non-gc) 1,502 msec user, 0 msec system ; cpu time (gc)
> 30 msec user, 0 msec system ; cpu time (total) 1,532 msec user, 0 msec
> system ; real time 1,532 msec
> ; space allocation:
> ; 100,001 cons cells, 0 symbols, 48 other bytes, 0 static bytes That's
> making resourced instances, where I have things rigged so one actual
> instance gets made and then reused for all other "allocations". The
> code below produced the results above. Note that I whacked an assertion
> to go for a little more speed. Not shown is code-mangling to suppress
> making the slots unbound (not that that would be a Good Idea, just to
> see where the time was going): ; cpu time (non-gc) 1,082 msec user, 0
> msec system ; cpu time (gc) 0 msec user, 0 msec system ; cpu time
> (total) 1,082 msec user, 0 msec system ; real time 1,082 msec
> ; space allocation:
> ; 100,006 cons cells, 0 symbols, 48 other bytes, 0 static bytes ...
> and even defeating the whole scheme, the pop and the push: ; cpu time
> (non-gc) 1,122 msec user, 0 msec system ; cpu time (gc) 40 msec
> user, 0 msec system ; cpu time (total) 1,162 msec user, 0 msec system ;
> real time 1,162 msec
> ; space allocation:
> ; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes OK
> that got rid of the consing, but no faster. Am I screwing things up? I
> am using ACL5 on NT4 on a 1.4ghz Dell. Speed 2, debug 3, safety 1, space
> 1.
> kenny
> clinisys
> ;------------------------------------------------------------------------------------------
> (defparameter *resources* (make-hash-table :test #'eq)) (defclass
> resourced-standard-class (standard-class) ()) (defmethod
> allocate-instance ((class resourced-standard-class)
> &rest args &key &allow-other-keys)
> (declare (ignore args))
> (or (pop (gethash class *resources*))
> (call-next-method)))
> (defmethod deallocate-instance (instance)
> #+courage (assert (typep (class-of instance)
> 'resourced-standard-class)
> nil
> "Only instances whose metaclasses are of type
> RESOURCED-STANDARD-CLASS can be deallocated.")
> (loop
> for slot-def in (clos:class-slots (class-of instance)) for slot-name
> = (clos:slot-definition-name slot-def) do (slot-makunbound instance
> slot-name))
> (pushnew instance (gethash (class-of instance) *resources*) :test
> #'eq)
> instance)
> (defclass estd ()
> ((etime :initform nil :accessor etime :initarg etime)
> (mpos :initform nil :accessor mpos :initarg mpos) (kstroke :initform
> nil :accessor kstroke :initarg kstroke)))
> (defclass eresourced ()
> ((etime :initform nil :accessor etime :initarg etime)
> (mpos :initform nil :accessor mpos :initarg mpos) (kstroke :initform
> nil :accessor kstroke :initarg kstroke))
> (:metaclass resourced-standard-class))
> (defun test-estd (count)
> (dotimes (x count)
> (make-instance 'estd)))
>
> (defun test-eresourced (count)
> (dotimes (x count)
> (deallocate-instance
> (make-instance 'eresourced))))
> #+test
> (let ((n 100000))
> (time
> (test-estd n))
> (clrhash *resources*)
> (time
> (test-eresourced n)))
> Tim Bradshaw wrote:
>> Alain Picard <·······@optushome.com.au> writes:
>> > I want to make it clean that the above was my _paraphrasing_ my
>> > understanding of another poster [Jochen]. It is not a belief I
>> > personally hold. This was clear from the original context, but not
>> > from the lone extract which Tim posted.
>> My apologies. As far as I can see it was (what I understood to be)
>> Jochen's position I was arguing against, and not yours: you'd just
>> summarised my understanding of his position so concisely and well that
>> it was easier to sloppily quote you.
>> --tim
From: Dave Bakhash
Subject: Re: puzzled Re: efficient instance reallocation in CLOS
Date:
Message-ID: <c29ofnpojw0.fsf@no-knife.mit.edu>
"Tim Moore" <·····@bricoworks.com> writes:
> Dunno if this is the cause of your results, but MAKE-INSTANCE is often
> heavily optimized for standard-class and short-circuits the MOP if
> possible. So, by using objects of non-standard metaclasses you may be
> pissing away any performance gain you might get by avoiding consing.
yeah...but this raises another point, which are the kinds of
optimizations done by delivery. For example, since I defined the
DEALLOCATE-INSTANCE function to be generic, but there was only a single
method, what was the price there? What if I used DEFUN? Note that the
delivery process often notices such things and delivers as if DEFUN were
used. I don't suspect that this particular optimization will be
significant, but in general, there are things that occur during deliery
that might favor MOP-based customizations.
One moral I've learned from this exercise is that it's a bit hard to
out-perform standard MOP processes using the MOP. It seems that the MOP
is there for adding functionality but not as much for speedwise
performance.
dave
From: Pierre R. Mai
Subject: Re: puzzled Re: efficient instance reallocation in CLOS
Date:
Message-ID: <87669xis4m.fsf@orion.bln.pmsf.de>
Kenny Tilton <·······@nyc.rr.com> writes:
> I was curious as to whether reusing instances would be any faster, got
> some contrary results (but I am no Lisp expert so maybe I screwed
> something up:
>
> The results for 100,000 instances were (all code below):
>
> Standard allocation:
>
> ; cpu time (non-gc) 20 msec user, 0 msec system
> ; cpu time (gc) 40 msec user, 0 msec system
> ; cpu time (total) 60 msec user, 0 msec system
> ; real time 60 msec
> ; space allocation:
> ; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes
[...]
> ... and even defeating the whole scheme, the pop and the push:
>
> ; cpu time (non-gc) 1,122 msec user, 0 msec system
> ; cpu time (gc) 40 msec user, 0 msec system
> ; cpu time (total) 1,162 msec user, 0 msec system
> ; real time 1,162 msec
> ; space allocation:
> ; 1 cons cell, 0 symbols, 4,800,000 other bytes, 0 static bytes
>
> OK that got rid of the consing, but no faster. Am I screwing things up?
> I am using ACL5 on NT4 on a 1.4ghz Dell. Speed 2, debug 3, safety 1,
> space 1.
This is not that curious, given that most serious implementations will
be able to employ many low-level optimizations and short-cuts to
instance allocation, IFF only implementation-controlled methods are
applicable on the relevant generic functions. Once you introduce your
own methods on these GFs, all bets are off, and the implementation
must follow the MOP-mandated protocol to the letter, in order to let
your methods work as expected. Hence the bad performance even for the
null case (here for CMU CL, but similar results will apply for most
implementations):
(defclass new-standard-class (pcl::standard-class) ())
(defmethod mop:validate-superclass
((class new-standard-class) (super pcl::standard-class))
t)
(defmethod allocate-instance :around ((class new-standard-class)
&key &allow-other-keys)
(call-next-method))
(defclass estd ()
((etime :initform nil :accessor etime :initarg etime)
(mpos :initform nil :accessor mpos :initarg mpos)
(kstroke :initform nil :accessor kstroke :initarg kstroke)))
(defclass enew ()
((etime :initform nil :accessor etime :initarg etime)
(mpos :initform nil :accessor mpos :initarg mpos)
(kstroke :initform nil :accessor kstroke :initarg kstroke))
(:metaclass new-standard-class))
(defun test-estd (count)
(dotimes (x count)
(make-instance 'estd)))
;;; The following uses low-level constructs that allocate the raw
;;; instance with all slots unbound (though initialization according
;;; to initforms could be done with little additional overhead),
;;; bypassing all the normal initialization processing overhead.
(defun test-estd-alloc (count)
(let ((wrapper (pcl::class-wrapper (pcl::find-class 'estd))))
(dotimes (x count)
(pcl::allocate-standard-instance wrapper))))
(defun test-enew (count)
(dotimes (x count)
(make-instance 'enew)))
On an AMD K6-2/550 using a current CMU CL binary, this yields:
Form Real-Time GC-Time Bytes Consed
--------------------------------------------------------------------
Using standard 2MB nursery generation:
(test-null 10000000) 0.04s 0.00s 0
(test-estd 10000000) 36.54s 6.43s 400037304
(test-enew 10000000) 138.26s 6.36s 400035368
(test-estd-alloc 10000000) 19.09s 6.33s 400037344
Using a 10MB nursery generation:
(test-estd 10000000) 29.58s 1.36s 400001120
(test-estd-alloc 10000000) 14.24s 1.33s 400004960
Since the major overhead in instance creation lies in instance
initialization (even test-estd-alloc-only does some initialization)
and not in the low-level instance allocation, any resourcing scheme
that modifies standard GFs in such a way that it inhibits
implementation optimizations will have a hard time making up
for the speed-loss this incurs.
We have seen that each loop iteration takes around 365ns/201 cycles at
most, and that the raw instance-allocation and initialization of slots
to the unbound value takes plus related deallocation/GC costs can be
had for as low as between 142-190ns or 78-104 cycles. That suggests
that even a magic resourcing scheme can save us at most 104 of 201
cycles, and that is far less than the additional 550 cycles that
non-standard initialization costs us.
And even if we ignore the non-standard overhead, it seems unlikely
that a resourcing scheme will be able to improve much on the 78-104
cycles needed to prepare a new raw instance. Let's take a look at the
implementation of PCL::ALLOCATE-STANDARD-INSTANCE:
(defun allocate-standard-instance (wrapper &optional (slots-init nil slots-init-p))
(let ((instance (%%allocate-instance--class))
(no-of-slots (wrapper-no-of-instance-slots wrapper)))
(setf (std-instance-wrapper instance) wrapper)
(setf (std-instance-slots instance)
(cond (slots-init-p
;; Inline the slots vector allocation and initialisation.
(let ((slots (make-array no-of-slots :initial-element 0)))
(do ((rem-slots slots-init (rest rem-slots))
(i 0 (1+ i)))
((>= i no-of-slots)) ;endp rem-slots))
(declare (list rem-slots)
(type kernel:index i))
(setf (aref slots i) (first rem-slots)))
slots))
(t
(make-array no-of-slots
:initial-element pcl::*slot-unbound*))))
instance))
In our case we take the "fast-path", i.e the code boils down to:
(defun allocate-standard-instance (wrapper)
(let ((instance (%%allocate-instance--class))
(no-of-slots (wrapper-no-of-instance-slots wrapper)))
(setf (std-instance-wrapper instance) wrapper)
(setf (std-instance-slots instance)
(make-array no-of-slots :initial-element pcl::*slot-unbound*))
instance))
%%allocate-instance--class is a very fast and can be gotten for as
little as 28 cycles (including GC costs). And at some point the
resourcing framework will want to reinitialize all slots to make them
unbound, so only the allocation cost of the make-array call, the
instance allocation and initialization itself can realistically be
optimized away, and replaced by the code needed to find a suitable
pre-made instance.
We have further seen that even with very slight tuning the amount of
time GC costs us is really negligible, ranging from 35 cycles per
iteration (or 17.6% of total run-time) for the untuned GC to 7
cycles per iteration (4.6% of total run-time) for the slightly tuned
GC. And that is using CMU CL's conservative generational GC, which
isn't nearly as intelligent or as well tuned and polished as
commercial implementations. Ditto for the CLOS implementation (though
it is often fairly competitive with commercial implementations).
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
"Pierre R. Mai" <····@acm.org> writes:
> And even if we ignore the non-standard overhead, it seems unlikely
> that a resourcing scheme will be able to improve much on the 78-104
> cycles needed to prepare a new raw instance.
My actual thought was that the savings wouldn't come from the creation
of the instances, but rather on the collection of the more abundant
garbage that's created. It turned out (on LispWorks) that the GC didn't
factor in as much as some of the other overhead. I'm sure that
depending on the Lisp implementation it could potentially be the other
way around...but I'm certainly satisfied with the exercise, and it was
worthwhile for me.
I very much appreicate all the constructive posts. thanks.
dave
"Pierre R. Mai" wrote:
> Once you introduce your
> own methods on these GFs, all bets are off, and the implementation
> must follow the MOP-mandated protocol to the letter,
gotcha. nice post, btw.
i once got a feel for how much goes on behind the scenes when i got bit
by a nasty (presumed) bug in ACL: (going on faint recall) a
specialization on slot-value-using-class (SVUC) was ignored by ACL
unless I had a similarly specialized (setf SVUC) defined! I might have
that backwards. But I think the sequence was:
1. (setf SVUC) got called (on the arg combo matching my setter) to
install an initform value
2. i had no such setter defined some optimization memoized the arglist
(not memoizing in that it was specifically a 'setf) to be handled by a
standard function
3. when the time came to /read/ via SVUC the memoization led ACL to
bypass my reader.
that took a long time to diagnose, esp. since I was MOP newbie at the
time. but once I saw that defining a reader suddenly made the setter
"visible" I presumed that some faulty optimization was, well, at fault.
btw, it took me about ten iterations to convince Franz of the bug, then
silence. Weary of the exercise I gave up and just left a dummy setter in
place as a head fake to the presumed buggy optimization and got on with
my project, so that bad boy might still be out there.
In fairness to Franz this is the only thing they ever whiffed on and I
am sure if I had made a stink they would have come around, but I spent
hours crafting a reproducible and making the bug more and more obvious,
finally decided I had to cut my losses and move on.
Until now. :)
kenny
clinisys
OK, I did have it backwards (and I see in my post I also got it
backwards-backwards toward sthe end, anyway...found this in my code:
; -----------------------
; following must it exist, it seems, for the preceding to be "seen" by
CLOS dispatch
;
(defmethod slot-value-using-class :around ((mClass ModelClass)
(self ModelObject)
esd)
(declare (ignorable esd))
(call-next-method))
I don't show "the preceding" because I am not even sure the current
preceding is the one that was preceding when I wrote the comment! (OT:
That's part of why I do not like comments.)
And there is probably more to it than that, other setters esp.,
definitely other readers. In fact that is how I figured it out, I
started defining all sorts of other methods trying to figure out what
was going on. Suddenly the invisible method was getting kicked off!
Puzzled, I figured I was up too late, happily took out the unwanted
methods and (perhaps also after exiting ACL and coming back in?) it
/stopped/ getting kicked off. Finally put two and two together.
I also am not sure now that the setter even had the same
specializations. "self" may have been unspecialized in the setter.
Anyway, the reproducible I sent Franz made an ironclad case.
kenny
clinisys
"Pierre R. Mai" <····@acm.org> writes:
> This is not that curious, given that most serious implementations will
> be able to employ many low-level optimizations and short-cuts to
> instance allocation, IFF only implementation-controlled methods are
> applicable on the relevant generic functions. Once you introduce your
> own methods on these GFs, all bets are off, and the implementation
> must follow the MOP-mandated protocol to the letter, in order to let
> your methods work as expected. Hence the bad performance even for the
> null case (here for CMU CL, but similar results will apply for most
> implementations):
I was thinking about this, and it dawned on me...
It seems that something like the aforementioned MOP-based allocation
scheme could probably be more helpful if the following were true:
1) the system is already using non-standard allocation (i.e. defining
their own metaclass(es), defined :around methods on
ALLOCATE-INSTANCE, etc.
2) the new allocation overhead in (1) is expensive...esp. in such a way
so as to make it inefficient to allocate instances that can
otherwise be cached in the scheme we've been discussing in this
thread.
If (1) were true, then the optimizations that Pierre mentioned would
likely not apply anyway. And in that case, the RESOURCED-STANDARD-CLASS
would be like a mixin...there only to possibly add the resourcing aspect
to the code.
In more thinking about all this, I (for some reason) keep thinking of
the CLOS reallocation as if you had these hashtables, and when you put
them back in the resource pool, you just CLRHASH them. This, of course,
is the extreme case, since in my case specfically, I usually had zero
slots (or sometimes one, seldom 2, and never more than two). But one
interesting thing I wonder is how CLRHASH does relative to the loop
through the slot defs of the class and then unbinding those instances
slot-by-slot...I bet clrhash doesn't work that way in most
implementations.
dave
David Bakhash wrote:
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> > I'm not sure I see this to be the case. The downside to the
>> > DEF-VIEW-CLASS system is that it deviates from using DEFCLASS, and
>> > so you lose control of defining the metaclass of the classes you
>> > define. But I'd guess that you can still make the metaclass an
>> > option which defaults to STANDARD-DB-CLASS or whatever it is, but
>> > why you couldn't use a metaclass that inherited both from
>> > STANDARD-DB-CLASS and RESOURCED-STANDARD-CLASS or whatever I
>> > called it...I don't know.
>>
>> As I said... it get's ugly. IMHO it is much more difficult to read
>> the sources and prove that RESOURCE-STANDARD-CLASS does not
>> interfere with e.g. VIEW-METACLASS.
First let me say that I in _no_ way wanted to insult or offense you - I
thought that it is ok to express an opinion why _I_ would think twice
using your approach - this does _not_ mean that I say your approach
is bad or you are wrong.
Furthermore: I'm not the author of UncommonSQL.
> Obviously, you have a bias here. It is a silly bias. You think:
>
> 1) very many people care about the whole DEF-VIEW-CLASS thing.
> 2) the possibility that DEF-VIEW-CLASS _might_ be used supersedes the
> right for *anyone* to use the MOP, and to define new metaclasses.
> (this is, of course, absurd).
No you got me wrong - my point was simply that you would not necessarily
need the MOP for resourcing (particularily since there actually is a
facility that does not use the MOP...). The same could be said to using
persistent objects yes. I'm certainly not able and do not even want to cut
the right of anyone to use something - I simply make use of _my_ right to
express my opinion.
> The way I see it, you've worked on something called UncommonSQL, and
> feel that it is a superior package, and because you've used the same
> tools I'm suggesting, and in such a way so as to possibly preclude
> others from using them, that *others* should find alternative
> approaches. Nonsense. I'm starting to think you really are a
> dictator.
No I don't think that it is a "superior package" since it is something
_completely_ different. As I already said I only express my opinion that
_I_ would not use the MOP if it does not give me something that would have
a relevant gain - IMHO (!) It does not give me much gain over using the
DEFRESOURCE facility but I would loose some other things (that you might
not need). YMMV but I still think that I've the right to have and express
my own opinion without being called a dictator.
>> IMHO the approach looses it's elegance if you have to inherit new
>> classes in each of this cases. Btw: DEF-VIEW-CLASS is only
>> syntactic sugar - there are even examples that use DEFCLASS.
>
> The purpose of ALLOCATE-INSTANCE, and the reason it was made to be a
> generic function was exactly so that if you wanted to change the way
> allocation takes place, that's the function you want to be using.
> Defining all sorts of new macros is *not* the best way, and how you
> can possibly try to argue that using ALLOCATE-INSTANCE is less elegant
> is beyond me.
As I already said I somewhat doubt that this approach will get you much
time - it could get even worse than the standard allocation...
So it is not really a question of "being elegant" but more "being useful".
>> DEF-VIEW-CLASS is only syntactic sugar - IMHO it is ok to use
>> DEFCLASS in the case you would create your own inherited version of
>> VIEW-METACLASS (it is not called STANDARD-DB-CLASS as you call
>> it). You could break the protocol in a way that makes DEF-VIEW-CLASS
>> not work anymore - therefore it is IMHO ok to let this option to
>> people who _know_ what they are doing (and therefore should have no
>> problems using DEFCLASS instead).
>
> Again...there's no reason why anyone should define a DEFCLASS-like
> macro that prevents people who know what they're doing from using a
> metaclass other than the default one.
Ask Xanalys or the authors of UncommonSQL why they thought a macro like
DEF-VIEW-CLASS would be a good idea - Fact is that you do not _need_ it to
use UncommonSQL - it's simply some kind of syntactic sugar.
>> Yes - that's ok - but the use of DEFRESOURCE is IMHO not very code
>> intensive but you immediately can manage CLOS objects, buffers
>> (arrays), open sockets and every resourced construct you can imagine
>> with the same facility.
>
> This is true, though I said since the first or second post that I was
> only interested in CLOS objects, and they happen to be ones which I'm
> defining. I do realize that DEFRESOURCE is very generic, and very
> nice, and I do like it a lot. It solves a different problem, in a
> sense, and a more general problem too.
Not necessarily a different one but certainly a more general one.
>
>> >> If you don't want to hear my opinion - run! ;-)
>> >
>> > I think what you might then be saying is "If you don't _agree_
>> > with my opinions, then run", and of course, this is either some
>> > kind of joke (not terribly funny humor, with or without the
>> > smiley), or it's some kind of "I'm dictator here...agree or leave"
>> > thing...again, silly.
>>
>> Yes it was a joke. No it has nothing to do with being dictator or
>> having you to agree or leave - it is an old quote with a similar
>> meaning like "IMHO" or expressing that I realized that I'm
>> opinionated in this discussion. If it was offending for you - sorry
>> - this was not intented...
>
> No problem. But I get this aweful sense of militant superiority from
> you. Like arguing against using metaclasses for pooling because you
> have used metaclasses for persistence...I just see this as a
> superiority issue you'll probably have to deal with. No one's running
> because you say run. An no one's gonna be scared to ever use
> metaclasses because it might pose some problems if they try later to
> interface to UncommonSQL or any other such package, according to your
> dictates.
Sorry if I may have insulted you - to make it a bit more clear: I only
expressed my opinion and criticized your approach on what I think/thought of
it at the first glance. I cannot really follow your rationale of me being a
dictator because I do not share all of your opinions...
Regards,
Jochen Schmidt
Jochen Schmidt <···@dataheaven.de> writes:
> > Again...there's no reason why anyone should define a DEFCLASS-like
> > macro that prevents people who know what they're doing from using
> > a metaclass other than the default one.
>
> Ask Xanalys or the authors of UncommonSQL why they thought a macro like
> DEF-VIEW-CLASS would be a good idea - Fact is that you do not _need_ it to
> use UncommonSQL - it's simply some kind of syntactic sugar.
USQL has it because Xanalys had it and we were attempting to achieve
source compatability with the SQL package from Xanalys.
--
Craig Brozefsky <·····@red-bean.com>
http://www.red-bean.com/~craig
The outer space which me wears it has sexual intercourse. - opus
Jochen Schmidt <···@dataheaven.de> writes:
> David Bakhash wrote:
>
> > Jochen Schmidt <···@dataheaven.de> writes:
> >
> > Obviously, you have a bias here. It is a silly bias. You think:
> >
> > 1) very many people care about the whole DEF-VIEW-CLASS thing.
> > 2) the possibility that DEF-VIEW-CLASS _might_ be used supersedes the
> > right for *anyone* to use the MOP, and to define new metaclasses.
> > (this is, of course, absurd).
>
> No you got me wrong - my point was simply that you would not
> necessarily need the MOP for resourcing (particularily since there
> actually is a facility that does not use the MOP...).
That was certainly _not_ your only point, but it seems that looking back
at the whole thread, you'd like to believe that that was your only
point. Jeez...are you so afraid to make mistakes here?
> I'm certainly not able and do not even want to cut the right of anyone
> to use something - I simply make use of _my_ right to express my
> opinion.
Of course, no one can cut such rights. I wasn't arguing against Jochen
Law...I was arguing against your opinion not to use a MOP-based program
since there was this other program that used the MOP out there.
> > The way I see it, you've worked on something called UncommonSQL, and
> > feel that it is a superior package
>
> No I don't think that it is a "superior package" since it is something
> _completely_ different.
I was using the word `superior' in the general sense...not superior to a
resourcing tool that solves a different problem altogether.
> As I already said I only express my opinion that _I_ would not use the
> MOP if it does not give me something that would have a relevant gain
That is a fine opinion, though that is not really what you said...it's
what you're saying now, and that's all that matters. I don't see why
you are trying so hard to obfuscate what you said.
> > The purpose of ALLOCATE-INSTANCE, and the reason it was made to be a
> > generic function was exactly so that if you wanted to change the way
> > allocation takes place, that's the function you want to be using.
> > Defining all sorts of new macros is *not* the best way, and how you
> > can possibly try to argue that using ALLOCATE-INSTANCE is less elegant
> > is beyond me.
>
> As I already said I somewhat doubt that this approach will get you
> much time - it could get even worse than the standard allocation...
> So it is not really a question of "being elegant" but more "being
> useful".
As you said already? Really? Where did you say that?
Really? I'd *love* to see that post! Guess what? It's not there. Not
even on Google. In fact, Jochen, you *never* said that in this whole
thread! Not even after the results came in!
But, I *will* tell you what you said:
Jochen wrote:
The bad thing is that you _have_ to use metaclasses then - You will
have to change existing classes to use this metaclass and it gets ugly
if your code already uses metaclasses. What if you would want to
manage UncommonSQL view-class instances? Yes it is a nice hack - but
If I had to choose between having a nice DB-binding or some syntactic
sugar for resource-management I would opt for the first. Another nice
thing with DEFRESOURCE is that it scales with your design and is more
consistent. If you later need to manage resources other than CLOS
instances or you want to use other metaclasses, then you would have to
redo all resource-managment.
You don't like dictator? Well, how about liar?
To actually drill this point in more...the closest thing you _did_ say
was:
Jochen wrote:
>> [Jochen]
>> What if you would want to manage UncommonSQL view-class instances?
>> Yes it is a nice hack - but If I had to choose between having a nice
>> DB-binding or some syntactic sugar for resource-management I would
>> opt for the first.
>
> [Dave]
> This was some argument against using the MOP for something other than
> UncommonSQL.
No it was not. I _explicitely_ mentioned _me_ . If *I* _had_ to choose!
Since using multiple inheritance can buy us both if both packages are
well written this might be a non-issue but I still think that it is a
valid argument _against_ the MOP approach that the MOP is not really
needed as the gain is somewhat minor (IMHO !!!).
^^^ ^^^^
Oh, really? What gain is that? is it...um, a performance gain? Well,
maybe you meant stylistic, and maybe you meant performance...but I doubt
you meant stylistic, because you said over and over that you wouldn't
want to use the MOP if it might interfere with other MOP programs. So,
not only *didn't* you say what you claim, but you may even have
contradicted it. I guess in light of the results, again, you're finding
that you'd *liked* to have said something you didn't, and in Jochen's
world, that means you did say it. Pretty funny!
> I cannot really follow your rationale of me being a dictator because I
> do not share all of your opinions...
Really? I called you a dictator *not* because you didn't share my
opinions (note: I've had many disagreements with other people on this
newsgroup, and have never called anyone that). But again, nice try...I
think you might want to assume that people who read these posts have
access to the full threads, and so your tactics will have to change.
dave
Dave Bakhash wrote:
> That was certainly _not_ your only point, but it seems that looking back
> at the whole thread, you'd like to believe that that was your only
> point. Jeez...are you so afraid to make mistakes here?
You already proved that you are neither willing nor able to understand my
writing. I've now tried *multiple* times to bring the discussion back to be
less personal by apologizing for things *you* did not understand right.
I now no longer excuse your stupidity.
It was very nice from you that you send the prove that you did not read my
posts with your last post here (see below).
>> I'm certainly not able and do not even want to cut the right of anyone
>> to use something - I simply make use of _my_ right to express my
>> opinion.
>
> Of course, no one can cut such rights. I wasn't arguing against Jochen
> Law...I was arguing against your opinion not to use a MOP-based program
> since there was this other program that used the MOP out there.
Nonsense. My point was exactly the one Alain outlined - so others was able
to decipher my writings - you can criticize me for not being able to spell
out understandable enough what I meant but you cannot tell lies about me
the way you do now.
>> As I already said I somewhat doubt that this approach will get you
>> much time - it could get even worse than the standard allocation...
>> So it is not really a question of "being elegant" but more "being
>> useful".
>
> As you said already? Really? Where did you say that?
>
> Really? I'd *love* to see that post! Guess what? It's not there. Not
> even on Google. In fact, Jochen, you *never* said that in this whole
> thread! Not even after the results came in!
This paragraph show perfectly well how good you have read the posts of this
thread I *DID* say that in the post
MessageID: <············@rznews2.rrze.uni-erlangen.de>
Date: 29. Sep. 2001
Google:
http://groups.google.com/groups?q=g:thl1325241013d&hl=de&rnum=25&selm=9p4bor%2437e%241%40rznews2.rrze.uni-erlangen.de
I wrote:
"But first you should try if pooling the objects will get you _really_ some
speed gain - since the only thing you really would gain is allocating of
memory for the instances - and that could be _very_ fast compared to
initializing the slots."
So the only thing you proved with your dumb offense is that you are an
asshole and liar that is unable to read. I call you an asshole and a liar
now since you begun to talk on that level in your last post and it
definitely describes best how you behave here.
I think you're unable to behave and read so I don't think further
discussing would lead to anythig useful - only personal attacks and insults
from your side.
Jochen Schmidt
Jochen Schmidt <···@dataheaven.de> writes:
> "But first you should try if pooling the objects will get you _really_
> some speed gain - since the only thing you really would gain is
> allocating of memory for the instances - and that could be _very_ fast
> compared to initializing the slots."
This is true, indeed, and I must apologize.
See? I accept the mistake. See how that works? Easy.
> So the only thing you proved with your dumb offense is that you are an
> asshole and liar that is unable to read.
Remember. That's why I asked where you said this, since I usually
remember who said what, and mostly recalled your argument about using
DEFRESOURCE since it was more general, and also remembered that you
thought the gain, if there was one, would be minor.
What this proves is that you've identified a mistake I made, and I'm
happy to take it back, and I'm happy to. It's most important to be
correct. You've corrected me, and that's all I care about. If course,
*anyone* would have guessed that this new allocation method could
possibly and would likely be less efficient, and that in fact was the
whole purpose of the exercise.
dave
Tim Moore <·····@herschel.bricoworks.com> writes:
> I think you want to take a look at REINITIALIZE-INSTANCE and
> SHARED-INITIALIZE in the HyperSpec and MOP document. Do you really mean
> the system ALLOCATE-INSTANCE, or your own resource management function?
> I'd stay away from the former.
I didn't look at this issue closely, nor do I plan to. However, whenever I
see this long list of gfs come up in a single discussion, I feel compelled
to mention that during the original design of CLOS, I did look at this and
that it seemed to me at the time that the absence of a REALLOCATE-INSTANCE
was going to create a lurking problem. No one thought my comment important
and my sense was they had worked too long on their design to really want to
go back and RETHINK-DESIGN having so recently done a THINK-DESIGN on the
same instance, and I just had too many irons in the fire to pursue it.
Nevertheless, I set a bit in my mind to remember taht there MIGHT be a
lingering design error that makes certain things harder than they should be.
I'll leave you guys who are reading this thread more closely to figure out
if this is the place where that might factoid might be useful.
Kent M Pitman <······@world.std.com> writes:
> to mention that during the original design of CLOS, I did look at this
> and that it seemed to me at the time that the absence of a
> REALLOCATE-INSTANCE was going to create a lurking problem. No one
> thought my comment important
I'm not sure I see it. If it existed, then when would it be called?
Would this be something on the class (i.e. EVENT, in my case), or on the
metaclass, (e.g. STANDARD-CLAS or something like CACHED-STANDARD-CLASS)?
When would it be called instead of its cousin, ALLOCATE-INSTANCE?
Before knowing if something like REALLOCATE-INSTANCE could be useful,
you have to specify, at least in general, what it does and when it does
it.
I tend to think that when you have a system where the fundamental
bahavior of objects (i.e. how they get created, initialized, and changed
from one class to another) changes, the MOP is there to help programmers
redefine their intended behavior. The names of the methods were mostly
well chosen in the CLOS/MOP, and as it turns out, implementing the
behavior I want will most likely affect MAKE-INSTANCE and/or
ALLOCATE-INSTANCE, and those are indeed the obvious names that one would
guess could help address the issue of resourcing in a non-standard, but
elegant and transparent way, calling the same functions we know and
love, and having Lisp handle things correctly underneath.
dave
In article <···············@no-knife.mit.edu>,
Dave Bakhash <·····@alum.mit.edu> wrote:
>I havn't posted here for a long, long time, and also havn't had the
>opportunity to keep up with all the threads I've been missing for months
>now. I hope that the following isn't re-hashing recent postings.
LOL! This newsgroup is hardly ever used. It can go for weeks or months
without a message. I can't recall it ever being very active. Most people
post in comp.lang.lisp; I'm not sure there's really much justification for
a newsgroup specific to CLOS.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
I think that what you want can be done with ALLOCATE-INSTANCE, but note
that ALLOCATE-INSTANCE is defined on standard-class. You would have to
define a new meta-class, say pool-allocated-class, define methods
allocate/deallocate-instance for this. and add a slot for the pool on
the class object, or a hashtable for pools keyed by class in a global or
attached to the metaclass object. Then, make your event class an
instance of your new metaclass, and go from there.
Jeff
"Dave Bakhash" <·····@alum.mit.edu> wrote in message
····················@no-knife.mit.edu...
> Hi,
>
> I havn't posted here for a long, long time, and also havn't had the
> opportunity to keep up with all the threads I've been missing for
months
> now. I hope that the following isn't re-hashing recent postings.
>
> I have a Lisp program which now reads events of a socket stream. Each
> event comes in, effectively, as (make-instance 'some-kind-of-event
...)
> form. I have a simple event loop in the program, with the main GF
being
> HANDLE-EVENT.
>
> The issue is that once events are handled, they're done with. No one
> cares about them, and they're garbage. So, since this program does
this
> 24/7, and events are streamed in continuously, and relatively fast, I
> figured I'd reuse events.
>
> Since my events are CLOS objects, I figured I could do something like
> this:
>
> (defmethod handle-event :around ((event event))
> (unwind-protect (call-next-method)
> ;; protected:
> (deallocate-instance event)))
>
> where DEALLOCATE-INSTANCE is also a GF which, for events, stores the
> events in an event cache, which is a hashtable keyed by the specific
> class of the event, and the values being, say, a list of events of
that
> class.
>
> In this case, one has to update ALLOCATE-INSTANCE for each subclass of
> the common ancestor class EVENT (not a problem -- just define new
events
> types with a DEFINE-EVENT-TYPE macro that takes care of this).
>
> What I want to know is what is necessary for me to put in the
> ALLOCATE-INSTANCE (or maybe in DEALLOCATE-INSTANCE or in both) to
> somehow make sure that when ALLOCATE-INSTANCE gets one of the cached
> events, that it's just like a freshly allocated instance that would
have
> been returned by ALLOCATE-INSTANCE had I not bothered. I'd much
rather
> put this `cleanup' stuff in ALLOCATE-INSTANCE than DEALLOCATE-INSTANCE
> for design reasons.
>
> I'd also like to have know what others think about this general
> strategy. I'm sure there may be other ways (e.g. by possibly using
> CHANGE-CLASS ??), and I'd like to know about them before I write this
> up.
>
> thanks,
> dave
>
>
Dave Bakhash <·····@alum.mit.edu> writes:
> The issue is that once events are handled, they're done with. No one
> cares about them, and they're garbage. So, since this program does this
> 24/7, and events are streamed in continuously, and relatively fast, I
> figured I'd reuse events.
>
Well, one thing I wonder about is if just allocating new events is
that bad. It seems to me that reusing an event will involve a fair
amount of slot-initialising and so on, and this might be about as
expensive as initialising one in the first place. So then the costs
of not reusing are that the GC will run more often, and you will walk
through more memory. Assuming your events are small and you can
afford lots of memory and to tune the GC, you can push the
GC-running-more-often cost down into the noise for many GCs (I did
some experiments along these lines with Allegro and Liquid, and if you
set the first generation (or nursery or whatever notion the GC has for
transient objects) size to more than a hundred Mb or so (and
*assuming* you have lots of physical memory!) the GC overhead starts
to become really small, even for high allocation rates, so long as
they are transient (you may also want to do tricks with `areas' or
whatever, so that the transient objects get allocated into a space
which is used only for them so there aren't inconvenient long-lived
objects in there). The other cost that I can see is that you walk
over more memory, so the cache behaviour of the program may be much
worse. Against that is the cost of reusing, where you have to do some
kind of find-next-free-object-and-reinitialize thing, which may be
slower than the increment-a-pointer-and-initialize that not reusing
does.
Against all that is the fact that various serious Lisp systems *do*
use pools of objects for this kind of purpose, and presumably they
knew what they were doing. (But, of course they were often written in
a world where it really wasn't reasonable to make generation sizes
large enough that GC overhead became very small.)
--tim
Tim Bradshaw <···@tfeb.org> writes:
> Well, one thing I wonder about is if just allocating new events is
> that bad.
I wondered the same thing. I have been infected with the idea that
the cost of consing something that is not going to be around for very
long is really cheap in Lisps with generational/ephemeral GC, so cheap
that explicit resource pooling is usually not a win (maybe I got this
idea from On Lisp, I don't remember).
I'm sure that initialization and reinitialization costs will have to
be taken into account, but the idea that just allocating new objects
all the time is OK is plausible enough that I would do some profiling
before changing anything. (Also, I want to believe it because I am
sooo lazy.)
John Wiseman
·······@inetmi.com writes:
> Tim Bradshaw <···@tfeb.org> writes:
>
>
> > Well, one thing I wonder about is if just allocating new events is
> > that bad.
>
> I wondered the same thing. I have been infected with the idea that
> the cost of consing something that is not going to be around for very
> long is really cheap in Lisps with generational/ephemeral GC, so cheap
> that explicit resource pooling is usually not a win (maybe I got this
> idea from On Lisp, I don't remember).
The cost of consing is essentially proportional to the amount of
retained storage (this is a gross oversimplification, but a useful
one). For small things, the GC cost is down in the noise.
(defun foo (n)
(dotimes (i n)
(declare (type fixnum i))
(cons 'a 'b)))
(time (foo 100000000))
; cpu time (non-gc) 4,796 msec user, 0 msec system
; cpu time (gc) 298 msec user, 0 msec system
; cpu time (total) 5,094 msec user, 0 msec system
; real time 5,094 msec
; space allocation:
; 100,000,055 cons cells, 0 symbols, 192 other bytes, 0 static bytes
If you are allocating larger structures, it can be noticable.
(defun foo (n)
(dotimes (i n)
(declare (type fixnum i))
(make-string 1000)))
; cpu time (non-gc) 3,498 msec user, 0 msec system
; cpu time (gc) 1,548 msec user, 0 msec system
; cpu time (total) 5,046 msec user, 0 msec system
; real time 5,047 msec
; space allocation:
; 49 cons cells, 0 symbols, 1,570,065,600 other bytes, 0 static bytes
But, if you have any sort of initialization, that will probably overwhelm
the GC costs.
(defun foo (n)
(dotimes (i n)
(declare (type fixnum i))
(copy-readtable)))
(time (foo 100000))
; cpu time (non-gc) 14,765 msec user, 0 msec system
; cpu time (gc) 218 msec user, 0 msec system
; cpu time (total) 14,983 msec user, 0 msec system
; real time 14,984 msec
; space allocation:
; 200,172 cons cells, 0 symbols, 314,400,384 other bytes, 0 static bytes
> I'm sure that initialization and reinitialization costs will have to
> be taken into account, but the idea that just allocating new objects
> all the time is OK is plausible enough that I would do some profiling
> before changing anything. (Also, I want to believe it because I am
> sooo lazy.)
The cost of debugging accidental object re-use can be high as well.
·······@inetmi.com writes:
> I wondered the same thing. I have been infected with the idea that
> the cost of consing something that is not going to be around for very
> long is really cheap in Lisps with generational/ephemeral GC, so cheap
> that explicit resource pooling is usually not a win (maybe I got this
> idea from On Lisp, I don't remember).
>
> I'm sure that initialization and reinitialization costs will have to
> be taken into account, but the idea that just allocating new objects
> all the time is OK is plausible enough that I would do some profiling
> before changing anything. (Also, I want to believe it because I am
> sooo lazy.)
It generally works out that way. Some time ago I had to process some
large squid logs. There was a C program in their contrib dir that did
what I needed. First run on a 100 MB file it dumped core. I wrote
something in CL that parsed the log line for line returning subseq's
with (values ...) that get used and immediately thrown away. I knew I
could provide storage to the parser function to save its output in as
an optimisation but I didn't want to bother in the first version. I
haven't done since. It just is fast enough.
--
Lieven Marchand <···@wyrd.be>
She says, "Honey, you're a Bastard of great proportion."
He says, "Darling, I plead guilty to that sin."
Cowboy Junkies -- A few simple words
<·······@inetmi.com> wrote in message
··················@chicago.inetmi.com...
> Tim Bradshaw <···@tfeb.org> writes:
>
>
> > Well, one thing I wonder about is if just allocating new events is
> > that bad.
>
> I wondered the same thing. I have been infected with the idea that
> the cost of consing something that is not going to be around for very
> long is really cheap in Lisps with generational/ephemeral GC, so cheap
> that explicit resource pooling is usually not a win (maybe I got this
> idea from On Lisp, I don't remember).
>
> I'm sure that initialization and reinitialization costs will have to
> be taken into account, but the idea that just allocating new objects
> all the time is OK is plausible enough that I would do some profiling
> before changing anything. (Also, I want to believe it because I am
> sooo lazy.)
>
I strongly recommend doing some careful measurements. I recall
that in CLIM, allocating new events and re-initializing resourced
events had different behaviors under different conditions on different
Lisp implementations. My memory is fuzzy, but I think that on Genera
and MCL, it was more efficient to just take the GC hit. I can't
remember for Lucid, LispWorks, or Franz. That code was written
so long ago, that I am sure that any recollection I have is completely
irrelevant anyway.
I would definitely come up with some realistic test cases and do the
measurements. The results would probably be pretty interesting.
Sorry for the digression, but what about structs? This is irrelevant if
one really needs CLOS instances, but for the immediate problem (events)
structs might suffice, and some timings I did under MCL a long time ago
showed that making structs was mad faster than making CLOS instances.
kt
"Kenny Tilton" <·······@nyc.rr.com> wrote in message
······················@nyc.rr.com...
> Sorry for the digression, but what about structs? This is irrelevant if
> one really needs CLOS instances, but for the immediate problem (events)
> structs might suffice, and some timings I did under MCL a long time ago
> showed that making structs was mad faster than making CLOS instances.
>
I'm sure structs are faster, because their initialization protocol is
so light-weight -- and is in fact identical to the reinitialization.
Kenny Tilton <·······@nyc.rr.com> writes:
> Sorry for the digression, but what about structs? This is irrelevant if
> one really needs CLOS instances,
Uh, not to be too picky here, but we call them "standard objects".
Because of metaclasses, all objects are CLOS instances; it's just
that MAKE-INSTANCE can't make all such instances.
> but for the immediate problem (events)
> structs might suffice, and some timings I did under MCL a long time ago
> showed that making structs was mad faster than making CLOS instances.
"... making structure objects was made faster than making standard objects."
Kent M Pitman wrote:
>
> Kenny Tilton <·······@nyc.rr.com> writes:
>
> > Sorry for the digression, but what about structs? This is irrelevant if
> > one really needs CLOS instances,
>
> Uh, not to be too picky here, but we call them "standard objects".
No problem. Mongo like precision.
> Because of metaclasses, all objects are CLOS instances; it's just
> that MAKE-INSTANCE can't make all such instances.
>
> > but for the immediate problem (events)
> > structs might suffice, and some timings I did under MCL a long time ago
> > showed that making structs was mad faster than making CLOS instances.
>
> "... making structure objects was made faster than making standard objects."
Mongo wonder: do we also say "structure classes" vs "standard classes"?
ken
Kenny Tilton <·······@nyc.rr.com> writes:
> Kent M Pitman wrote:
> >
> > Kenny Tilton <·······@nyc.rr.com> writes:
> >
> > > Sorry for the digression, but what about structs? This is irrelevant if
> > > one really needs CLOS instances,
> >
> > Uh, not to be too picky here, but we call them "standard objects".
>
> No problem. Mongo like precision.
>
> > Because of metaclasses, all objects are CLOS instances; it's just
> > that MAKE-INSTANCE can't make all such instances.
> >
> > > but for the immediate problem (events)
> > > structs might suffice, and some timings I did under MCL a long time ago
> > > showed that making structs was mad faster than making CLOS instances.
> >
> > "... making structure objects was made faster than making standard objects."
>
> Mongo wonder: do we also say "structure classes" vs "standard classes"?
Yes. STRUCTURE-CLASS and STANDARD-CLASS are the respective metaclasses
of structure objects and standard objects, and "structure class" and
"standard class" are their common names.