From: Jim Newton
Subject: customizing LOOP for a custom class
Date: 
Message-ID: <2mha58Fmlt8sU1@uni-berlin.de>
I generally find the LOOP constructs more readable than
various DO constructs or MAP-this-and-that with LAMBDA.
However, sometimes i need to iterate in some way over
a custom object.

Python has the ability to define what the FOR loop does
with an instance of a custom class.  This does not seem
soooo difficult to implement in CL.  I'd try to do
it myself, but i'd perfer to use a standard mechanism if
there is one.

Here is an example.

;; here i define what it means to iterate over a MOSAIC
(defmethod map-mosaic (fun (mosaic mosaic))
   (loop for row from 1 to (rows mosaic)
         do (loop for col from 1 to (cols mosaic)
                  ;; note MOSAIC-PT is a method defined on MOSAIC
                  do (funcall fun (mosaic-pt mosaic col row)))))

(defmethod remove-pts-from-hash (( mosaic mosaic) pt-hash)
   (map-mosaic #'(lambda ( pt)
		  (remhash pt pt-hash))
	      mosaic)
   pt-hash)

(defmethod get-points (( mosaic mosaic))
   (let ( points)
     (map-mosaic #'(lambda ( pt)
		    (push pt points))
		mosaic)
     points))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; I would like to do the following instead
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defmethod remove-pts-from-hash (( mosaic mosaic) pt-hash)
   (loop for pt in mosaic
       (remhash pt pt-hash))
    pt-hash)

(defmethod get-points (( mosaic mosaic))
   (loop for pt in mosaic
      collect pt))

From: Carl Shapiro
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <ouy658cfmtm.fsf@panix3.panix.com>
Jim Newton <·····@rdrop.com> writes:

> Python has the ability to define what the FOR loop does
> with an instance of a custom class.  This does not seem
> soooo difficult to implement in CL.  I'd try to do
> it myself, but i'd perfer to use a standard mechanism if
> there is one.

If you have the misfortune of using a Common Lisp system which does
not document or ship with the extension interface, download an
uncrippled LOOP such as GSB LOOP or the publicly available Symbolics
LOOP and read up on its extension protocol.  With just a few lines of
code you can start writing definitions like

  (defun vertex-list (object)
    (loop for vertex being the vertices of object
          collect vertex))

The excellent Symbolics LOOP code is still available at the CMU AI
repository.  If you poke around the test code which comes with the
sources you should find examples of inclusive LOOP paths and other
such winnage the purists wouldn't dare tell you about.

  http://www-cgi.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/iter/loop/0.html
From: Jim Newton
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <2mhcl3Fmp1snU1@uni-berlin.de>
Here is my first attempt

(defclass mosaic ( ...)
    ...)



(defmacro foreach ( var obj &rest exprs)
   `(foreach-map ,obj #'(lambda (,var) (progn ,@exprs))))



(defmethod foreach-map ( expr ( list cons))
   (mapc expr list))

(defmethod foreach-map ( expr  ( hash hash-table))
   (maphash expr hash))

(defmethod foreach-map ( expr ( mosaic mosaic))
   (map-mosaic expr mosaic))


I guess this could be reduced further by defining
foreach-map on class t... Not sure if it makes
things better or more obscure?

(defmethod foreach-map ( expr obj)
   (funcall (foreach-get-map obj) expr obj))

(defmethod foreach-get-map (( obj cons))
    #'mapc)

(defmethod foreach-get-map (( obj hash-table))
    #'map-hash)

(defmethod foreach-get-map (( obj mosaic))
    #'map-mosaic)




Jim Newton wrote:
> I generally find the LOOP constructs more readable than
> various DO constructs or MAP-this-and-that with LAMBDA.
> However, sometimes i need to iterate in some way over
> a custom object.
> 
> Python has the ability to define what the FOR loop does
> with an instance of a custom class.  This does not seem
> soooo difficult to implement in CL.  I'd try to do
> it myself, but i'd perfer to use a standard mechanism if
> there is one.
> 
> Here is an example.
> 
> ;; here i define what it means to iterate over a MOSAIC
> (defmethod map-mosaic (fun (mosaic mosaic))
>   (loop for row from 1 to (rows mosaic)
>         do (loop for col from 1 to (cols mosaic)
>                  ;; note MOSAIC-PT is a method defined on MOSAIC
>                  do (funcall fun (mosaic-pt mosaic col row)))))
> 
> (defmethod remove-pts-from-hash (( mosaic mosaic) pt-hash)
>   (map-mosaic #'(lambda ( pt)
>           (remhash pt pt-hash))
>           mosaic)
>   pt-hash)
> 
> (defmethod get-points (( mosaic mosaic))
>   (let ( points)
>     (map-mosaic #'(lambda ( pt)
>             (push pt points))
>         mosaic)
>     points))
> 
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> ;; I would like to do the following instead
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> 
> (defmethod remove-pts-from-hash (( mosaic mosaic) pt-hash)
>   (loop for pt in mosaic
>       (remhash pt pt-hash))
>    pt-hash)
> 
> (defmethod get-points (( mosaic mosaic))
>   (loop for pt in mosaic
>      collect pt))
> 
From: Rahul Jain
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <873c3gszd2.fsf@nyct.net>
Jim Newton <·····@rdrop.com> writes:

> I generally find the LOOP constructs more readable than
> various DO constructs or MAP-this-and-that with LAMBDA.
> However, sometimes i need to iterate in some way over
> a custom object.

Many implementations of LOOP have a way to extend the iteration pathways
available, but this feature is not standardized. SERIES is a comparable
system to LOOP, but allows for easy extension. You express SERIES
operations as lazy sequence operations and then, if possible, the SERIES
library will convert that to an iterative construct. Adding your own
SERIES operator is a matter of a simple DEFUN with a declaration of how
it is to behave when SERIES tries to compile it as part of an iteration. 
Documentation is available in the appendices of CLtL2. A maintained
implementation is available at <http://series.sourceforge.net>.

-- 
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Jim Newton
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <2miediFmk8k0U1@uni-berlin.de>
so why not just define a sort of foreach method
on any class?  am i thinking too simple?
-jim

Rahul Jain wrote:
> Jim Newton <·····@rdrop.com> writes:
> 
> 
>>I generally find the LOOP constructs more readable than
>>various DO constructs or MAP-this-and-that with LAMBDA.
>>However, sometimes i need to iterate in some way over
>>a custom object.
> 
> 
> Many implementations of LOOP have a way to extend the iteration pathways
> available, but this feature is not standardized. SERIES is a comparable
> system to LOOP, but allows for easy extension. You express SERIES
> operations as lazy sequence operations and then, if possible, the SERIES
> library will convert that to an iterative construct. Adding your own
> SERIES operator is a matter of a simple DEFUN with a declaration of how
> it is to behave when SERIES tries to compile it as part of an iteration. 
> Documentation is available in the appendices of CLtL2. A maintained
> implementation is available at <http://series.sourceforge.net>.
> 
From: Rahul Jain
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <87u0vvswlw.fsf@nyct.net>
Jim Newton <·····@rdrop.com> writes:

> so why not just define a sort of foreach method
> on any class?  am i thinking too simple?

That wouldn't be optimizable, unless, of course, you knew the type of
the object being passed in at compile time. But in that case, you're not
making use of OOP, so you might as well not use it. Also, there are many
ways in which you might want to traverse a specific type. For example,
scan, scan-plist, and scan-alist in series. For binary trees, you might
want a depth-first or breadth-first traversal. You won't get that kind
of control with the scheme you propose...

Actually, you can, but it's FAR more complex than you might figure at
first. You'd need all kinds of keyword parameters that would take effect
or not depending on the method that is invoked. E.g, you would have
something like:

(scan-generic data-structure :tree-traversal :breadth-first :list-traversal :plist)

When called with, say, a binary tree of elements, would return a single
series that traverses the elements in breadth-first order. When called
with a list, it would return two values, a series of keys in the plist
and a series of values in the plist. Usage of such a function can get
complicated if input types are so unrestrained. And you won't get the
optimization that SERIES can give you, as the traversal code won't be
known at compile time and so, can't be inlined into the loop. This will
cause memory allocation as well as function call overhead throughout the
traversal.

-- 
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Jim Newton
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <2mih4pFmqhk2U1@uni-berlin.de>
ok, so then you could use a multi method

first argument being the traversable/iteratable/loopable object,
the second being an object whose various methods guide the
traversal type.

E.g., you might want to iterate over a list with
mapc, mapcar, or mapcan, or your own mapping function.

so to iterate you'd have to allocate an object
to iterate over, plus an interator object, plus
a function to eval on each iteration.

how's that sound?

-jim

Rahul Jain wrote:
> Jim Newton <·····@rdrop.com> writes:
> 
> 
>>so why not just define a sort of foreach method
>>on any class?  am i thinking too simple?
> 
> 
> That wouldn't be optimizable, unless, of course, you knew the type of
> the object being passed in at compile time. But in that case, you're not
> making use of OOP, so you might as well not use it. Also, there are many
> ways in which you might want to traverse a specific type. For example,
> scan, scan-plist, and scan-alist in series. For binary trees, you might
> want a depth-first or breadth-first traversal. You won't get that kind
> of control with the scheme you propose...
> 
> Actually, you can, but it's FAR more complex than you might figure at
> first. You'd need all kinds of keyword parameters that would take effect
> or not depending on the method that is invoked. E.g, you would have
> something like:
> 
> (scan-generic data-structure :tree-traversal :breadth-first :list-traversal :plist)
> 
> When called with, say, a binary tree of elements, would return a single
> series that traverses the elements in breadth-first order. When called
> with a list, it would return two values, a series of keys in the plist
> and a series of values in the plist. Usage of such a function can get
> complicated if input types are so unrestrained. And you won't get the
> optimization that SERIES can give you, as the traversal code won't be
> known at compile time and so, can't be inlined into the loop. This will
> cause memory allocation as well as function call overhead throughout the
> traversal.
> 
From: Rahul Jain
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <87pt6jsusr.fsf@nyct.net>
Jim Newton <·····@rdrop.com> writes:

> ok, so then you could use a multi method
>
> first argument being the traversable/iteratable/loopable object,
> the second being an object whose various methods guide the
> traversal type.
>
> E.g., you might want to iterate over a list with
> mapc, mapcar, or mapcan, or your own mapping function.
>
> so to iterate you'd have to allocate an object
> to iterate over, plus an interator object, plus
> a function to eval on each iteration.
>
> how's that sound?

Too slow for me. :)

You can already get this functionality by using SERIES, you just won't
get any optimization. In fact, if you put the second parameter (which is
extremely unlikely to be useful as an instance or for subclassing) as
part of the function name, you get what SERIES already does.

To those used to Java, this seems to be the "normal" way to do things,
but many projects I've seen have avoided using Iterators because of this
excessive overhead.

-- 
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Marco Baringer
Subject: Re: customizing LOOP for a custom class
Date: 
Message-ID: <m2pt6k6w9i.fsf@convey.it>
Jim Newton <·····@rdrop.com> writes:

> I generally find the LOOP constructs more readable than
> various DO constructs or MAP-this-and-that with LAMBDA.
> However, sometimes i need to iterate in some way over
> a custom object.

may i suggest using to iterate? (http://www.cliki.net/iterate)
it's a simple matter to define a FOR ... IN-MOSAIC ... clause.

-- 
-Marco
Ring the bells that still can ring.
Forget your perfect offering.
There is a crack in everything.
That's how the light gets in.
     -Leonard Cohen