From: Karol Skocik
Subject: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129276206.781939.189220@g47g2000cwa.googlegroups.com>
Hello guys,
  I have another macro problem. I have changed the definition of the
edge in the graph, and now it looks like this :

(defstruct edge
  attribute
  source
  target
  source-out-index
  target-in-index)

where source, target now points to nodes themselves instead just
holding index to array where they reside. The problem now is, that one
macro expands indefinitely.
I have this macro, which is needed to construct a function which will
match (graph isomorphism) some parts of the graph. It looks like this
(my macro style is not very advanced yet) :

(defmacro genproxy ((&body sc-body)
		    &key (node-attr-comparator #'(lambda (x y) (declare (ignore x y))
t))
		    (edge-attr-comparator #'(lambda (x y) (declare (ignore x y)) t)))
  (let ((main-proxy-graph (create-graph (gensc sc-body))))
    (arrange-graph-with-proxies! main-proxy-graph)
    (let* ((proxies-only-edges (remove-proxies-only-edges!
main-proxy-graph))
	   (proxy-graphs (make-connected-graphs! main-proxy-graph)))
      (labels ((proxy-edge-test (edge-desc)
		 `(progn
		   (setf search-graph-edge (get-out-edge search-graph
					    (aref core->search ,(caar edge-desc)) (aref core->search
,(cdar edge-desc))))
		   (unless (and search-graph-edge (funcall ,edge-attr-comparator
,(cdr edge-desc) (edge-attribute search-graph-edge)))
		     (return-from lambda-block nil))))

               ;; *HERE* is the problem
	       (proxy-graph-test (proxy-graph)
		 (unless (typep (node-attribute (node-of proxy-graph (1-
(nodes-count-of proxy-graph)))) 'sc-proxy)
		   `(when (zerop (rematch (make-rematch-state ,proxy-graph
search-graph core->search
							      :node-attr-comparator ,node-attr-comparator
							      :edge-attr-comparator ,edge-attr-comparator)
				  :find-more nil))
		     (return-from lambda-block nil)))))



	`#'(lambda (search-graph core->search)
	     (block lambda-block
	       ,(unless (null proxies-only-edges)
			`(let (search-graph-edge)
			  ,@(let ((edge-test-forms '()))
				 (mapc #'(lambda (edge-desc) (push (proxy-edge-test edge-desc)
edge-test-forms))
				       proxies-only-edges)
				 (nreverse edge-test-forms))))
	       ,@(let ((proxy-graph-test-forms '()))
		      (mapc #'(lambda (proxy-graph)
				(let ((form (proxy-graph-test (car proxy-graph))))
				  (unless (null form) (push form proxy-graph-test-forms))))
			    proxy-graphs)
		      (nreverse proxy-graph-test-forms))
	       t))))))

Hmm, looks like a mess little bit in here... There is a function
defined in labels - 'proxy-graph-test' which creates a piece of code
which is then a part of macro. But that is the place where it stops
working. Is there any way how Lisp can handle recursive structures in
macros? When printing, it's enough to set *print-circle* t,
*print-level* 3. In macros, I thought, that the structure which is
passed in s-exp format will have the same #1=#S{node ... pieces in the
structure definition like when printing the struct in REPL, but looks
like it's not true. However, I can rewrite graph structure to be a
class, but I would like to have this part of code as lightweight as
possible. 

Thanks,
  Karol Skocik

From: Kaz Kylheku
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129304728.003907.77570@g44g2000cwa.googlegroups.com>
Karol Skocik wrote:
> Hello guys,
>   I have another macro problem. I have changed the definition of the
> edge in the graph, and now it looks like this :
>
> (defstruct edge
>   attribute
>   source
>   target
>   source-out-index
>   target-in-index)
>
> where source, target now points to nodes themselves instead just
> holding index to array where they reside. The problem now is, that one
> macro expands indefinitely.

The macro does not generate code which calls the macro again. So that
does not appear to be the case. I think what you are seeing is that the
macro does not terminate? That's not the same thing.
Macros which expand indefinitely do actually terminate; they just cause
the macro expansion logic above them to fail to terminate (until it
hits a resource limit).

(A macro /can/ generate code that contains more calls to that macro,
but those calls should be simpler cases that eventually reduce to
expressions that no longer call that macro).

> 	   (proxy-graphs (make-connected-graphs! main-proxy-graph)))

The list of proxy-graphs comes from this MAKE-CONNECTED-GRAPHS!
function. This is later processed through MAPC, so it better be a
proper list that is free of cycles.

>                ;; *HERE* is the problem
> 	       (proxy-graph-test (proxy-graph)
> 		 (unless (typep (node-attribute (node-of proxy-graph (1-
> (nodes-count-of proxy-graph)))) 'sc-proxy)
> 		   `(when (zerop (rematch (make-rematch-state ,proxy-graph
> search-graph core->search
> 							      :node-attr-comparator ,node-attr-comparator
> 							      :edge-attr-comparator ,edge-attr-comparator)
> 				  :find-more nil))
> 		     (return-from lambda-block nil)))))

Everything actually done here is quite harmless. But this block relies
on three functions: NODES-COUNT-OF, NODE-OF and NODE-ATTRIBUTE.  Is it
possible that these have a termination problem? How does NODE-OF
retrieve the n-th node of the graph? Does it chase any edges, that
could possibly lead it into an infinite cycle?

> 	       ,@(let ((proxy-graph-test-forms '()))
> 		      (mapc #'(lambda (proxy-graph)
> 				(let ((form (proxy-graph-test (car proxy-graph))))
> 				  (unless (null form) (push form proxy-graph-test-forms))))
> 			    proxy-graphs)
> 		      (nreverse proxy-graph-test-forms))

Here is the MAPC that actually drives the above code. Does this
terminate?

If PROXY-GRAPHS is really a list of proxy graphs, why do you have to
call CAR on its elements to retrieve a proxy graph argument for the
PROXY-GRAPH-TEST function?

    (mapc #'(lambda (PROXY-GRAPH) ... (proxy-graph-test (CAR
PROXY-GRAPH))) PROXY-GRAPHS)

what's wrong with this picture? I have a list of proxy graphs. I MAPC
over it, using a lambda whose argument is a proxy graph. But then that
argument turns out to be a cons whose CAR is a proxy graph? Hmm.

Better change the names so they reflect what these things really are!


> Hmm, looks like a mess little bit in here... There is a function
> defined in labels - 'proxy-graph-test' which creates a piece of code
> which is then a part of macro. But that is the place where it stops
> working. Is there any way how Lisp can handle recursive structures in
> macros?

It can handle things just fine. I think you might want to write your
macro as a module of functions that transform an input expression and
return the output one. Test it interactively:

  (my-graph-transformer '(sample input expression)) ==> (correct
output?)

Don't be afraid to break it out into small, top-level functions.

When it all works, wrap it behind a macro that calls the transformer
funtion. Don't forget to wrap all the functions in a suitable EVAL-WHEN
so they get defined within the compiler at macro-expansion time.
From: Karol Skocik
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129306645.579666.143590@z14g2000cwz.googlegroups.com>
Well, I should sometimes choose better names, you are right here :
>If PROXY-GRAPHS is really a list of proxy graphs, why do you have to
>call CAR on its elements to retrieve a proxy graph argument for the
>PROXY-GRAPH-TEST function?
>    (mapc #'(lambda (PROXY-GRAPH) ... (proxy-graph-test (CAR
>PROXY-GRAPH))) PROXY-GRAPHS)

this is harmless, proxy-graphs are is list of (cons graph mapping)
where the mapping is not used yet.

The point is, that it worked as it should before I made the graph
structure recursive. And since it expands the graph structure into
s-exps, for some reason, it looks like compiler follows the source and
target nodes in edge structure and this is where it hangs.

node-count-of, node-of and node-attribute are just wrappers over array
access to be the code more readable - they should be fine.

I have already written a lot of the functionality of the macro as a
separate functions (in fact - all the functions used in macro are used
just for the macro's purpose) but the problem is somewhere else I think
- it's expanding of the graph structure. But it look like my only
chance here is to rewrite the macro and avoid the compiler to see the
graph expressed as raw s-exp, but as a direct structure which comes out
of create-graph.
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <87k6gguxve.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:
>   I have another macro problem. 
> [...]
> Hmm, looks like a mess little bit in here... 

Yes.

> There is a function
> defined in labels - 'proxy-graph-test' which creates a piece of code
> which is then a part of macro. But that is the place where it stops
> working. Is there any way how Lisp can handle recursive structures in
> macros? When printing, it's enough to set *print-circle* t,
> *print-level* 3.

That seems to be to be completly irrelevant, since I see no
recursivity in this macro.  It doesn't seem to use itself, and the
functions defined in labels are not recursive, they're even not
dependent on each other (you could use flet).

> In macros, I thought, that the structure which is
> passed in s-exp format will have the same #1=#S{node ... pieces in the
> structure definition like when printing the struct in REPL, but looks
> like it's not true. 

Try it:  (loop :for item :in '#1=(x . #1#) :do (print x) (sleep 0.5))

There's no #1= or #1# in the lisp object.

But anyway, if you don't recurse or otherwise walk the structure, you
won't have any circle problem.



Also, are you sure you need a macro here?
For example:
  (let ((main-proxy-graph (create-graph (gensc sc-body))))
it seems like you're taking a list of s-expressions and turn it into data.
This is not the macro essence, which is to take s-expressions and turn
them into s-expressions.
I note that your macro returns a function:
        `#'(lambda (search-graph core->search)
this is not a s-expression, this is data. (functions are first class
objects in lisp).  
Perhaps you are transmitting thru the graph some of sc-body into the code
generated into this lambda and you are wanting to capture the lexical
scope?  If this is not the case, then you don't need a macro.


> However, I can rewrite graph structure to be a
> class, but I would like to have this part of code as lightweight as
> possible. 

You'd have the same problem I guess.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You never feed me.
Perhaps I'll sleep on your face.
That will sure show you.
From: Karol Skocik
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129293534.924463.259160@o13g2000cwo.googlegroups.com>
The macro worked fine before I changed the indexes to arrays (where the
specific source or target was) to a nodes itselves. I know the macro is
not recursive, the problem is not in the macro but in the
representation of the graph which is as a structure expanded to
s-expressions, and this is where the problem is, I think.
  I am not sure if macro is needed for this, but the input is structure
of graph written in s-exps, and the output is s-exps too - body of
lambda. Can I build the body of function piece by piece by normal
function with the similar way?

when I have a sc-body like this :

(defvar *g-struct-2* '((0 a ((1 x) (2 y)))
		       (1 b ((2 z)))
		       (2 c ((0 g)))
		       (3 d ((0 a) (1 f) (2 k) (4 b)))
		       (4 e ((2 h) (3 t)))))
format : (index node-attribute (out-edges (target-index
edge-attribute)))

it builds graph like this : (with *print-level* = 4)

GI> (create-graph *g-struct-2*)
#S(GRAPH
     :NODES #(#S(NODE
                   :ATTRIBUTE A
                   :ARRAY-INDEX 0
                   :OUT-EDGES #(#1=# #2=#)
                   :IN-EDGES #(#3=# #4=#))
              #S(NODE
                   :ATTRIBUTE B
                   :ARRAY-INDEX 1
                   :OUT-EDGES #(#5=#)
                   :IN-EDGES #(#1# #6=#))
              #S(NODE
                   :ATTRIBUTE C
                   :ARRAY-INDEX 2
                   :OUT-EDGES #(#3#)
                   :IN-EDGES #(#2# #5# #7=# #8=#))
              #S(NODE
                   :ATTRIBUTE D
                   :ARRAY-INDEX 3
                   :OUT-EDGES #(#4# #6# #7# #9=#)
                   :IN-EDGES #(#10=#))
              #S(NODE
                   :ATTRIBUTE E
                   :ARRAY-INDEX 4
                   :OUT-EDGES #(#8# #10#)
                   :IN-EDGES #(#9#)))
     :GROWTH-SPEED 1.5)

- the same structure written in the same s-exp format expands in macro.
but, while I can limit the nesting in REPL with *print-level*, the
macro is expanding the structure indefinitely, because it is following
source and target nodes in every edge.

I think, that when it will be CLOS instance, the problem will
dissappear, sice CLOS instance is written as a #<...> reference, not
showing it's internal structure. But having the same done with
structure will make me happier.

Any ideas will be greatly appreciated.
Karol
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <87wtkgt63s.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:

> The macro worked fine before I changed the indexes to arrays (where the
> specific source or target was) to a nodes itselves. I know the macro is
> not recursive, the problem is not in the macro but in the
> representation of the graph which is as a structure expanded to
> s-expressions, and this is where the problem is, I think.
>   I am not sure if macro is needed for this, but the input is structure
> of graph written in s-exps, and the output is s-exps too - body of
> lambda. Can I build the body of function piece by piece by normal
> function with the similar way?
>
> when I have a sc-body like this :
>
> (defvar *g-struct-2* '((0 a ((1 x) (2 y)))
> 		       (1 b ((2 z)))
> 		       (2 c ((0 g)))
> 		       (3 d ((0 a) (1 f) (2 k) (4 b)))
> 		       (4 e ((2 h) (3 t)))))
> format : (index node-attribute (out-edges (target-index
> edge-attribute)))
>
> it builds graph like this : (with *print-level* = 4)
>
> GI> (create-graph *g-struct-2*)
> #S(GRAPH
>      :NODES #(#S(NODE
>                    :ATTRIBUTE A
>                    :ARRAY-INDEX 0
>                    :OUT-EDGES #(#1=# #2=#)
>                    :IN-EDGES #(#3=# #4=#))
>               #S(NODE
>                    :ATTRIBUTE B
>                    :ARRAY-INDEX 1
>                    :OUT-EDGES #(#5=#)
>                    :IN-EDGES #(#1# #6=#))
>               #S(NODE
>                    :ATTRIBUTE C
>                    :ARRAY-INDEX 2
>                    :OUT-EDGES #(#3#)
>                    :IN-EDGES #(#2# #5# #7=# #8=#))
>               #S(NODE
>                    :ATTRIBUTE D
>                    :ARRAY-INDEX 3
>                    :OUT-EDGES #(#4# #6# #7# #9=#)
>                    :IN-EDGES #(#10=#))
>               #S(NODE
>                    :ATTRIBUTE E
>                    :ARRAY-INDEX 4
>                    :OUT-EDGES #(#8# #10#)
>                    :IN-EDGES #(#9#)))
>      :GROWTH-SPEED 1.5)
>
> - the same structure written in the same s-exp format expands in macro.
> but, while I can limit the nesting in REPL with *print-level*, the
> macro is expanding the structure indefinitely, because it is following
> source and target nodes in every edge.


Instead of *print-level*, you should (setf *print-circle* t) and have
a look at the whole structure:

(defstruct graph
  (nodes (make-array 0) :type vector)
  (growth-speed 1.5))

(defstruct node attribute array-index out-edges in-edges)
(defstruct edge attribute source target source-out-index target-in-index)

(setf *print-circle* t)

(defun create-graph (g)
  (let ((gr (make-graph :nodes (make-array (length g)))))
    (loop
       :for node :in g
       :do (cond
             ((aref (graph-nodes gr) (first node))
              (error "Node slot occupied"))
             ((<= (length (graph-nodes gr)) (first node))
              (error "Node index too big"))
             (t (setf (aref (graph-nodes gr) (first node))
                      (make-node :attribute (second node)
                                 :array-index (first node)
                                 :out-edges (make-array 0 :adjustable t
                                                        :fill-pointer 0)
                                 :in-edges  (make-array 0 :adjustable t
                                                        :fill-pointer 0))))))
    (loop
       :for e :in (mapcan (lambda (n) (mapcan (lambda (e) (list (cons (first n) e)))
                                          (third n))) g)
       :for ed = (make-edge
                 :attribute (third e)
                 :source (aref (graph-nodes gr) (first  e))
                 :target (aref (graph-nodes gr) (second e))
                 :source-out-index (node-array-index
                                     (aref (graph-nodes gr) (first  e)))
                 :target-in-index (node-array-index
                                    (aref (graph-nodes gr) (second e))))
       :do
       (vector-push-extend
            ed (node-out-edges
                (aref (graph-nodes gr) (edge-source-out-index ed))))
       (vector-push-extend
            ed (node-in-edges
                (aref (graph-nodes gr) (edge-target-in-index ed)))))
    gr))

(defvar *g* '((0 a ((1 x) (2 y)))
              (1 b ((2 z)))
              (2 c ((0 g)))
              (3 d ((0 a) (1 f) (2 k) (4 b)))
              (4 e ((2 h) (3 t)))))


[131]> (create-graph *g*)
#S(GRAPH
   :NODES
   #(#1=#S(NODE :ATTRIBUTE A :ARRAY-INDEX 0
     :OUT-EDGES
     #(#2=#S(EDGE :ATTRIBUTE X :SOURCE #1#
       :TARGET
       #3=#S(NODE :ATTRIBUTE B :ARRAY-INDEX 1
       :OUT-EDGES
       #(#4=#S(EDGE :ATTRIBUTE Z :SOURCE #3#
         :TARGET
         #5=#S(NODE :ATTRIBUTE C :ARRAY-INDEX 2
               :OUT-EDGES
               #(#6=#S(EDGE :ATTRIBUTE G :SOURCE #5#
                 :TARGET #1# :SOURCE-OUT-INDEX 2
                 :TARGET-IN-INDEX 0))
               :IN-EDGES
               #(#7=#S(EDGE :ATTRIBUTE Y :SOURCE #1#
                 :TARGET #5# :SOURCE-OUT-INDEX 0
                 :TARGET-IN-INDEX 2)
           #4#
           #8=#S(EDGE :ATTRIBUTE K
                 :SOURCE
                 #9=#S(NODE :ATTRIBUTE D
                 :ARRAY-INDEX 3
                 :OUT-EDGES
                 #(#10=#S(EDGE :ATTRIBUTE A
                    :SOURCE #9#
                    :TARGET #1#
                    :SOURCE-OUT-INDEX
                    3
                    :TARGET-IN-INDEX
                    0)
                   #11=#S(EDGE :ATTRIBUTE F
                    :SOURCE #9#
                    :TARGET #3#
                    :SOURCE-OUT-INDEX
                    3
                    :TARGET-IN-INDEX
                    1)
                   #8#
                   #12=#S(EDGE :ATTRIBUTE B
                    :SOURCE #9#
                    :TARGET
                    #13=#S(NODE
                     :ATTRIBUTE
                     E
                     :ARRAY-INDEX
                     4
                     :OUT-EDGES
                     #(#14=#S(EDGE
                        :ATTRIBUTE
                        H
                        :SOURCE
                        #13#
                        :TARGET
                        #5#
                        :SOURCE-OUT-INDEX
                        4
                        :TARGET-IN-INDEX
                        2)
                       #15=#S(EDGE
                        :ATTRIBUTE
                        T
                        :SOURCE
                        #13#
                        :TARGET
                        #9#
                        :SOURCE-OUT-INDEX
                        4
                        :TARGET-IN-INDEX
                        3))
                     :IN-EDGES
                     #(#12#))
                    :SOURCE-OUT-INDEX
                    3
                    :TARGET-IN-INDEX
                    4))
                 :IN-EDGES #(#15#))
                 :TARGET #5# :SOURCE-OUT-INDEX 3
                 :TARGET-IN-INDEX 2)
           #14#))
         :SOURCE-OUT-INDEX 1 :TARGET-IN-INDEX 2))
       :IN-EDGES #(#2# #11#))
       :SOURCE-OUT-INDEX 0 :TARGET-IN-INDEX 1)
       #7#)
     :IN-EDGES #(#6# #10#))
     #3# #5# #9# #13#)
   :GROWTH-SPEED 1.5)


Now, if you look at your math book, you'll see that the definition for
a graph is a couple ( set-of-nodes, set-of-edges ).

You've decided to implement your graph structure without the
set-of-edges, just because it can be derived from the data you keep
with the nodes, but some processing on the graph need to work on each
edge independantly of the nodes.  In your "macro", I don't see that
you're waking the graph following the edges either. Why not just
enumerate them and process them in turn?  Or, if you're really walking
the graph, then you can just mark any edge traversed to avoid looping.

(defun graph-edges (gr)
  (loop
     :for node :across (graph-nodes gr)
     :nconc (coerce  (node-out-edges node) 'list)))
;; try: (setf *print-circle* t) (graph-edges (create-graph *g*))


Then you can process all edges at once:
`(when (or ,@(mapcar (function generate-edge-test) (graph-edges gr)))
   (return-from lambda-block))



(defun walk-edges-once-from-node (start-node node-fun edge-fun)
  (let ((walked '()))
    (labels ((walk (node)
               (funcall node-fun node)
               (loop
                  :for edge :across (node-out-edges node)
                  :unless (member edge walked)
                  :do (progn (push edge walked)
                             (funcall edge-fun edge)
                             (walk (edge-target edge))))))
      (walk start-node))))

(walk-edges-once-from-node
 (aref (graph-nodes (create-graph *g*)) 0)
 (lambda (node) (format t "NODE: ~A (~D)~%"
                   (node-attribute node) (node-array-index node)))
 (lambda (edge) (format t "EDGE: ~A (~D->~D)~%"
                   (edge-attribute edge)
                   (edge-source-out-index edge)
                   (edge-target-in-index edge))))

NODE: A (0)
EDGE: X (0->1)
NODE: B (1)
EDGE: Z (1->2)
NODE: C (2)
EDGE: G (2->0)
NODE: A (0)
EDGE: Y (0->2)
NODE: C (2)

Then you can generate your stuff with;


(let ((edge-forms '()) (node-forms '()))
  (walk-edges-once-from-node 
   (aref (graph-nodes (create-graph *g*)) 0)
   (lambda (node) (push `(when (test-node-p ',node) (do-something)) node-forms))
   (lambda (edge) (push `(when (test-edge-p ',edge) (do-something)) edge-forms)))
  `(progn ,@edge-forms ,@node-forms))


> I think, that when it will be CLOS instance, the problem will
> dissappear, sice CLOS instance is written as a #<...> reference, not
> showing it's internal structure. But having the same done with
> structure will make me happier.

Ok, I'll leak you the secret: structures are objects!

(class-of (make-graph)) -> #<STRUCTURE-CLASS GRAPH>

So that'd make no difference for your bugs.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Wanna go outside.
Oh, no! Help! I got outside!
Let me back inside!
From: Karol Skocik
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129310543.036513.246540@f14g2000cwb.googlegroups.com>
OK, so I resurrected my old working code to point out exactly where is
the problem. this uses the old functions like (the interesting stuff is
at the bottom) :
(sorry for messy look, I write quite a long lines which fits ok in
emacs, but not here)

(defstruct edge
  attribute
  source-index                  ;; **** this is the old working
definition
  target-index                   ;; **** the same - in new - here are
direct nodes
  source-out-index           ;; where in source node out-edges is
placed
  target-in-index)           ;; where in target node in-edges is placed

(defstruct node
  attribute
  array-index                  ;; where in graph nodes is this node
placed
  out-edges
  in-edges)

(defstruct graph
  nodes
  growth-speed)

;; these could be inline
(defmacro nodes-count-of (graph) `(length (graph-nodes ,graph)))
(defmacro node-of (graph index) `(aref (graph-nodes ,graph) ,index))

(defun create-graph (graph-struct &key (growth-speed 1.5))
  (let* ((node-count (length graph-struct))
	 (graph (make-graph :nodes (make-array node-count :fill-pointer 0)
			    :growth-speed (if (< growth-speed 1.01) 1.5 growth-speed))))
    (dolist (node-struct graph-struct)
      (destructuring-bind (index attribute edges-struct) node-struct
	(let ((node (make-node :attribute attribute
			       :array-index index
			       :out-edges (make-array (max (length edges-struct) 1)
:fill-pointer 0)
			       :in-edges (make-array (max (in-edges-count index
graph-struct) 1) :fill-pointer 0))))
	  (dolist (edge-struct edges-struct)
	    (vector-push (make-edge :attribute (second edge-struct)
				    :source-index index
				    :target-index (car edge-struct))
			 (node-out-edges node)))
	  (sort (node-out-edges node) (lambda (x y) (< (edge-target-index x)
(edge-target-index y))))
	  (vector-push node (graph-nodes graph)))))
    (dovector (node (graph-nodes graph))
      (dotimes (out-edge-array-pos (length (node-out-edges node)))
	(let* ((edge (aref (node-out-edges node) out-edge-array-pos))
	       (in-edges (node-in-edges (aref (graph-nodes graph)
(edge-target-index edge)))))
	  (setf (edge-source-out-index edge) out-edge-array-pos
		(edge-target-in-index edge) (fill-pointer in-edges))
	  (vector-push edge in-edges))))
    graph))

(defun gensc (&rest sc-body)
  (mapcar #'(lambda (node)
	      (append (list (car node))
		      (let* ((rest-of-form '())
			     (node-ac (case (second node)
					('- +negative-ac+)
					('+ +positive-ac+)
					(t +neutral-ac+)))
			     (node-prop (if (equal node-ac +neutral-ac+)
					    (second node)
					    (third node))))
			(push (cond ((keywordp node-prop) ;; references like :0 ..
				     (make-instance 'sc-proxy
						    :original-array-index (parse-integer (symbol-name
node-prop))))
				    ((or (characterp node-prop) (stringp node-prop) (numberp
node-prop))
				     (make-instance 'sc-constant :value node-prop :app-condition
node-ac))
				    (t (make-instance node-prop :app-condition node-ac)))
			      rest-of-form)
			(let ((edges (car (nthcdr (if (equal node-ac +neutral-ac+) 2 3)
node))))
			  (push (if edges
				    (mapcar #'(lambda (edge)
						(list (car edge)
						      (let* ((edge-ac (case (second edge)
									('- +negative-ac+)
									('+ +positive-ac+)
									(t +neutral-ac+)))
							     (edge-prop (if (equal edge-ac +neutral-ac+)
									    (second edge)
									    (third edge))))
							(if edge-prop
							    (make-instance 'sc-accessor :property edge-prop
:app-condition edge-ac)
							    (make-instance 'sc-relation :app-condition edge-ac)))))
					    edges)
				    nil)
				rest-of-form))
			(nreverse rest-of-form))))
	  (car sc-body)))

;; ======== the same genproxy macro here ============

now, genproxy expanded :

GI> (genproxy ((0 :1 ((1) (2)))
	       (1 sc-variable ((0)))
	       (2 sc-constant ((0) (2)))))

C-c C-m looks like this :

#'(LAMBDA (SEARCH-GRAPH CORE->SEARCH)
    (BLOCK LAMBDA-BLOCK
      NIL
      (WHEN
          (ZEROP
           (REMATCH
            (MAKE-REMATCH-STATE
             #S(GRAPH
                  :NODES #(#S(NODE
                                :ATTRIBUTE #<SC-PROXY {58635A7D}>
                                :ARRAY-INDEX 0
                                :OUT-EDGES #(#S(EDGE
                                                  :ATTRIBUTE
#<SC-RELATION

{58635AED}>
                                                  :SOURCE-INDEX 0
                                                  :TARGET-INDEX 1
                                                  :SOURCE-OUT-INDEX 0
                                                  :TARGET-IN-INDEX 0)
                                             #S(EDGE
                                                  :ATTRIBUTE
#<SC-RELATION

{58635AB5}>
                                                  :SOURCE-INDEX 0
                                                  :TARGET-INDEX 2
                                                  :SOURCE-OUT-INDEX 1
                                                  :TARGET-IN-INDEX 0))
                                :IN-EDGES #(#S(EDGE
                                                 :ATTRIBUTE
#<SC-RELATION

{58635EAD}>
                                                 :SOURCE-INDEX 2
                                                 :TARGET-INDEX 0
                                                 :SOURCE-OUT-INDEX 0
                                                 :TARGET-IN-INDEX 0)
                                            #S(EDGE
                                                 :ATTRIBUTE
#<SC-RELATION

{586378BD}>
                                                 :SOURCE-INDEX 1
                                                 :TARGET-INDEX 0
                                                 :SOURCE-OUT-INDEX 0
                                                 :TARGET-IN-INDEX 1)))
                           #S(NODE
                                :ATTRIBUTE #<SC-CONSTANT {58637865}>
                                :ARRAY-INDEX 1
                                :OUT-EDGES #(#S(EDGE
                                                  :ATTRIBUTE
#<SC-RELATION

{586378BD}>
                                                  :SOURCE-INDEX 1
                                                  :TARGET-INDEX 0
                                                  :SOURCE-OUT-INDEX 0
                                                  :TARGET-IN-INDEX 1)
                                             #S(EDGE
                                                  :ATTRIBUTE
#<SC-RELATION

{586378F5}>
                                                  :SOURCE-INDEX 1
                                                  :TARGET-INDEX 1
                                                  :SOURCE-OUT-INDEX 1
                                                  :TARGET-IN-INDEX 1))
                                :IN-EDGES #(#S(EDGE
                                                 :ATTRIBUTE
#<SC-RELATION

{58635AED}>
                                                 :SOURCE-INDEX 0
                                                 :TARGET-INDEX 1
                                                 :SOURCE-OUT-INDEX 0
                                                 :TARGET-IN-INDEX 0)
                                            #S(EDGE
                                                 :ATTRIBUTE
#<SC-RELATION

{586378F5}>
                                                 :SOURCE-INDEX 1
                                                 :TARGET-INDEX 1
                                                 :SOURCE-OUT-INDEX 1
                                                 :TARGET-IN-INDEX 1)))
                           #S(NODE
                                :ATTRIBUTE #<SC-VARIABLE {58635E55}>
                                :ARRAY-INDEX 2
                                :OUT-EDGES #(#S(EDGE
                                                  :ATTRIBUTE
#<SC-RELATION

{58635EAD}>
                                                  :SOURCE-INDEX 2
                                                  :TARGET-INDEX 0
                                                  :SOURCE-OUT-INDEX 0
                                                  :TARGET-IN-INDEX 0))
                                :IN-EDGES #(#S(EDGE
                                                 :ATTRIBUTE
#<SC-RELATION

{58635AB5}>
                                                 :SOURCE-INDEX 0
                                                 :TARGET-INDEX 2
                                                 :SOURCE-OUT-INDEX 1
                                                 :TARGET-IN-INDEX 0))))
                  :GROWTH-SPEED NIL)
             SEARCH-GRAPH
             CORE->SEARCH
             :NODE-ATTR-COMPARATOR #<Function "DEFMACRO GENPROXY"
{58617B79}>
             :EDGE-ATTR-COMPARATOR #<Function "DEFMACRO GENPROXY"
{58617BB9}>)
            :FIND-MORE NIL))
        (RETURN-FROM LAMBDA-BLOCK NIL))
      T))

so - here is visible, that the whole graph as a s-exp is in the macro!!
so this is why it works, when graph does not have nodes in edge
structure but just numbers.

when I want to use my newer version, with source and target as nodes -
here it continues to expand the graph structure, instead that I would
like to stop it with something like #1=... reference like I can see
with *print-circle* t.

Karol

P.S.: sorry for my previous not clearly given question taking your
time. I should post the expansion as the first thing.
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <87br1rucut.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:
> now, genproxy expanded :
>
> GI> (genproxy ((0 :1 ((1) (2)))
> 	       (1 sc-variable ((0)))
> 	       (2 sc-constant ((0) (2)))))
>
> C-c C-m looks like this :
>
> #'(LAMBDA (SEARCH-GRAPH CORE->SEARCH)
>     (BLOCK LAMBDA-BLOCK
>       NIL
>       (WHEN
>       (ZEROP
>        (REMATCH
>         (MAKE-REMATCH-STATE
>          #S(GRAPH
>           :NODES #(#S(NODE
>                 :ATTRIBUTE #<SC-PROXY {58635A7D}>
>                 :ARRAY-INDEX 0
>                 :OUT-EDGES #(#S(EDGE
>                           :ATTRIBUTE #<SC-RELATION {58635AED}>
>                           :SOURCE-INDEX 0
>                           :TARGET-INDEX 1
>                           :SOURCE-OUT-INDEX 0
>                           :TARGET-IN-INDEX 0)
> [...]
>                   :GROWTH-SPEED NIL)
>              SEARCH-GRAPH
>              CORE->SEARCH
>              :NODE-ATTR-COMPARATOR #<Function "DEFMACRO GENPROXY" {58617B79}>
>              :EDGE-ATTR-COMPARATOR #<Function "DEFMACRO GENPROXY" {58617BB9}>)
>             :FIND-MORE NIL))
>         (RETURN-FROM LAMBDA-BLOCK NIL))
>       T))
>
> so - here is visible, that the whole graph as a s-exp is in the macro!!
> so this is why it works, when graph does not have nodes in edge
> structure but just numbers.

It is also visible that you've got no use for a macro.  

The problem you have is that the CL specification doesn't mandate the
compiler to cope with circular _sources_.

Just return a closure from a function:

(defun gen-proxy (...)
   (let ((proxy-graph (build-the-proxy-graph ...)))
     (lambda (search-graph core->search)
          ...  proxy-graph ...)))

(funcall (gen-proxy '((0 :1 ((1) (2)))
                      (1 sc-variable ((0)))
                      (2 sc-constant ((0) (2)))))
        search-graph core->search)


If you don't like this ', you can write in addition:

(defmacro genproxy ((&body sc-body)
                    &key (node-attr-comparator
                          #'(lambda (x y) (declare (ignore x y)) t))
                    (edge-attr-comparator
                     #'(lambda (x y) (declare (ignore x y)) t)))
   `(genproxy ',sc-body
          :node-attr-comparator ',node-attr-compartor                   
          :edge-attr-comparator ',edge-attr-compartor))


 
-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Until real software engineering is developed, the next best practice
is to develop with a dynamic system that has extreme late binding in
all aspects. The first system to really do this in an important way
is Lisp. -- Alan Kay
From: Karol Skocik
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129552328.037002.109290@f14g2000cwb.googlegroups.com>
Well, I don't know if it can be function. Can I build the body of
lambda returned by function genproxy piece by piece like defmacro?
The proxy graphs will be immutable when the program will normally run,
they are something like graph pattern. So I wanted to build a lambda,
which will have in body just as much code as necessary, and I wanted to
remove run-time checks (like - is the proxy connected or is it a bunch
of separate graphs, ...) which can be computed during compilation.
Because given any other proxy graph, the returned lambda should look
quite different.
In C++ (which was my main language) I would write a simple interpreter
just for this purpose (if I already get to this stage of development,
since I tried to write it in C++ before, and I failed).
In Lisp, I was thinking that macros would be perfect for this, and I
think they still are, I have just not realized that even the graph
itself is expanded in macro as a source. 

Cheers,
  Karol
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <87vezw9t6n.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:

> Well, I don't know if it can be function. Can I build the body of
> lambda returned by function genproxy piece by piece like defmacro?
> The proxy graphs will be immutable when the program will normally run,
> they are something like graph pattern. So I wanted to build a lambda,
> which will have in body just as much code as necessary, and I wanted to
> remove run-time checks (like - is the proxy connected or is it a bunch
> of separate graphs, ...) which can be computed during compilation.
> Because given any other proxy graph, the returned lambda should look
> quite different.
> In C++ (which was my main language) I would write a simple interpreter
> just for this purpose (if I already get to this stage of development,
> since I tried to write it in C++ before, and I failed).
> In Lisp, I was thinking that macros would be perfect for this, and I
> think they still are, I have just not realized that even the graph
> itself is expanded in macro as a source. 

Macros may be useful for this. But it will be easier to debug and
structure correctly your code with functions.  Then when you've got the
generating functions working correctly, you can put then in an
(eval-when (:compile-toplevel :load-toplevel :execute) ...) and write
a trivial macro that call them.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
In deep sleep hear sound,
Cat vomit hairball somewhere.
Will find in morning.
From: Kaz Kylheku
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129303308.350312.236860@f14g2000cwb.googlegroups.com>
Pascal Bourguignon wrote:
> I note that your macro returns a function:
>         `#'(lambda (search-graph core->search)
> this is not a s-expression, this is data. (functions are first class
> objects in lisp).

The macro returns a (FUNCTION ...) expression: note the backquote.
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <87k6ggt3ai.fsf@thalassa.informatimago.com>
"Kaz Kylheku" <········@gmail.com> writes:

> Pascal Bourguignon wrote:
>> I note that your macro returns a function:
>>         `#'(lambda (search-graph core->search)
>> this is not a s-expression, this is data. (functions are first class
>> objects in lisp).
>
> The macro returns a (FUNCTION ...) expression: note the backquote.

Yes, but when you evaluate a (FUNCTION ...) expression, nothing
happens.  The function self evaluates and that's it.

You could  as well  return this function  from a function,  there's no
need for a macro.

(defun gen-fun (args)
   (lambda (x) (do-something-with args :and x)))


You can even write:

(defun gen-fun (args)
   `(function (lambda (x) (do-something-with ',args :and x))))

(gen-fun 2) --> #'(LAMBDA (X) (DO-SOMETHING-WITH '2 :AND X))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
From: Kaz Kylheku
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <1129312474.069149.28790@z14g2000cwz.googlegroups.com>
Pascal Bourguignon wrote:
> "Kaz Kylheku" <········@gmail.com> writes:
>
> > Pascal Bourguignon wrote:
> >> I note that your macro returns a function:
> >>         `#'(lambda (search-graph core->search)
> >> this is not a s-expression, this is data. (functions are first class
> >> objects in lisp).
> >
> > The macro returns a (FUNCTION ...) expression: note the backquote.
>
> Yes, but when you evaluate a (FUNCTION ...) expression, nothing
> happens.  The function self evaluates and that's it.

Not true! FUNCTION is a massively powerful operator that makes a
lexical closure out of a LAMBDA expression.

Roast LAMBDA anyone? Rack of LAMBDA? LAMBDA souvlaki? :)

A simple list headed by LAMBDA goes in, a function object pops out.
That's hardly self-evaluation.

> You could  as well  return this function  from a function,  there's no
> need for a macro.
>
> (defun gen-fun (args)
>    (lambda (x) (do-something-with args :and x)))

No, you still need the macro if you want to customize the body of the
lambda based on parameters, rather than pass values. Those parameters
might be expressions that contain lexical references to the environment
that surrounds the macro call.

>
> You can even write:
>
> (defun gen-fun (args)
>    `(function (lambda (x) (do-something-with ',args :and x))))

But now you need an EVAL to make it do anything, and that will happen
in a null lexical environment, unless you do it by means of a macro.

Otherwise your ARGS expression won't be able to refer to local
variables.

> (gen-fun 2) --> #'(LAMBDA (X) (DO-SOMETHING-WITH '2 :AND X))

Now:

(defmacro gen-mac (&rest args) (apply #'gen-fun args))

add EVAL-WHEN around the DEFUN, and we are there. :)
From: Pascal Bourguignon
Subject: Re: Macro working on recursive structure expands indefinitely.
Date: 
Message-ID: <877jcfucnp.fsf@thalassa.informatimago.com>
"Kaz Kylheku" <········@gmail.com> writes:

> Pascal Bourguignon wrote:
>> "Kaz Kylheku" <········@gmail.com> writes:
>>
>> > Pascal Bourguignon wrote:
>> >> I note that your macro returns a function:
>> >>         `#'(lambda (search-graph core->search)
>> >> this is not a s-expression, this is data. (functions are first class
>> >> objects in lisp).
>> >
>> > The macro returns a (FUNCTION ...) expression: note the backquote.
>>
>> Yes, but when you evaluate a (FUNCTION ...) expression, nothing
>> happens.  The function self evaluates and that's it.
>
> Not true! FUNCTION is a massively powerful operator that makes a
> lexical closure out of a LAMBDA expression.
>
> Roast LAMBDA anyone? Rack of LAMBDA? LAMBDA souvlaki? :)
>
> A simple list headed by LAMBDA goes in, a function object pops out.
> That's hardly self-evaluation.
>
>> You could  as well  return this function  from a function,  there's no
>> need for a macro.
>>
>> (defun gen-fun (args)
>>    (lambda (x) (do-something-with args :and x)))
>
> No, you still need the macro if you want to customize the body of the
> lambda based on parameters, rather than pass values. Those parameters
> might be expressions that contain lexical references to the environment
> that surrounds the macro call.
>
>>
>> You can even write:
>>
>> (defun gen-fun (args)
>>    `(function (lambda (x) (do-something-with ',args :and x))))
>
> But now you need an EVAL to make it do anything, and that will happen
> in a null lexical environment, unless you do it by means of a macro.
>
> Otherwise your ARGS expression won't be able to refer to local
> variables.

You can, with closures.


You're right but mostly these consideration should come later, when
you add the syntactic sugar.  First make it work with functions and
closures, then add the macros to make it easy to write and to get
"compile-time" effects.

-- 
"Klingon function calls do not have "parameters" -- they have
"arguments" and they ALWAYS WIN THEM."