From: Matthew D Swank
Subject: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.05.16.42.44.928003@c.net>
The presence and use of call/cc has always been a subject of healthy
debate between the Scheme and Common Lisp communities. 

Just to give one example, in
http://www.nhplace.com/kent/PFAQ/unwind-protect-vs-continuations.html Kent
Pitman outlines the issues (at least with respect to unwind-protect) in
supporting continuations in Common Lisp.  In addition, he references
Will Clinger's and Dorai Sitaram's work in response.

However, it is my impression that continuation support is the most common
Scheme feature that Common Lisp users end up wanting to implement. 

Would delimited continuations be a better fit for Common Lisp?

Matt
-- 
"You do not really understand something unless you
 can explain it to your grandmother." — Albert Einstein.

From: Pascal Costanza
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <5a3s2fF2n2afuU1@mid.individual.net>
Matthew D Swank wrote:
> The presence and use of call/cc has always been a subject of healthy
> debate between the Scheme and Common Lisp communities. 
> 
> Just to give one example, in
> http://www.nhplace.com/kent/PFAQ/unwind-protect-vs-continuations.html Kent
> Pitman outlines the issues (at least with respect to unwind-protect) in
> supporting continuations in Common Lisp.  In addition, he references
> Will Clinger's and Dorai Sitaram's work in response.
> 
> However, it is my impression that continuation support is the most common
> Scheme feature that Common Lisp users end up wanting to implement. 

I think you have the wrong impression here. See for example 
http://groups.google.com/group/comp.lang.lisp/msg/5f92f1356e2b3523

Of course, it could just be that those who want full continuations just 
switch to other languages, most probably Scheme, but that's very 
speculative. I also have the impression that some Schemers talk about 
advantages of continuations when they actually "just" have simple uses 
in mind which are already available in Common Lisp in the form of 
block/return-from and catch/throw.

> Would delimited continuations be a better fit for Common Lisp?

Maybe. One advantage of delimited continuations is that you can treat 
them as plain functions. Another advantage is that they can be 
relatively easily added on top of Common Lisp while full continuations 
can't.

However, they don't get rid of some of the issues that continuations 
have, as far as I can tell. A problem with continuations is that they 
may expose implementation details of library functions that are 
otherwise hidden by such a library. See for example 
http://www.r6rs.org/formal-comments/comment-36.txt

The problem here is that two different kinds of effects (side effects 
and control effects) don't mesh very well.

My current take on this is that continuations are indeed useful but that 
call/cc is probably the wrong abstraction. I suspect that shifting 
continuations to a meta-level API would be more adequate, like in 
3-Lisp. But that's just a guess...

Apparently, the goal of the ANSI CL committee was to not forbid a CL 
implementation to use something like call/cc in the background, but to 
also not require implementations to provide first-class continuations. 
Very theoretically, this leaves the door open for other designs.


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Ray Dillinger
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <463cc6ed$0$27249$742ec2ed@news.sonic.net>
For what it's worth, I kinda like the schemish unlimited
continuations.  The ability to abstract flow of control
is very cool and in principle I'm all for unlimited
expressiveness.

But that's also the ability to subvert flow of control,
and I've come to dread actually using call/cc because the
bugs it generates can be hard to understand and fix.  It's
a lot like "Goto" in that way.

Call/cc can unexpectedly jump the flow of control out of
the middle of something that takes a function as an
argument and calls it, but depends for its correctness on
flow of control returning from that function.  My code
that implements trees using cons cells, for example, can
silently leave a branch of the tree unlinked, losing data,
if the ordering predicate unexpectedly jumps away.  And I
don't think it's a bug in my tree library; I just think
it's a type error to use that library with an ordering
function of the wrong type.

But my problem is that there's no way to catch such a type
error without crashing. If I could put asserts or type
declarations on my tree-insert! routine that would stop it
instantly rather than losing data if it got a non-returning
function for an ordering predicate, I would.  But there's
not.

This is the sort of thing that makes me want to invoke
something like static type checking; functions that may
transfer control to some other part of the program rather
than returning, are fundamentally unlike functions that
will definitely return, and there needs to be some way
to tell the difference *before* calling them, or to
restrict the unintended type of functions from being used
as arguments.


				Bear
From: Russell McManus
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <87vef5zxll.fsf@thelonious.cl-user.org>
Ray Dillinger <····@sonic.net> writes:

> But my problem is that there's no way to catch such a type
> error without crashing. If I could put asserts or type
> declarations on my tree-insert! routine that would stop it
> instantly rather than losing data if it got a non-returning
> function for an ordering predicate, I would.  But there's
> not.

Can't dynamic-wind do what you want in this case?

-russ
From: Ray Dillinger
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <46460290$0$27226$742ec2ed@news.sonic.net>
Russell McManus wrote:
> Ray Dillinger <····@sonic.net> writes:
> 
> 
>>But my problem is that there's no way to catch such a type
>>error without crashing. If I could put asserts or type
>>declarations on my tree-insert! routine that would stop it
>>instantly rather than losing data if it got a non-returning
>>function for an ordering predicate, I would.  But there's
>>not.
> 
> 
> Can't dynamic-wind do what you want in this case?
> 
> -russ


Can Dynamic-wind figure out exactly in what places and in what
ways escaping continuations can screw the logic and internal
consistency of my code that doesn't expect them, and install
guarding windings only where they're needed?  They bring about
flow-of-control mysteries in the code being escaped from, as
badly as Intercal's come-from command!

My objection, I guess, is not really about there being no
way to do it; as you point out, dynamic-wind provides a way.
It's about the cognitive burden of figuring out what and how
much you have to protect.  I object to having to be
responsible for protecting perfectly reasonable code from
random jumps in and out.

Passing escaping continuations to someone else's code that
expects any other kind of argument is, IMO, a perverse way
of subverting the flow of control of that code, and in my
estimation requiring the author of that code to take into
account such malicious perversity in order to write correct
code is extraordinarily rude.

The power of escaping continuations to subvert code in
unexpected ways is something that other code should not
be subject to by default.  Protection of other code from
subversion by escaping continuations should be enabled by
default with a means to opt-out, not another task for
the programmer.

(I think my opt-out strategy can be realized with
define-syntax, dynamic-wind, and an adept redefinition
of 'lambda' - but I'm not feeling that masochistic
today....)

				Bear
From: Matthew D Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.05.17.48.42.960336@c.net>
On Sat, 05 May 2007 19:09:35 +0200, Pascal Costanza wrote:

> I think you have the wrong impression here. See for example 
> http://groups.google.com/group/comp.lang.lisp/msg/5f92f1356e2b3523
> 
> Of course, it could just be that those who want full continuations just 
> switch to other languages, most probably Scheme, but that's very 
> speculative. I also have the impression that some Schemers talk about 
> advantages of continuations when they actually "just" have simple uses 
> in mind which are already available in Common Lisp in the form of 
> block/return-from and catch/throw.

Well, the existence of projects like Arnesi seem to indicate
there is some demand.  However, the problem with mini(or maxi in this
case) languages like with-call/cc is that they end up being second class
citizens compared to compiler supported facilities like catch and throw.

Matt

-- 
"You do not really understand something unless you can
 explain it to your grandmother." — Albert Einstein.
From: Pascal Costanza
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <5a40lmF2mmb95U1@mid.individual.net>
Matthew D Swank wrote:
> On Sat, 05 May 2007 19:09:35 +0200, Pascal Costanza wrote:
> 
>> I think you have the wrong impression here. See for example 
>> http://groups.google.com/group/comp.lang.lisp/msg/5f92f1356e2b3523
>>
>> Of course, it could just be that those who want full continuations just 
>> switch to other languages, most probably Scheme, but that's very 
>> speculative. I also have the impression that some Schemers talk about 
>> advantages of continuations when they actually "just" have simple uses 
>> in mind which are already available in Common Lisp in the form of 
>> block/return-from and catch/throw.
> 
> Well, the existence of projects like Arnesi seem to indicate
> there is some demand.  However, the problem with mini(or maxi in this
> case) languages like with-call/cc is that they end up being second class
> citizens compared to compiler supported facilities like catch and throw.

IIUC, Arnesi is a library that has been developed for use in a 
continuation-based web application framework. The notion of 
continuation-based web applications is a very recent idea, and I 
seriously think that it is only a passing fad. AJAX, Flash, Silverlight, 
Vistascript, HOP, what have you, will take over. Who cares about the 
back button. ;)

I could be wrong, of course.

Anyway, delimited continuations are good enough for such applications. 
If this idea sticks, someone will probably integrate them in some CL 
implementation sooner or later...


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.05.19.24.25.349547@c.net>
On Sat, 05 May 2007 19:09:35 +0200, Pascal Costanza wrote:

> I also have the impression that some Schemers talk about 
> advantages of continuations when they actually "just" have simple uses 
> in mind which are already available in Common Lisp in the form of 
> block/return-from and catch/throw.

I would like to get away from the "you don't really want continuations"
thread of argument.  I am familiar with this point view.  However, my
uses aren't simple, and I am interested the kind things re-entrant
continautions facilitate. 

What I hope to do is provoke a discussion of the implementation burden and
feasibility of delimited continuations compared to the whole enchilada
that Scheme provides.

Matt

-- 
"You do not really understand something unless you can
 explain it to your grandmother." - Albert Einstein.
From: Pascal Costanza
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <5a448tF2njo9pU1@mid.individual.net>
Matthew Swank wrote:
> On Sat, 05 May 2007 19:09:35 +0200, Pascal Costanza wrote:
> 
>> I also have the impression that some Schemers talk about 
>> advantages of continuations when they actually "just" have simple uses 
>> in mind which are already available in Common Lisp in the form of 
>> block/return-from and catch/throw.
> 
> I would like to get away from the "you don't really want continuations"
> thread of argument.  I am familiar with this point view.  However, my
> uses aren't simple, and I am interested the kind things re-entrant
> continautions facilitate. 

OK

> What I hope to do is provoke a discussion of the implementation burden and
> feasibility of delimited continuations compared to the whole enchilada
> that Scheme provides.

I don't know, but maybe 
http://library.readscheme.org/servlets/cite.ss?pattern=sperber-icfp2002 
is a good starting point.


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.06.02.03.59.679668@c.net>
On Sat, 05 May 2007 21:29:33 +0200, Pascal Costanza wrote:

>> What I hope to do is provoke a discussion of the implementation burden and
>> feasibility of delimited continuations compared to the whole enchilada
>> that Scheme provides.
> 
> I don't know, but maybe 
> http://library.readscheme.org/servlets/cite.ss?pattern=sperber-icfp2002 
> is a good starting point.

This is good paper.  I've been reading "Abstacting Control" by Olivier
Danvy and Andrzej Filinski
(http://citeseer.ist.psu.edu/danvy90abstracting.html).
This dovetails nicely with that.  

Skimming "Final Shift for Call/cc" seems to confirm that the
precision of shift/reset can yield better performance and
memory consumption in a applications where call/cc might normally be used.


Also, though I am relatively new to shift/reset, the code I have written
with it seems easier to reason about.


Matt

-- 
"You do not really understand something unless you
 can explain it to your grandmother." - Albert Einstein.
From: Neelakantan Krishnaswami
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <slrnf3pumi.imi.neelk@gs3106.sp.cs.cmu.edu>
In article <······························@c.net>, Matthew Swank wrote:
> 
> What I hope to do is provoke a discussion of the implementation
> burden and feasibility of delimited continuations compared to the
> whole enchilada that Scheme provides.

IIUC, if you have references in your language, then delimited control
operators (eg, shift and reset) and full call/cc are equivalent. You
can implement shift/reset in terms of call/cc + set!, and vice
versa. So the overall implementation burden can't fundamentally
differ.

However, from the user pov, delimited control might be a better option
as the basic primitive, because you may get finer control over memory
usage by grabbing just the subcontinuation you're interested in. (I'm
imagining needless live pointers to large objects living in the
control context of a full continuation, that a delimited continuation
might not grab.)


-- 
Neel R. Krishnaswami
·····@cs.cmu.edu
From: ·············@hotmail.com
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <1178527111.653250.320560@p77g2000hsh.googlegroups.com>
On May 5, 6:42 pm, Matthew D Swank <akopa-is-very-much-like-my-mail-
·······@c.net> wrote:
> The presence and use of call/cc has always been a subject of healthy
> debate between the Scheme and Common Lisp communities.
>
> Just to give one example, inhttp://www.nhplace.com/kent/PFAQ/unwind-protect-vs-continuations.htmlKent
> Pitman outlines the issues (at least with respect to unwind-protect) in
> supporting continuations in Common Lisp.  In addition, he references
> Will Clinger's and Dorai Sitaram's work in response.
>
> However, it is my impression that continuation support is the most common
> Scheme feature that Common Lisp users end up wanting to implement.
>
> Would delimited continuations be a better fit for Common Lisp?
>
> Matt
> --
> "You do not really understand something unless you
>  can explain it to your grandmother." - Albert Einstein.

Do you personally need continuations support?I  do not. That doesn't
mean that I'm right and you're wrong but that I could live without it
in my projects. For your uses I suggest you to see first On Lisp
http://www.paulgraham.com/onlisp.html chapter 20 it might do the
trick.
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.07.18.37.22.945583@c.net>
On Mon, 07 May 2007 01:38:31 -0700, antoanjamison wrote:


> Do you personally need continuations support?I  do not. That doesn't
> mean that I'm right and you're wrong but that I could live without it
> in my projects. For your uses I suggest you to see first On Lisp
> http://www.paulgraham.com/onlisp.html chapter 20 it might do the
> trick.

I've used continuations in various things more times than I can count.
The problem is making things efficient and complete, which the hacks in On
Lisp don't really address.  A lot of my projects remain toys partly
because having to write a custom compiler for Common Lisp is a lot of work*.

Matt

*though the code walker in Arnesi is a nice tool for that.
-- 
"You do not really understand something unless you
 can explain it to your grandmother." - Albert Einstein.
From: Pebblestone
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <1178670223.886142.179820@y80g2000hsf.googlegroups.com>
>
> I've used continuations in various things more times than I can count.
> The problem is making things efficient and complete, which the hacks in On
> Lisp don't really address.  A lot of my projects remain toys partly
> because having to write a custom compiler for Common Lisp is a lot of work*.

I can't agree with you more. I've been missing delimited continuations
for long and I really appreciate if someone implements it as an
extension in SBCL (please! :-). I don't care if it's portable or not.


Jianshi
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.09.02.46.03.155100@c.net>
On Tue, 08 May 2007 17:23:43 -0700, Pebblestone wrote:

>>
>> I've used continuations in various things more times than I can count.
>> The problem is making things efficient and complete, which the hacks in On
>> Lisp don't really address.  A lot of my projects remain toys partly
>> because having to write a custom compiler for Common Lisp is a lot of work*.
> 
> I can't agree with you more. I've been missing delimited continuations
> for long and I really appreciate if someone implements it as an
> extension in SBCL (please! :-). I don't care if it's portable or not.
> 
> 
> Jianshi


Well, even I am willing to admit that might take a while.  Though
delimitied continuations have an area of active research for almost 20
years, they are still not well known outside the PL community.  

I'm hoping to genertate enough interest (or even just awareness) in the
Lisp community that someone more talented that I am will have a go at an
expermental implementation:).  SBCL would be ideal, but CLisp might be
easier.


Matt
-- 
"You do not really understand something unless you can
 explain it to your grandmother." - Albert Einstein.
From: Kent M Pitman
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <ulkfyvjy4.fsf@nhplace.com>
[ Replying to comp.lang.lisp only.
  http://www.nhplace.com/kent/PFAQ/cross-posting.html ]

Pebblestone <··········@gmail.com> writes:

> > I've used continuations in various things more times than I can
> > count.  The problem is making things efficient and complete, which
> > the hacks in On Lisp don't really address.  A lot of my projects
> > remain toys partly because having to write a custom compiler for
> > Common Lisp is a lot of work*.
> 
> I can't agree with you more. I've been missing delimited
> continuations for long and I really appreciate if someone implements
> it as an extension in SBCL (please! :-). I don't care if it's
> portable or not.

Can you sketch an example of how you'd like to use them, just for
discussion purposes?
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.09.06.25.35.962868@c.net>
On Tue, 08 May 2007 21:56:19 -0400, Kent M Pitman wrote:

> [ Replying to comp.lang.lisp only.
>   http://www.nhplace.com/kent/PFAQ/cross-posting.html ]
> 
> Pebblestone <··········@gmail.com> writes:
> 
>> > I've used continuations in various things more times than I can
>> > count.  The problem is making things efficient and complete, which
>> > the hacks in On Lisp don't really address.  A lot of my projects
>> > remain toys partly because having to write a custom compiler for
>> > Common Lisp is a lot of work*.
>> 
>> I can't agree with you more. I've been missing delimited
>> continuations for long and I really appreciate if someone implements
>> it as an extension in SBCL (please! :-). I don't care if it's
>> portable or not.
> 
> Can you sketch an example of how you'd like to use them, just for
> discussion purposes?

I welcome Jianshi to submit an example of his own.  However, I would like
to get the ball rolling with something simple.

Shift/reset makes generators almost comically simple:

(define (list-gen lst)
    (let ((gen (reset (map (lambda (item)
                             (shift k (cons item k)))
                           lst)
                      ;;end flag
                      (shift k #f))))
      (lambda ()
        (if gen
          (let ((current gen))
            (set! gen ((cdr current) #f))
            (car current))
          #f))))

(define foo (list-gen '(1 2 3)))

(foo)
-> 1

(foo)
-> 2

(foo)
-> 3

(foo)
-> #f

Since we're not doing anything exotic with k*, the reset/shift basically
acts like a block/return+ where we happen to be able to re-enter the
delimited computation through k! *Nothing* after reset is captured.

Matt

*like calling k inside the shift
+the reset block that shift throws to is determined lexically

-- 
"You do not really understand something unless you can
 explain it to your grandmother." - Albert Einstein.
From: Kent M Pitman
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <uvef21npd.fsf@nhplace.com>
Matthew Swank <································@c.net> writes:

> Shift/reset makes generators almost comically simple:

I think I see what the following is doing and how, but for our listening
audience, do you want to explain the subtleties of shift and reset, 
or has that been done earlier in the thread?  It's looking to me like reset
sets up a context of some kind and shift is binding a continuation k to
something that computes a next value for the closure returned by reset, 
creating some sort of co-routine call and with a co-return that allows
you to exit piecewise.  I can imagine this being useful for some kinds of
things (including comparing fringes of differently shaped trees) but I think
your example is overly simple to make your point.  (Plus the whole idiomatic
use of a cons and the icky set! stuff going on in the body is crying out for
some hiding in a better macro, though maybe you unfolded it from a macro
to make a point about the need for the underlying operator.)

> (define (list-gen lst)
>     (let ((gen (reset (map (lambda (item)
>                              (shift k (cons item k)))
>                            lst)
>                       ;;end flag
>                       (shift k #f))))
>       (lambda ()
>         (if gen
>           (let ((current gen))
>             (set! gen ((cdr current) #f))
>             (car current))
>           #f))))
> 
> (define foo (list-gen '(1 2 3)))
> 
> (foo)
> -> 1
> 
> (foo)
> -> 2
> 
> (foo)
> -> 3
> 
> (foo)
> -> #f
> 
> Since we're not doing anything exotic with k*, the reset/shift basically
> acts like a block/return+ where we happen to be able to re-enter the
> delimited computation through k! *Nothing* after reset is captured.

I'm willing to believe there's a case that is more compelling, but
isn't the above written better in CL without continuations as just:

   (defun list-gen (list)
     #'(lambda () (pop list)))

Where you would call it, 

 (setq foo (list-gen '(1 2 3)))
 => #<closure>

 (funcall foo)
 => 1

 (funcall foo)
 => 2

 (funcall foo)
 => 3

 (funcall foo)
 => NIL

I bet you can make something not so trivially expressed, but without
that thing to talk about, it's hard to give this discussion real teeth
because it's hard to talk abot just how often it comes up and what
people are forced into doing instead...
From: Pascal Costanza
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <5adcg1F2meo6mU1@mid.individual.net>
Kent M Pitman wrote:
> Matthew Swank <································@c.net> writes:
> 
>> Shift/reset makes generators almost comically simple:
> 
> I think I see what the following is doing and how, but for our listening
> audience, do you want to explain the subtleties of shift and reset, 
> or has that been done earlier in the thread? 

FWIW, I found http://paste.lisp.org/display/26252 quite useful for 
understanding shift/reset.


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.10.07.24.17.497212@c.net>
On Wed, 09 May 2007 03:05:34 -0400, Kent M Pitman wrote:

> I bet you can make something not so trivially expressed, but without
> that thing to talk about, it's hard to give this discussion real teeth
> because it's hard to talk abot just how often it comes up and what
> people are forced into doing instead...

Wow, thanks for the thoughtful reply.  I started working on a worthy
rebuttal when I realized I should probably put this off until after final
exams, so watch this space.  

I'll just leave with CPS versions the reset and shift operators (no
macros).

(defun shift* (k entry)
  ;; The default behavior of shift is to throw to
  ;; reset by discarding k; only by calling the second
  ;; argument to entry can we take that continuation.
  (funcall entry 
           #'identity 
           (lambda (k1 val)
             (funcall k1 (funcall k val)))))

(defun reset* (k thunk)
  ;; This is a barrier;
  ;; no captured continuation can "see" k
  (funcall k (funcall thunk #'identity)))

;; and function to play with

(defun cps-mapcar (k fun list)
  (if (atom list)
      (funcall k nil)
      (funcall fun 
               (lambda (item)
                 (cps-mapcar (lambda (rest)
                               (funcall k (cons item rest)))
                             fun
                             (cdr list)))
               (car list))))

;; try 
(reset* #'identity
        (lambda (k)
          (cps-mapcar k
                      (lambda (k item)
                        (shift* k (lambda (k f) 
                                    (funcall k (cons item f)))))
                      '(1 2 3))))
;; and
(funcall (cdr *) #'identity nil)
;; successively

Matt
-- 
"You do not really understand something unless you can
 explain it to your grandmother." - Albert Einstein.
From: Matthew Swank
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <pan.2007.05.13.08.25.41.942258@c.net>
On Wed, 09 May 2007 03:05:34 -0400, Kent M Pitman wrote:

> Matthew Swank <································@c.net> writes:
> 
>> Shift/reset makes generators almost comically simple:
> 
> I can imagine this being useful for some kinds of
> things (including comparing fringes of differently shaped trees)
...
>  
> I bet you can make something not so trivially expressed, but without
> that thing to talk about, it's hard to give this discussion real teeth
> because it's hard to talk abot just how often it comes up and what
> people are forced into doing instead...


O.k., part of the motivation in talking about coroutine-like objects like
generators is that they are a useful feature in more main-stream languages
like C# and Python.  However, to implement them cleanly in Common Lisp
requires a code walker:
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/b11f9094a642fd94/6cd352d407787d15
to capture relevant continuation information.

So for the purposes of this discussion I will move away from delimited
continuations generally and talk about this language feature.

I have written a small generator facility which has shown up here from
time to time in various forms that allows the user to re-enter a running
computation using a fairly restricted subset of lisp. The code's not
pretty or efficient, but does include some examples of what generators can
do:

http://paste.lisp.org/display/29695#1

for example:

(same-fringe '(3 (2 (1 NIL)) 5 (4 NIL) 6 NIL)
             '(3 (2 (1 NIL)) 5 (4 NIL) 6 NIL))
==> T

(same-fringe '(3 (2 (0 NIL)) 5 (4 NIL) 6 NIL)
             '(3 (2 (1 NIL)) 5 (4 NIL) 6 NIL))
==> NIL

(same-fringe '(3 (2 (1 NIL)) 5 (4 NIL) 6 NIL)
             '(0 (2 (1 NIL)) 5 (4 NIL) 6 NIL))

==> T

One thing to notice about generators in Python and (iterators in) C# is
that they are basically compiler hacks them-selves: sub-languages that
limit the amount of runtime stack that has to be saved in exchange for
expressive power. However I think it should be possible to do better than
that in lisp.  It irks me that I have to work around things like
coroutines rather than have safe, efficient compiler support for them.  

The two common workarounds to lack of co-routine support are native
threads and lazy-lists.  Neither are standard. Home rolled lazy-list
usually have comparable performance problems to home-rolled continuations,
and unless i/o is involved, threads seem like a bulldozer answer to a
garden spade problem.

So there it is.  In addition to the intellectual fascination I have the
continuations, there are pragmatic uses that sometimes make me jealous of
other languages.  

I purposely narrowed talking about continuations to talking about
generators to point out a lack in the Common Lisp feature set that could
be remedied with delimited continuation support.  However, this could also
be seen as a feature request for generators.  

If Common Lisp doesn't get continuations, what would a generators in
Common Lisp look like?  Would it be desirable to have a coroutine
like syntax that made it easy to implement an extensible stream or
sequence protocol?


Matt
-- 
"You do not really understand something unless you
 can explain it to your grandmother." - Albert Einstein.
From: Chas Emerick
Subject: Re: Delimited continuations in (Common) Lisp
Date: 
Message-ID: <1179064419.463883.158800@u30g2000hsc.googlegroups.com>
On May 8, 9:56 pm, Kent M Pitman <······@nhplace.com> wrote:
> [ Replying to comp.lang.lisp only.
>  http://www.nhplace.com/kent/PFAQ/cross-posting.html]
>
> Pebblestone <··········@gmail.com> writes:
> > > I've used continuations in various things more times than I can
> > > count.  The problem is making things efficient and complete, which
> > > the hacks in On Lisp don't really address.  A lot of my projects
> > > remain toys partly because having to write a custom compiler for
> > > Common Lisp is a lot of work*.
>
> > I can't agree with you more. I've been missing delimited
> > continuations for long and I really appreciate if someone implements
> > it as an extension in SBCL (please! :-). I don't care if it's
> > portable or not.
>
> Can you sketch an example of how you'd like to use them, just for
> discussion purposes?

First, I'd like to second Matthew Swank's desire to have proper
generators -- they're a marvelous idiom for many different kinds of
problems, and it's really one of the few language features that I find
myself wanting when using Lisp.

At the moment, I could really use continuations to simplify the
implementation of certain kinds of complex backtracking algorithms (in
my case at the moment, the optimization of a result of what you can
think of as a document processing tree, which collapses into a
pipeline for any particular set of "parameter" choices).  There are
lots of clever alternative approaches, some of which I use now, but
continuations fit the problem best, at least in my head.  The key
advantage is that the use of continuations would greatly simplify the
domain-specific code, which needs to be far too aware of the
backtracking machinery (at least in its current manifestation).

Chas Emerick
Snowtide Informatics Systems, Inc.
········@snowtide.com