Oh, and just to tie this thread to another debate that is raging over somewhere
else on cyberspace.
Ponder the relationship between call/cc and nondeterministic Lisp (i.e. McCarthy's
amb, Screamer). Then ponder the relationship between nondeterministic processes
and stochastic processes (see, e.g. Koller, Pfeffer, and McAllester). Then ponder
the relationship between Markov processes and proper tail recursion. If you are
looking for a better term than `proper tail recursion', maybe `Markovean' will do.
····@purdue.edu (Jeffrey Mark Siskind) writes, Re: No ordinary Calculus book:
> Oh, and just to tie this thread to another debate
> that is raging over somewhere else on cyberspace.
>
> Ponder the relationship between call/cc and
> nondeterministic Lisp (i.e. McCarthy's amb, Screamer).
> Then ponder the relationship between nondeterministic processes
> and stochastic processes (see, e.g. Koller, Pfeffer, and McAllester).
> Then ponder the relationship between Markov processes and proper tail
> recursion.
> If you are looking for a better term than `proper tail recursion',
> maybe `Markovean' will do.
Can you expand on the analogy, for those not familiar with all these fields?
When you transform the world in CPS, Proper tail recursion is about dropping
a certain class of useless frames/variables from the conceptual call heap
(though in the implementation, this heap may be represented with a stack).
In non-deterministic Lisp, the frames (bits of execution)
are entered many times, as time rolls back with a slightly different choice.
Thus, call/cc is used to walk the tree of possible evaluations
in a non-deterministic world.
I'm already not sure what PTR means in this context.
Then stochastic processes are non-deterministic processes where events
have some density of probability with time. And among them, Markovian
processes are such that the probability density law for future events
does not depend on past events.
There I'm even less sure what PTR means, and how you match them with
Markovian processes.
[ Fran�ois-Ren� �VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[ TUNES project for a Free Reflective Computing System | http://tunes.org ]
Paralysis through analysis.
Francois-Rene Rideau <····@tunes.org> wrote in message news:<·················@Samaris.tunes.org>...
> ····@purdue.edu (Jeffrey Mark Siskind) writes, Re: No ordinary Calculus book:
> Then stochastic processes are non-deterministic processes where events
> have some density of probability with time. And among them, Markovian
> processes are such that the probability density law for future events
> does not depend on past events.
> There I'm even less sure what PTR means, and how you match them with
> Markovian processes.
A Markovian process is essentially a pure functional program, as there
is no state from past events that affects the future. So you can
consider a Markovian process as a tail recursive program as you don't
need to keep any state on the stack.
I'm absolutely gob-smacked to see this come up on comp.lang.scheme as
I've literally just started writing up some notes on applying PL
theory ideas to reinforcement learning problems. The dominant
paradigm in reinforcement learning is the Markov Decision Process
(MDP). The expressive power of MDPs is limited by being, well,
Markovian, and I think there are tricks you can apply from PL theory
to increase their expressive power while still retaining the Markov
property. If any one has any pointers to research on this I'd really
like to see them. My thinking starts from
@InProceedings{parr98,
author = {Ronald Parr and Stuart Russell},
title = {Reinforcement Learning with Hierarchies of
Machines},
booktitle = {Advances in Neural Information Processing Systems
10},
year = 1998,
editor = {M. I. Jordan and M. J. Kearnsa and S. A. Solla},
publisher = {MIT Press}
}
http://citeseer.nj.nec.com/12917.html
····@purdue.edu (Jeffrey Mark Siskind) writes:
> ponder
> the relationship between Markov processes and proper tail recursion. If you are
> looking for a better term than `proper tail recursion', maybe `Markovean' will do.
Sound of head exploding...
david rush
--
(who has been spending a lot of time trying to wrap his head around
the relationship between Markov processes and Bayesian networks...)