Greetings,
I have been working on what is essentially my first Lisp app for the
last 6+ months. Its purpose is to manage high-level AI and gameplay
for our MMORPG. I'm going to describe the architecture so far, and
then I hope to get some feedback on it and discussion of the questions
I'll ask afterward.
At first we started with LISA, and I wrote about seven thousand lines
worth of rules. It worked OK, but it failed to scale to even a small
fraction of what we wanted it to do. After investigating hacking up
LISA and other alternatives, my coworker Andy Sloane wrote a
(non-RETE-like) workalike in a week (most of it in the first day; he's
rather brilliant). We haven't run into any performance issues since
then. The only LISA feature he skipped over that we were using is OR
in the LHS (essentially splits the rule for each clause). So, I only
had to change maybe a dozen lines. Quite a testament to the power of
Lisp macros!
Not long after this, I saw a crisis of maintainability on the horizon
if I kept writing everything as rules. Since then I've implemented an
embedded language for describing Objectives, and one for defining
Missions in terms of the objectives and other things. I'm quite proud
of it. Objective definitions expand into something like what I was
doing by hand with rules (including one or more generated rules for
each objective), but which take closures to call in their
right-hand-sides.
Twenty lines of linear, readable mission-definition is translated into
a set of closures (one for each mission stage), and a couple fact
assertions. The stage-closures call functions generated by the
objective macros and pass callbacks containing the 'meat' of the
mission definition (what to tell or give the mission takers). The
objectives and missions also share some code to handle such things as
pre-mission group-formation, time-outs, and mechanisms for generating
dialog, and determining difficulty/importance. Additionally, mission
instantiation informs a system for starting dependent missions if an
arbitrary expression (which has access to that mission's data) returns
true. One of the key features is that these missions are taken by
heterogeneous groups of players and NPCs. My hope is to integrate a
planning agent framework so that NPCs can learn to complete missions
without explicit instructions (which I'm currently writing as rules).
I also have a set of macros for defining persistent (in theory) data
structures, which help somewhat with managing the complexity of all the
data I need to track that doesn't fit neatly in the rule system and
requires global accessibility. This is perhaps the part of the program
I'm most displeased by. What I'd really like, I think, is to replace
it and a lot of the rule-based code with something like Cells (which I
recently found and haven't tried out yet). I have been envisioning
something along those lines since I first read about Xanadu and zigzag
years ago, but I could never figure out how to organize or index so
much data in a usefully scalable way.
So, enough background... I'd like to get feedback from the real Lisp
hackers about my architecture direction. I'd also like some advice
about how to approach using something like Cells, and/or
Elephant/AllegroCache (ie is Cells anywhere near production ready?).
AllegroCache really looks like it could work well in our environment
(where retrieving an object should also recreate it in the
game-universe and vice versa), and might even do what I want Cells to
do (based on my interpretation of the 8.0 features page), but I'd like
to stick with F/OSS if possible.
~Michael Warnock
From: Kenny Tilton
Subject: Re: Refining the arch. of my MMORPG high-level-AI/gameplay service?
Date:
Message-ID: <43CAC255.2080502@nyc.rr.com>
···············@gmail.com wrote:
> Greetings,
>
> I have been working on what is essentially my first Lisp app for the
> last 6+ months. Its purpose is to manage high-level AI and gameplay
> for our MMORPG. I'm going to describe the architecture so far, and
> then I hope to get some feedback on it and discussion of the questions
> I'll ask afterward.
>
> At first we started with LISA, and I wrote about seven thousand lines
> worth of rules. It worked OK, but it failed to scale to even a small
> fraction of what we wanted it to do. After investigating hacking up
> LISA and other alternatives, my coworker Andy Sloane wrote a
> (non-RETE-like) workalike in a week (most of it in the first day; he's
> rather brilliant). We haven't run into any performance issues since
> then. The only LISA feature he skipped over that we were using is OR
> in the LHS (essentially splits the rule for each clause). So, I only
> had to change maybe a dozen lines. Quite a testament to the power of
> Lisp macros!
I am sure Andy is brilliant, but as you seem to realize, Lisp gets a lot
of credit, too. Programming is always pulling teeth, even with Lisp, but
sometimes we get inconceivable amounts of work done in a day.
I think that is part of why libs are not so good in Lisp, everyone is
out there just rolling their own. eg, Next time you will not start with
something like Lisa, you will go straight to the homebrew.
>
> Not long after this, I saw a crisis of maintainability on the horizon
> if I kept writing everything as rules. Since then I've implemented an
> embedded language for describing Objectives, and one for defining
> Missions in terms of the objectives and other things. I'm quite proud
> of it. Objective definitions expand into something like what I was
> doing by hand with rules (including one or more generated rules for
> each objective), but which take closures to call in their
> right-hand-sides.
>
> Twenty lines of linear, readable mission-definition is translated into
> a set of closures (one for each mission stage), and a couple fact
> assertions. The stage-closures call functions generated by the
> objective macros and pass callbacks containing the 'meat' of the
> mission definition (what to tell or give the mission takers). The
> objectives and missions also share some code to handle such things as
> pre-mission group-formation, time-outs, and mechanisms for generating
> dialog, and determining difficulty/importance. Additionally, mission
> instantiation informs a system for starting dependent missions if an
> arbitrary expression (which has access to that mission's data) returns
> true. One of the key features is that these missions are taken by
> heterogeneous groups of players and NPCs. My hope is to integrate a
> planning agent framework so that NPCs can learn to complete missions
> without explicit instructions (which I'm currently writing as rules).
>
> I also have a set of macros for defining persistent (in theory) data
> structures, which help somewhat with managing the complexity of all the
> data I need to track that doesn't fit neatly in the rule system and
> requires global accessibility. This is perhaps the part of the program
> I'm most displeased by. What I'd really like, I think, is to replace
> it and a lot of the rule-based code with something like Cells (which I
> recently found and haven't tried out yet). I have been envisioning
> something along those lines since I first read about Xanadu and zigzag
> years ago, but I could never figure out how to organize or index so
> much data in a usefully scalable way.
>
> So, enough background... I'd like to get feedback from the real Lisp
> hackers about my architecture direction. I'd also like some advice
> about how to approach using something like Cells, and/or
> Elephant/AllegroCache (ie is Cells anywhere near production ready?).
It is in production, more I cannot say. Aside from that, it has been
through a lot and seems solid now. Bug reports or questions always welcome.
Or roll your own. :)
> AllegroCache really looks like it could work well in our environment
> (where retrieving an object should also recreate it in the
> game-universe and vice versa), and might even do what I want Cells to
> do (based on my interpretation of the 8.0 features page), but I'd like
> to stick with F/OSS if possible.
Persistence and Cells seem orthogonal to me, so it is not clear what you
are asking or how AllegroCache gives you Cells functionality. But, yes,
transparent CLOS persistence rocks. :) My experience was with AllegroStore.
fwiw, I did marry Cells and AllegroStore, but this was about giving a
database the functionality of Cells (automatic propagation of state
change according to application rules). Hellasweet, but then I just
prefer programming with Cells regardless, so I am biased.
Funny you should ask. Someone else was asking offline about (+ Cells
Acache), and I did take a quick look at recreating my astore marriage
under acache. As you suggest, it would be nicer under a (truly) free
persistent DB. But I am still not sure you need Cells at all based on
the remark above that Acache might do what you need out of the box.
Is /Elephant/ ready for prime time?
kenny
Thanks for your reply! I didn't expect to hear from the author, and I
didn't mean to imply that your code was buggy and Elephant wasn't or
anything. I simply haven't looked into either enough to know, yet.
>But I am still not sure you need Cells at all based on
>the remark above that Acache might do what you need out of the box.
The feature blurbs on Franz's site that make me think ACache might fit
my need (not necessarily in the same way, but solving the same
problems) are these:
# Convenient macros to loop over classes, maps and sets
# Indexed slots -- A mapping from slot-values to objects, retrieve
objects and object ids (oid).
# Pcache -- An update to Allegro Prolog, an implementation of Prolog
within Allegro CL. It allows Allegro Prolog to be used as a query
language for AllegroCache.
As it's a persistent game, persistence is a big plus, though it can
definitely be acheived at a higher level than all objects; many things
don't have to resume *exactly* after a server restart. What I'm more
concerned about is being able to access the various game info of online
and offline characters in various contexts without worrying about
whether an object is in memory. More than that (and this is where
Cells or Pcache comes in), I'd like many abstract values to be
calculated whenever a dependant slot is changed, and I'd like to be
able to refer to sets of values by constraints (I hope I'm using these
terms correctly). For instance, I'd like to be able to get the set of
all sectors in systems occupied by faction X where there are twice as
many chacters belonging to faction Y as faction X.
~Michael
From: Kenny Tilton
Subject: Re: Refining the arch. of my MMORPG high-level-AI/gameplay service?
Date:
Message-ID: <43CB90B0.6030304@nyc.rr.com>
···············@gmail.com wrote:
> Thanks for your reply! I didn't expect to hear from the author,..
One of the advantages of small communities. :)
> and I
> didn't mean to imply that your code was buggy and Elephant wasn't or
> anything.
It was a totally fair question. Cells has no mailing list traffic and
like four known users, unless you count Cells-Gtk. Come to think of it,
that might be another good sign. I have yet to get an error report or
question from them. But again, not a big community so that is not much
of a claim.
>
>
>>But I am still not sure you need Cells at all based on
>>the remark above that Acache might do what you need out of the box.
>
>
> The feature blurbs on Franz's site that make me think ACache might fit
> my need (not necessarily in the same way, but solving the same
> problems) are these:
>
> # Convenient macros to loop over classes, maps and sets
> # Indexed slots -- A mapping from slot-values to objects, retrieve
> objects and object ids (oid).
> # Pcache -- An update to Allegro Prolog, an implementation of Prolog
> within Allegro CL. It allows Allegro Prolog to be used as a query
> language for AllegroCache.
>
> As it's a persistent game, persistence is a big plus, though it can
> definitely be acheived at a higher level than all objects; many things
> don't have to resume *exactly* after a server restart. What I'm more
> concerned about is being able to access the various game info of online
> and offline characters in various contexts without worrying about
> whether an object is in memory.
Do you mean you want to keep the in-memory state consistent with
possible changes to the shared DB by other users?
I did a few different things with Cells and Astore:
-- persistent (PS) slots dependent on other PS slots of the same or
other PS instances. In a sense, the DB was "alive"; change one instance
and see a bunch of others change. Scary concept, but of course also a
nice guarantee of DB consistency at the level of application rules.
-- PS slots dependent on the /population/ of PS classes. ie, a rule
fires when a PS instance of a specified class gets created or deleted.
Normally Cells depend on other slot values. Existential dependency (if
you will) there is done simply by having some slot of one instance be a
list of other instances, ie, a population. With AStore one could depend
directly on the population of a persistent class.
-- non-PS slots of PS instances (Astore supoorted these) dependent on PS
slots of the same or other PS instances. Acache seems to have a much
different way of presenting PS instances to a running process, so here
is where the research might get interesting.
-- slots of non-PS instances dependent on PS slots and PS class
populations. A good example of this would be a GUI list box showing all
customers and staying current with the database as customers got added
and removed.
-- all the above worked in a multi-user environment, so my GUI list of
customers changed when some other client added a customer. (Yes,this
required a little cooperation from all clients touching the DB.)
Speaking of the latter, and this may be of more interest to you, I have
been toying with implementing what I will call IPCells (+ Cells IPC).
That would allow programmers to use Cell semantics to manage state
change across processes (hence the IPC in IPCells). That would let your
application rather seamlessly (from the prgrammer's standpoint) be aware
of actions taken by other players.
> More than that (and this is where
> Cells or Pcache comes in), I'd like many abstract values to be
> calculated whenever a dependant slot is changed,...
I am still not sure how Acache or Pcache (+ Prolog Acache for lurkers)
helps with that, unless you are referring to things like index slots or
some API call such as Astore's collect-references. But Pcache was news
to me so whadoIknow?
> and I'd like to be
> able to refer to sets of values by constraints (I hope I'm using these
> terms correctly). For instance, I'd like to be able to get the set of
> all sectors in systems occupied by faction X where there are twice as
> many chacters belonging to faction Y as faction X.
You do want to use A/PCache for the queries for efficiency since they
execute the query without loading every instance touched into the cache.
And then this has nothing to do with Cells. :)
What I am hearing is that you would say:
(make-instance 'strategy
:easy-targets (c? (find-if
(lambda (sector)
(> (/ (us sector)(them sector)) 2))
(mapcan 'sectors (occupied-systems us)))))
...translated appropriately to AllegroProlog/Acache query language. So,
yeah, it looks like Cells and Acache but really each is doing a
different job. And that is why it took a certain amount of glue to make
the above work, such that if our last guy left a system, the
occupied-systems slot got updated, and then the easy-targets slot got
updated, all automatically (given rules like the one above and the
simple act of moving our last guy out of the given system.
Fortunately, AStore provided all the hooks I needed to make it all
work... well, as I said above, participating clients needed also to
cooperate, but this too we were able to bury in PCells (persistent
Cells) internals so it was no concern to the application developer.
What is not clear to me is whether Acache offers the same hooks. I
started to look at Acache to assess this, but then the requirement got
tabled and I moved on to other things. Those still occupy me, but I will
be needing all this myself in a few months for my own peer-peer
application, so if you all want to explore this (or IPCells or both) now
as an open-source extension to Cells I could divert some energy to it.
IIRC, PCells was just a few weeks effort given Astore's excellent AMOP
adherence.
I CCed the Cells list to see if others want to hop aboard. Like I said,
someone else was asking about this just a few days ago. That makes three
-- what's that a movement? :)
kenny
Kenny Tilton wrote:
> What is not clear to me is whether Acache offers the same hooks. I
> started to look at Acache to assess this, but then the requirement got
> tabled and I moved on to other things. Those still occupy me, but I will
> be needing all this myself in a few months for my own peer-peer
> application, so if you all want to explore this (or IPCells or both) now
> as an open-source extension to Cells I could divert some energy to it.
> IIRC, PCells was just a few weeks effort given Astore's excellent AMOP
> adherence.
btw, if you do tackle persistent Cells on your own, the other person
doing so found a brain dead bug in defmodel should you specify the
:metaclass. append vs. list or something. Patch not yet committed.
kt