I'm decommutating a series of packets arriving over a local serial
port, translating the contents into instances of appropriate classes.
The device sending the packets defines their structure- its an
assembly program running on an 8051 and the packet data is a chunk of
functionally related bytes and words. Below is an example of the data
in the "status" packet;
STATS_START equ $
rxPktCount: db 0,0
rxByteCount: db 0,0
rxInvalidPkt: db 0,0
rxDiscardPkt: db 0,0
txPktCount: db 0,0
txByteCount: db 0,0
m0pwmTicks: db 0
m1pwmTicks: db 0
idlePasses: db 0
allStop: db 0
motorDeadman: db 0
motorScalar: db 0
flipMode: db 0
crawlMode: db 0
actionEnabled: db 0
fireWeapon: db 0
weaponEnable: db 0
STATS_END equ $
STATS_LEN equ (STATS_END - STATS_START)
I can manage the parsing reasonably enough. Naturally, the layout of
the packet is sensitive to fields moving around, appearing and
disappearing. I decided I wanted to make the Lisp side representation
simple enough to make keeping the packet definitions in sync with
those of the controller fairly easy. Below is an extract of the
related code;
(defclass tlm-slot ()
((slot-name :initarg :slot-name :accessor slot-name)
(byte-length :initarg :byte-length :accessor byte-length)
(accessor-name :initarg :accessor-name :accessor accessor-name)))
(let ((class-list '((("pkt-stats") ; class name
(("rxPktCount" . 2)
("rxByteCount" . 2)
("rxInvalidPkt" . 2)
("rxDiscardPkt" . 2)
("txPktCount" . 2)
("txByteCount" . 2)
("m0pwmTicks" . 1)
("m1pwmTicks" . 1)
("idlePasses" . 1)
("allStop" . 1)
("motorDeadman" . 1)
("motorScalar" . 1)
("flipMode" . 1)
("crawlMode" . 1)
("actionEnabled" . 1)
("fireWeapon" . 1)
("weaponEnable" . 1))) )))
The tlm-slot class lets me assemble a list of the packet class's slots
which is stored in the packet class's property list. It also provides
the packet parsing logic with field length info.
So, to keep the packet definitions in sync, I modify the class-list,
then run through it with a simple bit of code, assembling a defclass
form representing the class, which is sent thru read & eval to
define/redefine the class. As a consequence of creating the defclass
form, I can also set up accessors to taste. Changes to the packet
layout are infrequent and mostly consist of new fields.
My idea was the slots will always assume names consistent with the
cooresponding definitions in the Assembly side. By keeping the slot
names similar, I can write other code that uses the slots of the
packet class in a straightforward manner. Naturally if a field goes
away, so will the slot name. Instances of these classes are telemetry
coming from a robot and I won't be storing them, so the class
definition changing out from under instances of the class will only
occur in well-defined circumstances.
I could no doubt parse the 8051's assembly listing file to obtain
packet definitons, not to mention making the packet specification more
complete, but my questions are more about the way I'm approaching the
problem;
1. Is it a reasonable approach to compute, read and eval a new
defclass form when redefining the class? Or, is there a better
way?
2. Are there CL functions to obtain things like a list of slot names
from a class? I know it can be done, but I've not come across the
requisite functions or methods. I suspect MOP is the way, but I
could use a hint or two.
Thanks,
Greg Menke
>
> 1. Is it a reasonable approach to compute, read and eval a new
> defclass form when redefining the class? Or, is there a better
> way?
a. I wonder if you could use the MOP to dynamically add/remove slots. I
am thinking you might first modify the list of direct slot definitions
via gf. 'class-direct-slots and then rebuild the class via
'compute-slots. Not sure if those are the right entry-points or even if
the MOP supports this, but I think it does.
b. A bigger wholly gratuitous <g> question I have is whether you are
using too big a hammer for the job. Are you using any CLOS capabilities
other than storing the data in a coherent structure you can then access
with functions named for the slots?
>
> 2. Are there CL functions to obtain things like a list of slot names
> from a class? I know it can be done, but I've not come across the
> requisite functions or methods. I suspect MOP is the way, but I
> could use a hint or two.
Hint 1: Under ACL try 'class-slots, under MCL (going on three-year old
memory) you need to take the union of 'class-class-sots and
'class-instance-slots or somethin like that.
Hint 2: http://www.alu.org/mop/contents.html
kenny
clinisys
> b. A bigger wholly gratuitous <g> question I have is whether you are
> using too big a hammer for the job. Are you using any CLOS capabilities
> other than storing the data in a coherent structure you can then access
> with functions named for the slots?
I think it certainly is a bigger hammer than really necessary, but I
need practice making things go in Lisp. I think slot-value will be
handy when dumping records to the display- assuredly not a compelling
reason, but handy nevertheless.
>
> >
> > 2. Are there CL functions to obtain things like a list of slot names
> > from a class? I know it can be done, but I've not come across the
> > requisite functions or methods. I suspect MOP is the way, but I
> > could use a hint or two.
>
> Hint 1: Under ACL try 'class-slots, under MCL (going on three-year old
> memory) you need to take the union of 'class-class-sots and
> 'class-instance-slots or somethin like that.
Thanks for your help,
Gregm
Greg Menke <··········@mindspring.com> writes:
> > b. A bigger wholly gratuitous <g> question I have is whether you are
> > using too big a hammer for the job. Are you using any CLOS capabilities
> > other than storing the data in a coherent structure you can then access
> > with functions named for the slots?
>
> I think it certainly is a bigger hammer than really necessary, but I
> need practice making things go in Lisp. I think slot-value will be
> handy when dumping records to the display- assuredly not a compelling
> reason, but handy nevertheless.
But AREF or GETHASH might be just as handy, for example...?
Kent M Pitman <······@world.std.com> writes:
> Greg Menke <··········@mindspring.com> writes:
>
> > > b. A bigger wholly gratuitous <g> question I have is whether you are
> > > using too big a hammer for the job. Are you using any CLOS capabilities
> > > other than storing the data in a coherent structure you can then access
> > > with functions named for the slots?
> >
> > I think it certainly is a bigger hammer than really necessary, but I
> > need practice making things go in Lisp. I think slot-value will be
> > handy when dumping records to the display- assuredly not a compelling
> > reason, but handy nevertheless.
>
> But AREF or GETHASH might be just as handy, for example...?
my thoughts exactly (i.e. GETHASH). However...some thoughts.
First off, I am of the camp that believes that in almost all
circumstances of programming development, the data structures and thus
class structures are known when development occurs. Same pretty much
goes for methods, though not quite to the same degree [In fact, right
now I'm playing with a system that manipulates which methods are
activated at any given time...very fun stuff].
The answer to the question of when it's right to create classes
on-the-fly (at run-time) is (and I stress, in my opinion) when you're
using Common Lisp to create other programming tools
(e.g. special-purpose code-building tools, GUI tools, etc.) Once you
give another programmer the power to develop their own schema, you might
finally make use of class definition and instance creation and the
instance updates on redefined classes. So the short of it is: if you're
building tools for other programmers, then this is just about the only
time you'll want to use these features. I personally wouldn't feel
comfortable straying from this boundary, though it's probably just
because I have never been made aware of another application of this
particular capability of Common Lisp.
On that note, not just CLOS, but particularly the MOP extensions become
valuable. I worked on a commercial CL system that created such an
interface for programmers, and ran CL under the hood, and the whole
object and function system was build from scratch. Of course, I think
that that system was born before the MOP. But I can easily see how they
could have simplified their lives, and how their stuff could have been
implemented in 1/10 the LOC, 1/50 the pain, and 100x the elegance and
maintainability.
I'll end off with an example, just because everything above is pretty
elusive. Suppose you were building an IVR (Interactive Voice Response)
programming toolkit. These exist today, and you can buy them and use
them to create those fancy menus you use when you call any major agency
or company. Let's say, for the sake of discussion that you were using
LispWorks, and CLIM, and all the other nice provisions of LispWorks
(e.g. tight ANSI support, networking, Corba, database interface, etc.)
Suppose that you had certain building blocks, like MENU-SELECTION, and
GET-N-DIGITS, PLAY-RECORDING, and possibly RECORD-MESSAGE. These
building blocks vary in low- to high-level abstractions. For example,
the PLAY-RECORDING just needs a sound file to read off, whereas the
MENU-SELECTION requires multiple sound files, corresponding to different
digits, and actions for each.
Suppose now that the programmer notices a particular sequence or block
that doesn't exist as its own entity, but he or she can create it and
group it with the toolkit, and then give it a name, and re-use it.
Method combination governs how these blocks are strung together, and the
dynamic definition of a new class, with its own name (a label) lets you
use it over and over, and later on define additional behavior.
With the right metaclass, the right method combination, such a system
can be made so that these designs are safe, consistent, and even
persistent with little overhead to the tools developer. But dynamic
class definition is among the features of CLOS that would needed to be
used to do it with low overhead.
hope this helps. Again, I'd love to hear more ways that people have
used dynamic class definition in their CL-based apps.
dave
Greg Menke <··········@mindspring.com> writes:
> I'm decommutating a series of packets arriving over a local serial
> port, translating the contents into instances of appropriate
> classes.
This seems to be a perfect fit for my "binary-types" library, at
<URL:http://www.cs.uit.no/~frodef/sw/binary-types/>.
> (let ((class-list '((("pkt-stats") ; class name
>
> (("rxPktCount" . 2)
> ("rxByteCount" . 2)
> ("rxInvalidPkt" . 2)
> ("rxDiscardPkt" . 2)
> ("txPktCount" . 2)
> ("txByteCount" . 2)
> ("m0pwmTicks" . 1)
> ("m1pwmTicks" . 1)
> ("idlePasses" . 1)
> ("allStop" . 1)
> ("motorDeadman" . 1)
> ("motorScalar" . 1)
> ("flipMode" . 1)
> ("crawlMode" . 1)
> ("actionEnabled" . 1)
> ("fireWeapon" . 1)
> ("weaponEnable" . 1))) )))
(define-binary-class pkt-stats ()
((rxPktCount :accessor ... :binary-type 'u16)
(rxByteCount :accessor ... :binary-type 'u16)
(rxInvalidPkt :accessor ... :binary-type 'u16)
(rxDiscardPkt :accessor ... :binary-type 'u16)
(txPktCount :accessor ... :binary-type 'u16)
(txByteCount :accessor ... :binary-type 'u16)
(m0pwmTicks :accessor ... :binary-type 'u8)
(m1pwmTicks :accessor ... :binary-type 'u8)
(idlePasses :accessor ... :binary-type 'u8)
(allStop :accessor ... :binary-type 'u8)
(motorDeadman :accessor ... :binary-type 'u8)
(motorScalar :accessor ... :binary-type 'u8)
(flipMode :accessor ... :binary-type 'u8)
(crawlMode :accessor ... :binary-type 'u8)
(actionEnabled :accessor ... :binary-type 'u8)
(fireWeapon :accessor ... :binary-type 'u8)
(weaponEnable :accessor ... :binary-type 'u8)))
The ... means whatever regular slot-options you care to include.
If your data comes in through a stream, you can then do
(read-binary 'pkt-stats stream)
> So, to keep the packet definitions in sync, I modify the class-list,
> then run through it with a simple bit of code, assembling a defclass
> form representing the class, which is sent thru read & eval to
> define/redefine the class. As a consequence of creating the
> defclass form, I can also set up accessors to taste. Changes to the
> packet layout are infrequent and mostly consist of new fields.
If at all possible, wouldn't it be much more practical to generate the
assembly definitions from the lisp source, rather than the other way
around? Something along the lines of:
(with-open-file (stream "packet.s" :direction :output)
(dolist (slot (binary-record-slot-names 'pkt-stats))
(format stream ...)))
> 1. Is it a reasonable approach to compute, read and eval a new
> defclass form when redefining the class? Or, is there a better
> way?
Compiling/evaluating defclass forms would be appropriate, I believe.
> 2. Are there CL functions to obtain things like a list of slot names
> from a class? I know it can be done, but I've not come across the
> requisite functions or methods. I suspect MOP is the way, but I
> could use a hint or two.
The MOP will be able to tell you things like the slots of a class. Or,
as I hinted above, my library provides such functionality because it's
a common requirement for these kinds of applications.
--
Frode Vatvedt Fjeld
Frode Vatvedt Fjeld <······@acm.org> writes:
> > I'm decommutating a series of packets arriving over a local serial
> > port, translating the contents into instances of appropriate
> > classes.
>
> This seems to be a perfect fit for my "binary-types" library, at
> <URL:http://www.cs.uit.no/~frodef/sw/binary-types/>.
I looked very closely at it, the only thing that prevented me from
using it was I wasn't clear how I could parse varying length packets.
The packet header includes a data length field as shown below in the
"definition" of the controller's transmit buffer;
;;; transmit buffer
XMIT_PKT equ $
txhdr: db 0,0
txid: db 0
txchksum: db 0
txlen: db 0
txbuf: db 0
org $+PACKET_DATA_LEN+1
so when the controller sends a packet, it includes the # of payload
bytes in the txlen field. Since several packet types are defined, an
identifier indicating which is put in the txid field. These 2 fields
are used at different protocol levels in the receiving side. But I
think it could work anyhow;
If I defined a binary-class for the packet header, I could read-binary
to grab it. After which, I examine the txid slot to identify which
other binary-class corresponds to the data, calling read-binary again
supplying the correct class.
Thanks for making me think about it again, I wasn't thinking about it
this way. I'll give binary-types another try.
> > So, to keep the packet definitions in sync, I modify the class-list,
> > then run through it with a simple bit of code, assembling a defclass
> > form representing the class, which is sent thru read & eval to
> > define/redefine the class. As a consequence of creating the
> > defclass form, I can also set up accessors to taste. Changes to the
> > packet layout are infrequent and mostly consist of new fields.
>
> If at all possible, wouldn't it be much more practical to generate the
> assembly definitions from the lisp source, rather than the other way
> around? Something along the lines of:
>
> (with-open-file (stream "packet.s" :direction :output)
> (dolist (slot (binary-record-slot-names 'pkt-stats))
> (format stream ...)))
Yes, a more complete implementation would do something like this. At
the moment things are sort of quick and dirty. A friend & I are
building a robot to compete in Robotica (we're alternates at the
moment), my version of the PID controller emits the telemetry as above
& I'm processing it in Lisp for practice. Since the bot has to ship
next week, we'll probably use whichever controller actually
works... ;)
Photos are at http://www.mindspring.com/~menkesjg/robot for the
moment, moving to www.dredgenet.org at some point.
Thanks,
Greg Menke
> A friend & I are
> building a robot to compete in Robotica (we're alternates at the
> moment),
OK, I am totally jealous. I am a big BattleBots fan myself, watch it
down at the corner pub every week, and of course we are fantasy-planning
a PubBot all the time. Lemme know if I can help with the Lisp, be glad
to pitch in.
kenny
clinisys
> > A friend & I are
> > building a robot to compete in Robotica (we're alternates at the
> > moment),
>
> OK, I am totally jealous. I am a big BattleBots fan myself, watch it
> down at the corner pub every week, and of course we are fantasy-planning
> a PubBot all the time. Lemme know if I can help with the Lisp, be glad
> to pitch in.
Unfortunately the bot is not entirely cool as the firmware is written
in assembly and basic, not Lisp. I have an 80188 based sbc w/ 512k
RAM, perhaps I could coax xlisp to run on it...
To be honest, I'd rather build a multi-bot where several little bots
are controlled in concert to disable an opponent. More scope for
something interesting software-wise there. But thats nothing more
than an idea right now...
Gregm
Greg Menke <··········@mindspring.com> writes:
> Unfortunately the bot is not entirely cool as the firmware is
> written in assembly and basic, not Lisp. I have an 80188 based sbc
> w/ 512k RAM, perhaps I could coax xlisp to run on it...
If you took the effort to write an 8051 assembler in lisp (that uses
lispish assembly syntax), that'd probably be an excellent starting
point for a lisp-based IDE for the 8051. Keeping the lisp and 8051
sides of things in sync would be a breeze, and you'd probably find
you'd be able to write quite interesting 8051 macros using lisp as the
macro language rather than some crummy "traditional" assembler.
I started writing such an assembler for the PIC microcontrollers (it's
not quite completed as my PIC project is still somewhere on the
todo-list), but maybe you want to look at it for ideas:
<URL:http://www.cs.uit.no/~frodef/picl.lisp>
--
Frode Vatvedt Fjeld
>
> > Unfortunately the bot is not entirely cool as the firmware is
> > written in assembly and basic, not Lisp. I have an 80188 based sbc
> > w/ 512k RAM, perhaps I could coax xlisp to run on it...
>
> If you took the effort to write an 8051 assembler in lisp (that uses
> lispish assembly syntax), that'd probably be an excellent starting
> point for a lisp-based IDE for the 8051. Keeping the lisp and 8051
> sides of things in sync would be a breeze, and you'd probably find
> you'd be able to write quite interesting 8051 macros using lisp as the
> macro language rather than some crummy "traditional" assembler.
Crummy is certainly the operative word for the assembler I'm using.
Its two chief benefits are that its free and it works. It would be
very helpful to manage definitions using something Lisp-ish. Its an
interesting idea...
> I started writing such an assembler for the PIC microcontrollers (it's
> not quite completed as my PIC project is still somewhere on the
> todo-list), but maybe you want to look at it for ideas:
> <URL:http://www.cs.uit.no/~frodef/picl.lisp>
Thanks,
Gregm
Greg Menke <··········@mindspring.com> writes:
> [..] If I defined a binary-class for the packet header, I could
> read-binary to grab it. After which, I examine the txid slot to
> identify which other binary-class corresponds to the data, calling
> read-binary again supplying the correct class.
Yes, either create your own function that does this, or you can
specialize the read-binary-record method, for example like this:
(defmethod read-binary-record ((type-name (eql 'abstract-packet)) stream &key)
(let* ((header (read-binary 'header stream))
(packet (read-binary (slot-value header 'packet-type) stream)))
(setf (slot-value packet 'header) header)
packet))
Or, to read packets with variable sized payload:
(defmethod read-binary-record ((type-name (eql 'packet)) stream &key)
(let ((packet (call-next-method)))
(setf (slot-value packet 'payload)
(loop for i below (slot-value packet 'payload-size)
collect (read-binary 'payload-type stream)))
packet))
--
Frode Vatvedt Fjeld