From: jonathon
Subject: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <1130976842.700839.40640@g47g2000cwa.googlegroups.com>
Suppose you need to whip up a prototype of a data acquisition
application that needs to use the serial port of a *nix machine (i386
of course) to talk to an instrument, acquire data, and display it as a
graph.  Since this is only a prototype and you will not be using CL as
the final implementation language, you want a free implementation of
CL.

What would be the easiest and simplest combination of implementation
and GUI toolkit to do this?  SBCL and lambda-gtk
?

From: Alexander Schmolck
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <yfswtjq8qsy.fsf@gmail.com>
"jonathon" <···········@bigfoot.com> writes:

> Suppose you need to whip up a prototype of a data acquisition
> application that needs to use the serial port of a *nix machine (i386
> of course) to talk to an instrument, acquire data, and display it as a
> graph.  Since this is only a prototype and you will not be using CL as
> the final implementation language, you want a free implementation of
> CL.
>
> What would be the easiest and simplest combination of implementation
> and GUI toolkit to do this?  SBCL and lambda-gtk

You're not terribly specific about your needs, for example do you need general
GUI functionality? If not (and your want more than very rudimentary plotting
capabilities) you might be better off talking to something else rather then
rolling your own. 

If your graphs don't need to be partiuclarly pretty, your simplest option
would presumably be talking to gnuplot via a pipe (or grace, as long as you
don't need 3D). Libraries for that purpose both already exist plentyfully in
various languages; for gnuplot I've even come across various somewhat
rudimentary/broken CL versions.

Doing plotting well seems to be non-trivial; although I must have used over a
dozen plotting programs and libraries I've only come across two that are
fairly general and high quality: matlab (there already is a clisp interface,
but matlab is everything but free) and python's matplotlib (free and although
I haven't used it much, possibly the best; can be awkward to install, though).

If you come up with something that works well I'd for one be interested to
hear about it.

'as
From: Oyvin Halfdan Thuv
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <slrndmkioj.5uj.oyvinht@apollo.orakel.ntnu.no>
In article <···············@gmail.com>, Alexander Schmolck wrote:
> "jonathon" <···········@bigfoot.com> writes:
>>
>> What would be the easiest and simplest combination of implementation
>> and GUI toolkit to do this?  SBCL and lambda-gtk
>
> Doing plotting well seems to be non-trivial; although I must have used over a
> dozen plotting programs and libraries I've only come across two that are
> fairly general and high quality: matlab (there already is a clisp interface,
> but matlab is everything but free) and python's matplotlib (free and although
> I haven't used it much, possibly the best; can be awkward to install, though).
> 
> If you come up with something that works well I'd for one be interested to
> hear about it.

Yes, using gnuplot is a good idea. However, it's quite easy to make an add-on
to ltk[*] since it is CLOS-based. I made a (quick and _very_ dirty) graph add-on
just recently to visualize the results of a GA-program I've been working on.

Look at http://www.oyvins.net/plots/ for a snapshot (of a still running program)
and source code for the add-on. Making something quite more advanced should be
possible with very little effort.

[*] http://www.peter-herth.de/ltk/

-- 
Oyvin
From: Pascal Costanza
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <3st3tpFpv9q0U1@individual.net>
jonathon wrote:
> Suppose you need to whip up a prototype of a data acquisition
> application that needs to use the serial port of a *nix machine (i386
> of course) to talk to an instrument, acquire data, and display it as a
> graph.  Since this is only a prototype and you will not be using CL as
> the final implementation language, you want a free implementation of
> CL.
> 
> What would be the easiest and simplest combination of implementation
> and GUI toolkit to do this?  SBCL and lambda-gtk
> ?

I recall Mario Mommer describing his approach, which was basically 
dumping out the data in a format that can be read by some graph plotting 
program (I don't recall the concrete program), and then letting that 
program do the rest. This will probably give you very nice 
visualizations very quickly.

The cool thing was that he could interrupt his simulation programs at 
any point, dump the data and let the program continue when the graphs 
looked ok, with very little programming effort...

(I am just retelling this story from what he told me some time ago, so 
maybe it's not completely correct.)


Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: John Thingstad
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <op.szm4vhxppqzri1@mjolner.upc.no>
On Thu, 03 Nov 2005 01:14:02 +0100, jonathon <···········@bigfoot.com>  
wrote:

>
> Suppose you need to whip up a prototype of a data acquisition
> application that needs to use the serial port of a *nix machine (i386
> of course) to talk to an instrument, acquire data, and display it as a
> graph.  Since this is only a prototype and you will not be using CL as
> the final implementation language, you want a free implementation of
> CL.
>
> What would be the easiest and simplest combination of implementation
> and GUI toolkit to do this?  SBCL and lambda-gtk
> ?
>

For a protytype you might try the personal version of LispWorks.
I seem to remeber that it has a library for accessing the serial port.

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
From: ·········@cern.ch
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <yzo1x1ydv1j.fsf@cern.ch>
Jonathon> What would be the easiest and simplest combination of
Jonathon> implementation and GUI toolkit to do this?  SBCL and
Jonathon> lambda-gtk

CMUCL comes with "termios.h"-style serial line support built in - it
seems this baby was thrown out with the bathwater in the CMUCL -> SBCL
makeover. For the purpose of talking to an instrument, you'll probably
want to add UNIX:UNIX-CFMAKERAW - a trivial task (see man termios(3)).

Assuming your real need is plotting ("graphing"), I'd not even look at
the GUI libraries - I've found their plotting widgets are available
only as add-ons, hard to use, and visually ugly. Rather, consider
piping the data to gnuplot or similar - a number of lisp interfaces
are available (maxima, clocc) or you can easily roll your own.

Ole
From: jonathon
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <1131018781.833232.41720@o13g2000cwo.googlegroups.com>
Almost all the responses have suggested piping data to an outside
plotting program.  That's really a good idea, since we actually do
something similar on Windows with the current implementation.  The only
difference is that the real-time data is displayed in an instantiated
third-party graph control until the scan is done, then all the data is
dumped into a graphing container.

Here's a really crazy idea: what about running the data acquisition
program as a web application on the workstation, showing the control
interface on a web browser, but the data output in an instance of
gnuplot?
From: Marc Battyani
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <cuSdnUzWe8thHffeRVnysA@giganews.com>
"jonathon" <···········@bigfoot.com> wrote

> Almost all the responses have suggested piping data to an outside
> plotting program.  That's really a good idea, since we actually do
> something similar on Windows with the current implementation.  The only
> difference is that the real-time data is displayed in an instantiated
> third-party graph control until the scan is done, then all the data is
> dumped into a graphing container.
>
> Here's a really crazy idea: what about running the data acquisition
> program as a web application on the workstation, showing the control
> interface on a web browser, but the data output in an instance of
> gnuplot?

All my applications for our smart sensors are like that (web applications).
You don't even need gnuplot, you can generate SVG or VML(IE only) to draw
whatever chart you want. In one application I used cl-pdf to generate a
chart and pdf2swf to convert it in flash to display on a web page. You can
display pdf files directly on a page but it was too slow.

And of course use cl-pdf(*) and cl-typesetting(*) to generate high quality
printable and email-able reports.

(*) Advertising ;-)

Marc
From: jonathon
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <1131057945.352244.196830@g44g2000cwa.googlegroups.com>
Marc Battyani wrote:
> All my applications for our smart sensors are like that (web applications).
> You don't even need gnuplot, you can generate SVG or VML(IE only) to draw

Since data acquisition usually shows a real-time display of results so
far, is this doable with SVG?  I'm just getting into more advanced
browser tools like xsl transformations and AJAX.
If not, I could always use a separate display for the real-time data,
then dump the result into the browser as a container.

> whatever chart you want. In one application I used cl-pdf to generate a
> chart and pdf2swf to convert it in flash to display on a web page. You can
> display pdf files directly on a page but it was too slow.

Or is this what you mean by using flash?
From: Marc Battyani
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <jN6dnZHEIbZnC_feRVnyvA@giganews.com>
"jonathon" <···········@bigfoot.com> wrote in message
·····························@g44g2000cwa.googlegroups.com...
>
> Marc Battyani wrote:
> > All my applications for our smart sensors are like that (web
applications).
> > You don't even need gnuplot, you can generate SVG or VML(IE only) to
draw
>
> Since data acquisition usually shows a real-time display of results so
> far, is this doable with SVG?  I'm just getting into more advanced
> browser tools like xsl transformations and AJAX.
> If not, I could always use a separate display for the real-time data,
> then dump the result into the browser as a container.

Well it depends what kind of real time you need. In the case of my sensors
network, the update rate is every few seconds. My Lisp web app framework is
fully based on an AJAX like system so I can send notifications to the
browser whenever data changes.
Anyway in you case you probably only need to set the refresh directive of
you page to a few seconds.

> > whatever chart you want. In one application I used cl-pdf to generate a
> > chart and pdf2swf to convert it in flash to display on a web page. You
can
> > display pdf files directly on a page but it was too slow.
>
> Or is this what you mean by using flash?

Yes: cl-pdf chart => .pdf => .swf => flash player in a page.

Marc
From: Kai Kaminski
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <m2pspgls6m.fsf@Pupone.local>
"jonathon" <···········@bigfoot.com> writes:

> Marc Battyani wrote:
>> All my applications for our smart sensors are like that (web applications).
>> You don't even need gnuplot, you can generate SVG or VML(IE only) to draw
>
> Since data acquisition usually shows a real-time display of results so
> far, is this doable with SVG?  I'm just getting into more advanced
> browser tools like xsl transformations and AJAX.
You can embed Javascript programs in SVG documents. As far as I
understand at least the Adobe Viewer and maybe others support an
AJAX-like mechanism, so that you can have a self-updating SVG
document. Google for SVG and getURL or go to
http://jibbering.com/2002/5/dynamic-update-svg.html directly. This
doesn't seem to work on all browsers yet, but if you have control over
the environment that might not be a problem.

Kai
From: Zach Beane
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <m31x1xcrvt.fsf@unnamed.xach.com>
"Marc Battyani" <·············@fractalconcept.com> writes:

> All my applications for our smart sensors are like that (web applications).
> You don't even need gnuplot, you can generate SVG or VML(IE only) to draw
> whatever chart you want. In one application I used cl-pdf to generate a
> chart and pdf2swf to convert it in flash to display on a web page. You can
> display pdf files directly on a page but it was too slow.

Damn...I'll get cracking on a SWF backend for cl-pdf. I had no idea
you were resorting to this sort of hack!

Zach
From: Marc Battyani
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <9IOdnfvTm9ynCPfeRVnyiA@giganews.com>
"Zach Beane" <····@xach.com> wrote
> "Marc Battyani" <·············@fractalconcept.com> writes:
>
> > All my applications for our smart sensors are like that (web
applications).
> > You don't even need gnuplot, you can generate SVG or VML(IE only) to
draw
> > whatever chart you want. In one application I used cl-pdf to generate a
> > chart and pdf2swf to convert it in flash to display on a web page. You
can
> > display pdf files directly on a page but it was too slow.
>
> Damn...I'll get cracking on a SWF backend for cl-pdf. I had no idea
> you were resorting to this sort of hack!

Hi Zach,

At that time you told me your cl-flash stuff was not ready yet. ;-)
If it is now, It would be very interesting to add a cl-swf backend to
cl-pdf.

Marc
From: Zach Beane
Subject: Re: Data acquisition (with graphing) on *nix
Date: 
Message-ID: <m3wtjpb8nj.fsf@unnamed.xach.com>
"Marc Battyani" <·············@fractalconcept.com> writes:

> Hi Zach,
> 
> At that time you told me your cl-flash stuff was not ready yet. ;-)
> If it is now, It would be very interesting to add a cl-swf backend to
> cl-pdf.

Still not ready, but I know a bit more now and I don't think it would
be too hard for cases like this. Now it just has to float to the top
of my TODO list. :)

Zach