I have noted, that people use to say, that Common Lisp environment
lacks good library support. For me that doesn't seem obvious, as in my
3 years' experience with the language I've never encountered such
situation.
So I'd like to propose a challenge for those, who hold to this
viewpoint, to bring up specific examples of situations, when a decent
library for some common task couldn't be found for Common Lisp. I
think the answers could serve as a road-map for the potential library
developers for Lisp (and not just for my and, possibly, other people
enlightenment :)
Vsevolod wrote:
> I have noted, that people use to say, that Common Lisp environment
> lacks good library support. For me that doesn't seem obvious, as in my
> 3 years' experience with the language I've never encountered such
> situation.
>
> So I'd like to propose a challenge for those, who hold to this
> viewpoint, to bring up specific examples of situations, when a decent
> library for some common task couldn't be found for Common Lisp. I
> think the answers could serve as a road-map for the potential library
> developers for Lisp (and not just for my and, possibly, other people
> enlightenment :)
I'm working on the part I found lacking (see link.) No offense intended
of course to those whose libraries fill this gap.
http://img502.imageshack.us/my.php?image=screenshot1ph2.png
On Jul 7, 12:03 am, Sohail Somani <······@taggedtype.net> wrote:
> Vsevolod wrote:
> > I have noted, that people use to say, that Common Lisp environment
> > lacks good library support. For me that doesn't seem obvious, as in my
> I'm working on the part I found lacking (see link.) No offense intended
> of course to those whose libraries fill this gap.
>
> http://img502.imageshack.us/my.php?image=screenshot1ph2.png
I would love to be able to use/access Qt from Lisp. Is it possible to
do this now -- or are you working on a library to do it? My concern
with a one-off library is that it will break when updates to Qt are
released. Is there a way someone can use some sort of procedure to
access only a gateway or the parts that they want to use. For
instance, if I only wanted to use/access QString from Lisp, is there
an *easy* way that I could do some procedure to get access to just
this item (ie: without having to make gateways to 100s of other Qt
internals, just to be able to use QString, etc).
Scott
Scott D. Yelich wrote:
> On Jul 7, 12:03 am, Sohail Somani <······@taggedtype.net> wrote:
>> Vsevolod wrote:
>>> I have noted, that people use to say, that Common Lisp environment
>>> lacks good library support. For me that doesn't seem obvious, as in my
>> I'm working on the part I found lacking (see link.) No offense intended
>> of course to those whose libraries fill this gap.
>>
>> http://img502.imageshack.us/my.php?image=screenshot1ph2.png
>
> I would love to be able to use/access Qt from Lisp. Is it possible to
> do this now -- or are you working on a library to do it? My concern
> with a one-off library is that it will break when updates to Qt are
> released. Is there a way someone can use some sort of procedure to
> access only a gateway or the parts that they want to use. For
> instance, if I only wanted to use/access QString from Lisp, is there
> an *easy* way that I could do some procedure to get access to just
> this item (ie: without having to make gateways to 100s of other Qt
> internals, just to be able to use QString, etc).
Yes. If you gave Sohail, or most anyone on cffi-devel, a list of 10
Qt functions you wanted to use (including member accesses), then we
could whip up a portable binding in roughly as many minutes. The hard
thing is that GUI libraries generally present a large number of
functions -- so many that its impractical to hand generate the
individual bindings.
That said, Paul Reutz made good progress on a full binding; the very
usable results of his now abandoned effort are at
http://lisp-cffi-qt4.sourceforge.net/
- Daniel
D Herring wrote:
> Scott D. Yelich wrote:
>> On Jul 7, 12:03 am, Sohail Somani <······@taggedtype.net> wrote:
>>> Vsevolod wrote:
>>>> I have noted, that people use to say, that Common Lisp environment
>>>> lacks good library support. For me that doesn't seem obvious, as in my
>>> I'm working on the part I found lacking (see link.) No offense intended
>>> of course to those whose libraries fill this gap.
>>>
>>> http://img502.imageshack.us/my.php?image=screenshot1ph2.png
>>
>> I would love to be able to use/access Qt from Lisp. Is it possible to
>> do this now -- or are you working on a library to do it? My concern
The plan is to have full coverage with automatically generated bindings.
Or as close to full as is possible. Sorry for the delayed response, I
didn't see this message. If you are the same Scott I emailed with this
morning, then sorry for the double message!
> Yes. If you gave Sohail, or most anyone on cffi-devel, a list of 10 Qt
> functions you wanted to use (including member accesses), then we could
> whip up a portable binding in roughly as many minutes. The hard thing
> is that GUI libraries generally present a large number of functions --
> so many that its impractical to hand generate the individual bindings.
Very true.
> That said, Paul Reutz made good progress on a full binding; the very
> usable results of his now abandoned effort are at
> http://lisp-cffi-qt4.sourceforge.net/
The problem I found with Paul's was that the interface was too different
from regular Qt. That and I don't think I ever got it to actually run.
So my goals are:
* No C++ (I think Paul's requires C++ to add a new subclass)
* Match the C++ Qt API including overloading - benefit of this is of
course that I don't have to write my own API documentation.
* Fullish, mostly automatic coverage
* As Lispy as possible (whatever that means)
These goals are simply stated but in practice are quite difficult to
achieve. In fact, if inheriting from a C++ class in Lisp is too hard, I
may just punt on that. I would appreciate any ideas so I don't have to
though.
Hopefully, the products of this work will live on, so theoretically, it
shouldn't be immediately abandoned :-)
Sohail
Sohail Somani wrote:
> So my goals are:
> * No C++ (I think Paul's requires C++ to add a new subclass)
> * Match the C++ Qt API including overloading - benefit of this is of
> course that I don't have to write my own API documentation.
> * Fullish, mostly automatic coverage
> * As Lispy as possible (whatever that means)
>
You have probably already seen it, but I wonder if the new
QtScriptGenerator [1] Qt binding generator for QtScript (adapted from
the Qt "Jambi" java qt binding generator) might be further adapted to
spit out a CFFI lisp binding. Since the generator's codebase has now
been through the mill for two different languages with different native
interface apis, there should be a lot of metainfo to point the way.
It's as "from the horse's mouth" as you can get, and being able to
maintain a CL Qt binding with feature-parity with the
Trolltech-maintained Java and QtScript Qt bindings would be quite
nice...
I note it even includes some subclassing support [2] -i.e. you
can apparently "subclass" qt classes on the QtScript side.
[1] http://labs.trolltech.com/page/Projects/QtScript/Generator
[2] http://labs.trolltech.com/blogs/2008/03/10/bind-aid/
> So I'd like to propose a challenge for those, who hold to this
> viewpoint, to bring up specific examples of situations, when a decent
> library for some common task couldn't be found for Common Lisp.
> I
> think the answers could serve as a road-map for the potential library
> developers for Lisp (and not just for my and, possibly, other people
> enlightenment :)
due to nature of software, there are too many domains and
programs are very different from each other, so there are
vast quantities of different libraries one might need.
so i think such poll in a newsgroup won't be able to
considerably improve situation -- that would be a drop in an ocean.
if you really want to improve situation in this way, you need
to establish a process -- e.g. a site that will collect
complaints about a lack of libraries or features, and will
help people to cooperate if they have similar needs.
well, you don't believe that anybody has a problem with
libraries, so i'll post some list of examples anyway.
it's in no way complete -- i think i've forgot most cases..
to start with, there is no good object persistence
libraries IMHO. yep, we have lots of such libraries, but
they sort of suck in one way or another. i'm participating
in development of one of such libraries -- called Elephant,
and i can say it's very far from being good and complete,
but it at least is able to handle simplistic things i'm
doing with it.
it was mentioned here that in web field library support
is very good, but i can't say so. Drakma HTTP client
is assumed to be of quite high quality, but unfortunately
it's unusable for automated tasks like site monitoring
and spidering. both this tasks require a timeout parameter
(for both connection and read operations) -- otherwise
remote site that is either down or evil will hang your
process indefinitely. half a year ago (when i was working
with it) Drakma completely lacked timeout parameters, currently
documentation says: "All timeout keyword arguments are only
available for LispWorks". huh, i need commercial implementation
just to make spider or monitoring script? or wait another
year till it gets support for other implementations?
Java and Python had this functionality long time ago..
(i've implemented monitoring script in Python, and
spider sort of thing in Armed Bear Common Lisp using
Java's httpclient).
as i've already mentioned spidering, i haven't seen
tools for this thing either -- i mean parsing HTML for
links, recursive downloading etc. (in one case i've
implementing spider myself, in ABCL, in another case
i've called external program called httrack).
are there libraries for implementing proxies?
haven't seen such for lisp, while for Python, Perl
and Java there are tons of examples.
what's about web frameworks? sure we have lots of them,
but i haven't seen really good ones, so i had to hack
few myself..
ok, let's leave web field for a while. i'm interested
in computational linguistics. are there libs for it?
i'm pretty sure there are some old ones from 1980',
made by students, with questionable quality, but are
there modern and good ones, with support for wide range of
implementations? haven't seen such.. (have seen such for Python,
though, and they say there are libs for C..)
there is no lib for sparse SVD in Lisp. i had to call external
Fortran program (PROPACK) communicating with it via files,
so I/O takes most of processing time. (note that _some_
port of fortran programs won't cut -- people who use
sparse SVD tend to have really large matrices, so implementation
should have top performance).
to summarize, in almost all areas i was working with there
was a lack of libraries (or lack of features in libraries),
so i had to implement functionality myself (while there were
libraries in other languages) or call external programs or
just switch implementation language to something else.
however, i can't say situation is awful -- typically
there are libs at least for basic stuff, and for long
term projects absence of domain-specific libs rarely
is a problem; small, short term projects might be affected
though, and it affects how newcomers percieve language
-- e.g. while in python something is damn simple, just
import somelib, in lisp you'll have to implement it youself.
In article <·················@your-a80c79e8b3>,
"Alex Mizrahi" <············@gmail.com> wrote:
> > So I'd like to propose a challenge for those, who hold to this
> > viewpoint, to bring up specific examples of situations, when a decent
> > library for some common task couldn't be found for Common Lisp.
> > I
> > think the answers could serve as a road-map for the potential library
> > developers for Lisp (and not just for my and, possibly, other people
> > enlightenment :)
>
>
...
> it was mentioned here that in web field library support
> is very good, but i can't say so. Drakma HTTP client
> is assumed to be of quite high quality, but unfortunately
> it's unusable for automated tasks like site monitoring
> and spidering. both this tasks require a timeout parameter
> (for both connection and read operations) -- otherwise
> remote site that is either down or evil will hang your
> process indefinitely. half a year ago (when i was working
> with it) Drakma completely lacked timeout parameters, currently
> documentation says: "All timeout keyword arguments are only
> available for LispWorks". huh, i need commercial implementation
> just to make spider or monitoring script? or wait another
> year till it gets support for other implementations?
> Java and Python had this functionality long time ago..
> (i've implemented monitoring script in Python, and
> spider sort of thing in Armed Bear Common Lisp using
> Java's httpclient).
I'm using W4 for that:
W4 64 > (define-action-type
print-inferiors
(:generator
:documentation "Primary action for printing the inferiors of a URL."
:class generator-type)
(action activity url)
(declare (ignore action))
(when (satisfies-context-constraints-p t activity)
(loop for inferior in (url-inferiors activity url)
do (print inferior))))
#<GENERATOR-TYPE PRINT-INFERIORS 41601A93C3>
NIL
W4 65 > (defun test (url)
(with-activity ("rj-inferiors" (:report-stream '*standard-output*)
:constraints '((depth 1))
:actions '((print-inferiors)))
(walk url activity)))
TEST
W4 66 > (test "http://lisp-hh.de/wiki/Lisp-User%20in%20Hamburg%20und%20Umgebung")
#<HTTP-OBJECT http://lisp-hh.de/Lisp-User+in+Hamburg+und+Umgebung 40202E122B>
#<HTTP-OBJECT http://lisp-hh.de/Home 40202E13B3>
#<HTTP-OBJECT http://lisp-hh.de/Impressum 40202E153B>
#<HTTP-OBJECT http://lisp-hh.de/Hilfe 40202E16C3>
#<HTTP-PATH http://lispm.dyndns.org/ 40202E180B>
#<HTTP-PATH http://www.freiheit.com/ 40202E1953>
#<HTTP-PATH http://jensteich.de/ 40202E1A9B>
#<HTTP-PATH http://www.filemaker.de/ 40202E1BE3>
#<HTTP-PATH http://weitz.de/ 40202E1D2B>
#<HTTP-OBJECT http://www.moltkeplatz.de/~krewinkel 40202E1E93>
#<HTTP-OBJECT http://lisp-hh.de/Lisp-User 40202E210B>
#<ACTIVITY rj-inferiors 402000FBA3>
>
> as i've already mentioned spidering, i haven't seen
> tools for this thing either -- i mean parsing HTML for
> links, recursive downloading etc. (in one case i've
> implementing spider myself, in ABCL, in another case
> i've called external program called httrack).
> are there libraries for implementing proxies?
> haven't seen such for lisp, while for Python, Perl
> and Java there are tons of examples.
>
...
> ok, let's leave web field for a while. i'm interested
> in computational linguistics. are there libs for it?
This is so vague that it sounds like FUD...
--
http://lispm.dyndns.org/
> I'm using W4 for that:
looks good, but what is it?
>> ok, let's leave web field for a while. i'm interested
>> in computational linguistics. are there libs for it?
> This is so vague that it sounds like FUD...
well, you can check yourself.. type "python natural language"
(NLP -- natural language processing)
in google and you ket Natural Language Toolkit with wealth.
if you search for "lisp natural language".. well, you get something,
but closer look reveals that it's some basic tutorials, like on
dealing with symbols, some books, paper on ILC05. where are libraries?
if we check cl-user.net, there is a tag "Natural Language Processing"
(that's good) but just two pieces under that tag (that's bad) -- one
lib doing highly specific thing, and code from AIMA book. and that's all.
so even if there are good libs for NLP, they are well hidden.
as i said, there are things like code from books, tutorials, some
code snippets, works of students decades old, and one can find some
useful parts in it -- but it is very different from having
high-quality and well-tested libs.
let's check other stuff. maybe there is at least such a simple
thing like bayesian classifier? google search for "lisp bayesian
classifier"
again finds some articles and papers, but not actual library (neither
there is one at cl-user.net). while finding Python code (or actual
utilities working from command line) for doing bayesian classification
is very easy.
P� Mon, 07 Jul 2008 13:49:23 +0200, skrev Alex Mizrahi
<············@gmail.com>:
>> I'm using W4 for that:
>
> looks good, but what is it?
>
>>> ok, let's leave web field for a while. i'm interested
>>> in computational linguistics. are there libs for it?
>
>> This is so vague that it sounds like FUD...
>
> well, you can check yourself.. type "python natural language"
> (NLP -- natural language processing)
> in google and you ket Natural Language Toolkit with wealth.
>
> if you search for "lisp natural language".. well, you get something,
> but closer look reveals that it's some basic tutorials, like on
> dealing with symbols, some books, paper on ILC05. where are libraries?
>
> if we check cl-user.net, there is a tag "Natural Language Processing"
> (that's good) but just two pieces under that tag (that's bad) -- one
> lib doing highly specific thing, and code from AIMA book. and that's all.
> so even if there are good libs for NLP, they are well hidden.
>
> as i said, there are things like code from books, tutorials, some
> code snippets, works of students decades old, and one can find some
> useful parts in it -- but it is very different from having
> high-quality and well-tested libs.
>
> let's check other stuff. maybe there is at least such a simple
> thing like bayesian classifier? google search for "lisp bayesian
> classifier"
> again finds some articles and papers, but not actual library (neither
> there is one at cl-user.net). while finding Python code (or actual
> utilities working from command line) for doing bayesian classification
> is very easy.
If you don't know NLP well enough to write the library yourself it won't
do you much good anyhow.
Also it is a huge subject. I am not sure such a general purpose library
makes sense.
Like why not a AI library? Because AI is a huge topic covering a wide
range of mostly unrelated tequniques.
It all depends what you want to do.
--------------
John Thingstad
> If you don't know NLP well enough to write the library yourself it won't
> do you much good anyhow.
have you checked what's in http://nltk.org/ ?
do you find it totally useless?
tokenizers, stemmers, interface to wordnet, classifiers
etc -- all this should be implemented from scratch each time?
> Also it is a huge subject. I am not sure such a general purpose library
> makes sense.
sure there is no such thing as universal NLP library, but library
at least can simplify implementing typical stuff, so you can focus
on your advanced algorithms, for examle.
libraries are also very useful when you're experimenting with
something, making prototype or some small project -- then you
can do this really fast. for long term, huge projects they are
not that important, i've already mentioned this.
> Like why not a AI library? Because AI is a huge topic covering a wide
> range of mostly unrelated tequniques.
because AI is wider (orders of magnitude) than NLP.
but, hey, there are some books on AI -- PAIP, AIMA -- their authors
must be stupid or something.
"Alex Mizrahi" <············@gmail.com> writes:
>
> have you checked what's in http://nltk.org/ ?
> do you find it totally useless?
> tokenizers, stemmers, interface to wordnet, classifiers
> etc -- all this should be implemented from scratch each time?
Implement it once an GPL or LGPL it.
--
Robert Uhl <http://public.xdi.org/=ruhl>
When PCs run new applications successfully, most people feel relief
and almost pathetic gratitude--a standard of reliability tolerated in
no other consumer product. --Economist, 12 Sept. 1998
On Jul 9, 3:07 pm, Robert Uhl <·········@NOSPAMgmail.com> wrote:
> "Alex Mizrahi" <············@gmail.com> writes:
>
> > have you checked what's inhttp://nltk.org/?
> > do you find it totally useless?
> > tokenizers, stemmers, interface to wordnet, classifiers
> > etc -- all this should be implemented from scratch each time?
>
> Implement it once an GPL or LGPL it.
>
> --
> Robert Uhl <http://public.xdi.org/=ruhl>
> When PCs run new applications successfully, most people feel relief
> and almost pathetic gratitude--a standard of reliability tolerated in
> no other consumer product. --Economist, 12 Sept. 1998
Some of this is done in langutils by Ian Eslick. It needs testing
under
other flavors than allegro. lacks the wordnet interface, but that is
in prolog and could get prolog in lisp embedded, yes ?
In article <·················@your-a80c79e8b3>,
"Alex Mizrahi" <············@gmail.com> wrote:
> let's check other stuff. maybe there is at least such a simple
> thing like bayesian classifier? google search for "lisp bayesian
> classifier"
> again finds some articles and papers, but not actual library (neither
> there is one at cl-user.net). while finding Python code (or actual
> utilities working from command line) for doing bayesian classification
> is very easy.
Why not just call the Python code?
--
http://lispm.dyndns.org/
On Jul 7, 8:46 pm, Rainer Joswig <······@lisp.de> wrote:
I think maybe that there are plenty of libraries to do things, but in
most cases the documentation is lacking. *That* is what's really off-
putting coming from other languages.
> In article <·················@your-a80c79e8b3>,
> "Alex Mizrahi" <············@gmail.com> wrote:
>
> > let's check other stuff. maybe there is at least such a simple
> > thing like bayesian classifier? google search for "lisp bayesian
> > classifier"
> > again finds some articles and papers, but not actual library (neither
> > there is one at cl-user.net). while finding Python code (or actual
> > utilities working from command line) for doing bayesian classification
> > is very easy.
>
> Why not just call the Python code?
>
> --http://lispm.dyndns.org/
> I think maybe that there are plenty of libraries to do things, but in
> most cases the documentation is lacking. *That* is what's really off-
> putting coming from other languages.
personally i have no problems with reading code _instead_ of documentation,
if code quality is good and it is manageable size.
it's another things when code quality is bad or it's not packaged well:
then you can easily spend more time trying to understand what it is and
how to get it working on implementation you have, than to do everything
from scratch.
there are tons of code written in Lisp, but lots of it was abandoned
long time ago and is not maintained -- so it can be seen as a material
for implementation, but not a ready-to-use library.
> From: "Alex Mizrahi" <············@gmail.com>
> there are tons of code written in Lisp, but lots of it was
> abandoned long time ago and is not maintained -- so it can be seen
> as a material for implementation, but not a ready-to-use library.
There are over six billion people in the world, many living in
poverty, many intelligent/skilled/talented but unable to find
anyone willing to pay them for their services. (I'm one of those
people, been that way since I got laid off from Stanford during the
1991 recession.) Why not take advantage of this immense pool of
available intelligent/skilled/talented labor? Specifically, arrange
that people "adopt" software modules, both modules they find
somewhere written by somebody else, and modules they wrote
themselves for self-use, and be available to provide customer
support regarding those modules:
- Free support to tweak the documentation to make it more
comprehensible to newcomers trying the software for the first
time, trying to figure out how to use it as-is, thereby
providing encouragement to potential beta-testers/evaluators to
spend time *trying* the software and *negotiating* fixups rather
than just give up on the software if the online documentation
isn't already succinct and fully informative and optimally
organized/indexed.
- Free support to fix little bugs that are totally embarrassing to
the developer/maintainer and really should have been fixed already.
- For-pay support to fix other bugs that require serious study to get right.
- For-pay support to add new features, refactor the algorithms or
the organization of parts or the API etc.
A problem is that even if there are people who have already adopted
various software modules as described above, or who *would* if
anybody expressed interest, the *information*retrieval* and
natural-language understanding problem is still unsolved. There's
no practical way for a person with some software need to feed a
query into a search engine and have that search engine understand
what he/she is talking about and map the query to a category of
software and then do a lookup of that category and output nicely
organized info about which software modules are available to deal
with that type of task and who is currently available for
customer-support for each such module.
Unfortunately, although I realized as long ago as 1970 the need to
somehow produce a decent information retrieval system for mapping
initial queries into an organized information network, even today
hardly anyone considers this to be a serious problem worth putting
to practice. Instead there are A.I. "research projects" that spend
endless monies studying the problem from a theoretical basis
without ever putting the theory to any useful effect, and major
search engines Google and Yahoo which have simply punted on the
problem, providing keyword search services that simply return a
list of all matches, in an unorganized linear sequence, instead of
organizing the results in any way per *meaning* of the terms.
Note that dir.yahoo.com has search queries feed into a tree-like
directed graph of topics. Per the topic of this thread, a search
for "software libraries" turns up these apparently relevant categories:
- Software > Archives
<http://rds.yahoo.com/_ylt=A0oGkxsUiXNIgkkAGhqEzbkF/SIG=12id6gp98/EXP=1215617684/**http%3a//dir.yahoo.com/Computers_and_Internet/Software/Archives/>
- Library Services > Software
<http://rds.yahoo.com/_ylt=A0oGkxsUiXNIgkkAHRqEzbkF/SIG=13pgoufuc/EXP=1215617684/**http%3a//dir.yahoo.com/Business_and_Economy/Business_to_Business/Information/Library_Services/Software/>
- Computer Software > Directories
<http://rds.yahoo.com/_ylt=A0oGkxsUiXNIgkkAJhqEzbkF/SIG=13ieopm1j/EXP=1215617684/**http%3a//dir.yahoo.com/Business_and_Economy/Business_to_Business/Computers/Software/Directories/>
I'd like your opinion whether these categories to any degree
whatsoever partly satisfy the need to find software that performs
certain kinds of tasks as well as documentation and customer
support for the software. My own quick glance at those three
categories shows the first to be probably useless, the second
off-topic, but the third possibly useful but I don't have time to
study it in detail right now.
If that Yahoo category turns out to be useless for our needs, would
anybody be willing to brainstorm with me to develop a decent system
for organizing this kind of information?
By the way, I've written a lot of software, mostly for my personal
use, sometimes nicely modularized. Would anybody appreciate if I
created an outline of the general categories of software I have and
would be willing to maintain as described above? I'd spend a little
bit of time to survey what I have and try to figure out how to
organize the topics and then I'd list the major categories of
software tasks my software modules perform/solve, if somebody
showed genuine interest. Contrarywise, is there anyone else who
would be willing to make a similar effort if I or anyone else
showed genuine interest? If several people (including myself) did
indeed create a topic-outline of their own software modules, would
these people appreciate if I merged all the outlines into a single
outline?
> poverty, many intelligent/skilled/talented but unable to find
> anyone willing to pay them for their services.
i suspect you overestimate number of such people. of course
many get totally inadequate pay, but even such pay is substantial
comparing to what random freelancing (that's a term for free
programmers available on demand for small projects) can give.
> Unfortunately, although I realized as long ago as 1970 the need to
> somehow produce a decent information retrieval system for mapping
> initial queries into an organized information network, even today
> hardly anyone considers this to be a serious problem worth putting
> to practice.
huh, really? recently microsoft bought some intelligent search
engine startup for $100M. i think they exactly are _trying_
to solve this information retrieval problem.
you see, search engine tech is sort of hot this days.
however, sadly advanced info-retrieval methods offer
very little comparing to conventional search engines.
but it's not like there is no research at all..
> search engines Google and Yahoo which have simply punted on the
> problem, providing keyword search services that simply return a
> list of all matches, in an unorganized linear sequence, instead of
> organizing the results in any way per *meaning* of the terms.
by the way, i was working on a service that finds tags (sort of
categories) for web pages, and i have a demo how it works for
google search results (unfortunately demo requires Javascript):
http://we-tag.info/proxy/google.com/search?q=45
for example for search result like this:
45 iPod Cases: Digital meets Analog
5 ipod cases - the art of record folding. ... 45 iPod Cases are made from
reclaimed music media rebuilt to fit your iPod. iPod Nano 1st-2nd
generation ...
www.45ipodcases.com/ - 2k
it finds tags: (apple, music, hardware, technology, audio)
it doesn't work very good for now, but clearly shows some understanding,
at least in some topics. and i see this could be used for enhancing
search results..
> I'd like your opinion whether these categories to any degree
> whatsoever partly satisfy the need to find software that performs
> certain kinds of tasks as well as documentation and customer
> support for the software.
they don't have separate category for libraries, and they have
too few sub-categories, so i'm afraid this is not useful --
it's easier to search by keyword than browse directory.
if you are looking for a program rather than library this directory
makes some more sense.
> If that Yahoo category turns out to be useless for our needs, would
> anybody be willing to brainstorm with me to develop a decent system
> for organizing this kind of information?
yep this might be interesting for me, for example.
sometimes i was thinking of making sort of global database for
everything, and making decent database for software libraries
might be a good starting point.
i'll think about this a bit
> By the way, I've written a lot of software, mostly for my personal
> use, sometimes nicely modularized. Would anybody appreciate if I
> created an outline of the general categories of software I have and
> would be willing to maintain as described above? I'd spend a little
> bit of time to survey what I have and try to figure out how to
> organize the topics and then I'd list the major categories of
> software tasks my software modules perform/solve, if somebody
> showed genuine interest.
if they do not know what software you have written (and how),
how would they know if they have genuine interest? :)
as i've already mentioned, it's good to have large quantities
of libraries, so you can pick one once you need. but chances
that you can reuse one specific piece of software is pretty
low.
i guess in this case the more the better, so if you make
such list, people might occasionally find it via search
engine or something.
> Contrarywise, is there anyone else who
> would be willing to make a similar effort if I or anyone else
> showed genuine interest?
well, as for me, i don't have much to show: what is in
good form, either is already in open source (or is proprietary piece),
and stuff i didn't bother to put in a good form are too messy
to show them. and i suspect many people are in situation like this
-- stuff that is worthwile is already published, and the rest
will be hidden forever.
> If several people (including myself) did
> indeed create a topic-outline of their own software modules, would
> these people appreciate if I merged all the outlines into a single
> outline?
i don't know about that hypothetic people, but certainly sort
of bigger overview is typically better -- despite we have search
engines, it's sometimes very hard to find information if it's
not organized properly.
>> let's check other stuff. maybe there is at least such a simple
>> thing like bayesian classifier? google search for "lisp bayesian
>> classifier"
>> again finds some articles and papers, but not actual library (neither
>> there is one at cl-user.net). while finding Python code (or actual
>> utilities working from command line) for doing bayesian classification
>> is very easy.
> Why not just call the Python code?
you mean via FFI? doesn't make sense. bayesian classifier is quite
a simple thing, so it's more effort to make interface than
calculations itself. but nevertheless, both calculations
and interface take some time to implement, with debugging
and testing it can be a week or so.
in my case, i just wanted to test if methods i'm using
are better than classical classifiers for specific tasks
i was doing. in such situation library would be of great
help, because wasting week on just testing/evaluation is
not good.
OTOH if bayesian classifier is an essential element
of application, i see no problems spending time on
implementing it -- so it will be better tailored
to meet performance and space requirements, for example,
or have some ehancements.
In article <·················@your-a80c79e8b3>,
"Alex Mizrahi" <············@gmail.com> wrote:
> >> let's check other stuff. maybe there is at least such a simple
> >> thing like bayesian classifier? google search for "lisp bayesian
> >> classifier"
> >> again finds some articles and papers, but not actual library (neither
> >> there is one at cl-user.net). while finding Python code (or actual
> >> utilities working from command line) for doing bayesian classification
> >> is very easy.
>
> > Why not just call the Python code?
>
> you mean via FFI? doesn't make sense. bayesian classifier is quite
> a simple thing, so it's more effort to make interface than
> calculations itself. but nevertheless, both calculations
> and interface take some time to implement, with debugging
> and testing it can be a week or so.
>
> in my case, i just wanted to test if methods i'm using
> are better than classical classifiers for specific tasks
> i was doing. in such situation library would be of great
> help, because wasting week on just testing/evaluation is
> not good.
>
> OTOH if bayesian classifier is an essential element
> of application, i see no problems spending time on
> implementing it -- so it will be better tailored
> to meet performance and space requirements, for example,
> or have some ehancements.
If I Google for "bayesian classifier" and "common lisp" I get
several promising hints where people have reported
about their implementations. One is from NASA, another one
is a paper from the ILC 07, ...
http://www.cs.bilkent.edu.tr/~guvenir/courses/CS550/Programs/AutoClassII/
http://arxiv.org/ftp/cs/papers/0211/0211003.pdf
http://www.international-lisp-conference.org/2007/speakers
Usually what I do when I'm interested in some software, I'll
read the reports about it and then I try to contact the
authors...
--
http://lispm.dyndns.org/
> If I Google for "bayesian classifier" and "common lisp" I get
> several promising hints where people have reported
> about their implementations. One is from NASA, another one
> is a paper from the ILC 07, ...
if it's not packaged as a ready-to-use library, effort required
to find working implementation and use it will be comparable
with effort of writing it itself, or porting from other language,
at least for a relatively easy thing like bayesian classifier.
bayesian classifier can be written in something like 30 lines of
code and is definitely not a rocket science. if it deserves a
report or a paper, it's likely that it has enhancements or
is applied in some funky way. as you know people rarely
write in flexible way if they are doing code for demonstration
or research, so reusing it might be a challenge.
i know this because we wrote classifier ourselves. while it was
written in quite a good and clean style and with some degree of
flexibilty in mind, it was intergrated with guts of system that
reads corpora of text, so if i'll post code here, i'm sure
people will find it quite cryptic, and it will take considerable
effort to make it doing what they want.
> Usually what I do when I'm interested in some software, I'll
> read the reports about it and then I try to contact the
> authors...
this is not supposed to be fast -- it can take weeks to find
well burried information, communicate with authors, evaluate
different options, try adapting it for system you have etc.
sure, it makes sense if that software is really complex
and important. but if you have relatively typical problem,
it would be strange to wait that long before you can use a
library. for such cases ready-to-use libraries are important
-- you can save time and get results faster with them.
In article <·················@your-a80c79e8b3>,
"Alex Mizrahi" <············@gmail.com> wrote:
> > If I Google for "bayesian classifier" and "common lisp" I get
> > several promising hints where people have reported
> > about their implementations. One is from NASA, another one
> > is a paper from the ILC 07, ...
>
> if it's not packaged as a ready-to-use library, effort required
> to find working implementation and use it will be comparable
> with effort of writing it itself, or porting from other language,
> at least for a relatively easy thing like bayesian classifier.
>
> bayesian classifier can be written in something like 30 lines of
> code and is definitely not a rocket science. if it deserves a
> report or a paper, it's likely that it has enhancements or
> is applied in some funky way. as you know people rarely
> write in flexible way if they are doing code for demonstration
> or research, so reusing it might be a challenge.
might, likely, rarely, ...
>
> i know this because we wrote classifier ourselves. while it was
> written in quite a good and clean style and with some degree of
> flexibilty in mind, it was intergrated with guts of system that
> reads corpora of text, so if i'll post code here, i'm sure
> people will find it quite cryptic, and it will take considerable
> effort to make it doing what they want.
so you are requesting that Common Lisp gets more ready-to-use
libraries and you are not posting the software you wrote?
Strange.
>
> > Usually what I do when I'm interested in some software, I'll
> > read the reports about it and then I try to contact the
> > authors...
>
> this is not supposed to be fast -- it can take weeks to find
> well burried information, communicate with authors, evaluate
> different options, try adapting it for system you have etc.
>
> sure, it makes sense if that software is really complex
> and important. but if you have relatively typical problem,
> it would be strange to wait that long before you can use a
> library. for such cases ready-to-use libraries are important
> -- you can save time and get results faster with them.
--
http://lispm.dyndns.org/
> might, likely, rarely, ...
that's my experience with it. using software that is not
made ready to use is a considerable effort (obviously -- if
authors didn't do that, you have to), and often is not feasible.
> so you are requesting that Common Lisp gets more ready-to-use
> libraries and you are not posting the software you wrote?
1. i'm not requesting. i'm saying it will be a benefit,
in some cases at least; while other guys are saying that
everything is excelent right now
2. i'm releasing my code into open source wherever it makes
sense. for example, i've released ABCL-web. also now i'm
sort of maintainer of postgresql backend of Elephant
object persistence library.
3. as i've mentioned making piece of software fully reusable
requires additional efforts (and also it makes it less
integrated or coupled with the rest of program), and
unfortunately i don't have enough time to make everything
i write this way. and some some i write is, sigh,
proprietary. you know, people need to eat from time to
time, and making open source lisp libraries does not
automatically generate money :)
4. so i wonder, do i have to release _everything_ to
open source, or i can limit contribution to, say,
1000 LoC per month and still be proudly called open
source supporter?
> From: "Alex Mizrahi" <············@gmail.com>
> using software that is not made ready to use is a considerable
> effort (obviously -- if authors didn't do that, you have to), and
> often is not feasible.
You are considering only two options:
- Original author has moved onto a new project and is too busy to
go back to old software and portablize it.
- New potential user who is starting a new project and has very
little time available to evaluate software modules that might be
useful, surely can't spend a week just to evaluate one (1)
module, has to rush it, decide in a half hour whether to use it
or not, can't afford to spend six weeks refactoring the software
module before evaluating it.
You are not thinking of some third party "adopting" the software
module as I described in another article a couple hours ago. Now
that I've mentionned my idea three times (the original with text
"There are over six billion people ...", first re-mention with text
"database and query engine", and this here), what do you think of
my idea? Would it be helpful if I collect the text from my three
articles (so-far) into a more coherent single Web page formally
proposing the software-adoption system (which I forgot to mention
before would *also* involve critical review of software, sort of
like Consumer Reports and like the customer feedback on eBay)? Or
is my idea so stupid it'd be a waste of effort to write it up
better and you wouldn't read my Web page if I made one?
> ... i'm releasing my code into open source wherever it makes
> sense. for example, i've released ABCL-web. also now i'm sort of
> maintainer of postgresql backend of Elephant object persistence
> library.
Have you already created a master table-of-contents which organizes
the various types of software you've created (or you've "adopted"
and are now actively maintaining) and clearly tags each item as to
whether it's public domain etc. or whether it's for sale at
reasonable price or it's proprietary and you're letting us know
about just for our curiousity and in case we want to license it
from whomever you wrote it for?
> as i've mentioned making piece of software fully reusable
> requires additional efforts (and also it makes it less integrated
> or coupled with the rest of program),
How is being "integrated" to the point where it's impossible to
re-use any of the code a *good* thing?? It seems to me that it's
generally a good thing to refactor the syntax to pull out various
aspects of the code to make each of them independently reusable,
and even *your* project might benefit from that effort by allowing
you to adapt the code for a slightly different application yet need
to maintain just one (1) copy of the re-used module instead of
having duplicate code in multiple places where a tweak one place to
fix a subtle bug requires rummaging around in all the copies to
make the same tweak, or else have copies that diverge from each
other so that one application persists in having a bug that was
already fixed because it's too much trouble locating the place in
the copy of the code that needs the fix applied. Didn't you believe
the hype twenty or thirty years ago about how good "modular
programming" was supposed to be. Now, with GUI text editors with
cut/copy/paste its easire than ever to perform syntactic
refactoring, hence it's even more good than ever (better
benefit/cost ratio) than it was way back then, so the hype is
really true now! Don't you believe in "modular programming"? Don't
you believe in refactoring syntax toward the goal of re-usable
software components formerly known as "modular programming"?
> unfortunately i don't have enough time to make everything i write
> this way. ...
So how about you post a Web page that categorizes most of the
software you're making available (or would like to make available
except it's not modularized well enough so you would be ashamed to
release it as-is), and solicit other people to volunteer to "adopt"
one or more of your software modules and perform whatever syntax
refactoring would be desired that you don't have time to do
yourself?
> so i wonder, do i have to release _everything_ to open source, or
> i can limit contribution to, say, 1000 LoC per month and still be
> proudly called open source supporter?
This is where you really sound like you're boasting. I was thinking
of posting this followup, but this boasting is what convinced me
that I really ought to challenge you to put your ability to present
good information where your mouth is. Is that 1000 LoC/mo what
you've already been doing (which I wasn't aware of) or only
something you are *potentially* capable of doing when challenged to
put your LoC where your mouth is? This is an actual question, not
rhetorical, I honestly don't know whether you've released 1000
LoC/mo or not in recent months/years.
One more question to satisfy my curiosity: After you've written
some software, how easy do you find it to acquire somebody as a
beta tester to give you feedback on your software while it's still
fresh in your mind and you're still actively working on it? My own
personal situation is that I almost *never* can find anybody to
even glance at the software I write, much less seriously beta-test
it. I was able to find only one person in RL to seriously try my
very first CGI demo in early 2001, and not yet even one person at
all in RL to seriously try the more serious CGI applications I
wrote starting in late 2002. <cliche>YMMV, so I'm curious how the
beta grass is on your side of the fence.</cliche>
> You are considering only two options:
> - Original author has moved onto a new project and is too busy to
> go back to old software and portablize it.
> - New potential user who is starting a new project and has very
> little time available to evaluate software modules that might be
> useful, surely can't spend a week just to evaluate one (1)
> module, has to rush it, decide in a half hour whether to use it
> or not, can't afford to spend six weeks refactoring the software
> module before evaluating it.
note that it was in context of bayesian classifier -- relatively
small piece. if we were speaking about some complex piece of
software, it might make sense to reuse it even if it's in
a bad condition.
> Now
> that I've mentionned my idea three times (the original with text
> "There are over six billion people ...", first re-mention with text
> "database and query engine", and this here), what do you think of
> my idea?
honestly, it's good in theory, but won't work in practice for
various reasons:
1. there are competing systems like freelancing sites
and Sourceforge marketplace. on freelance sites
you can find coders (or teams of thereof) for some
specific project. usually they are organized in some way,
like "php projects", "cms" etc.
as you know, sourceforge
hosts lots of open source projects and libraries.
and they've made a button "get support for this project".
they have a sort of database of people who specified
that they have expertise in some area and agree to work
for money. so it's like freelance sites, but tied more
to open source.
2. if you'd like to make such system for lisp, it won't
work because of lack of popularity -- too few companies,
too few people, and people often already know each other
and libraries without any database
3. and making it for other languages/technologies is complex
for obvious reasons.
while i bet it won't work as you expected, in might work fine in
some other form -- for example, if you'll position it not as
another freelancing marketplace, but as some database of libraries
(with some additional features like marketplace, maybe).
we already have cl-user.net that indexes most lisp libraries out there,
but i think it misses lot of vague and weird stuff, and search
could be more detailed. so, to start with, you could be
making a database of such vague and weird stuff, like artifacts
from different research projects, code coming with papers, code]
posted into newsgroups etc.
> Would it be helpful if I collect the text from my three
> articles (so-far) into a more coherent single Web page formally
> proposing the software-adoption system
belive me or not, people are often scaried by "formal" stuff.
if you really want people to read it, make a short description
with feature list etc, and sort of short example of how it
could look like -- example often says more than large formal
description
> before would *also* involve critical review of software, sort of
> like Consumer Reports and like the customer feedback on eBay)
good idea
> Or
> is my idea so stupid it'd be a waste of effort to write it up
> better and you wouldn't read my Web page if I made one?
no i think this makes sense, although in some ways a bit detached
from reality
> Have you already created a master table-of-contents which organizes
> the various types of software you've created (or you've "adopted"
> and are now actively maintaining) and clearly tags each item as to
> whether it's public domain etc. or whether it's for sale at
> reasonable price or it's proprietary and you're letting us know
> about just for our curiousity and in case we want to license it
> from whomever you wrote it for?
my list would be sort of too short to be made with this kind
of sophistication :)
although i see your point..
> How is being "integrated" to the point where it's impossible to
> re-use any of the code a *good* thing??
i suggest you to read "Patterns of software" by Richard Gabriel.
(if you haven't done it yet)
in first few essays he says lots of interesting things about
abstractions and reuse -- like they are not panacea.
for me, good thing is when software does what it should.
being reusable is a secondary goal -- it might or might not
be important.
as Gabriel says, what's important is if software is "habitable"
and suitable for incremental modifications. even if piece
of code is not reusable as is, it might be trivial to copy
and modify -- and that's good.
as example (again from Gabriel), MISMATCH function in
Common Lisp -- it works
in plenty of modes. but isn't it as easy to write a loop
as to write MISMATCH? i think they are quite comparable.
but if you need to have some change that does not fit
in what MISMATCH can do, you'll have to throw out
MISMATCH code and write it as a loop. but if you had a loop,
you could just do a minor modification. so it's not that
obvious that having a MISMATCH abstraction makes sense..
> It seems to me that it's
> generally a good thing to refactor the syntax to pull out various
> aspects of the code to make each of them independently reusable,
i thought like that five years ago or so :) -- that reusability is
always good.
now i think that is only relatively good. there is no such thing
as 100% reusability. and even if you make piece 95% reusable,
there are still 5% of cases where you'd need to severely alter
or rewrite code. and getting 95% reusability is _hard_. so does
it make sense -- it depends.
> and even *your* project might benefit from that effort by allowing
> you to adapt the code for a slightly different application
or it might be more simple just to copy and adapt that thing..
> yet need
> to maintain just one (1) copy of the re-used module
but if i have two copies they can diverge -- like i can implement
specific stuff in one of projects using vanilla stuff in another.
> instead of
> having duplicate code in multiple places where a tweak one place to
> fix a subtle bug requires rummaging around in all the copies to
> make the same tweak, or else have copies that diverge from each
> other so that one application persists in having a bug that was
> already fixed because it's too much trouble locating the place in
> the copy of the code that needs the fix applied.
yep, that's known problem too. i think this can gradually evolve
with time -- first you have module that is not abstract, then
you see how you can use it as abstraction and make in reusable.
or maybe you won't return to that module again, and so
additional efforts won't be wasted..
> Don't you believe in "modular programming"?
no, i don't. it had quite enough time to show it's benefits,
and it did not. reuse across projects is typically pretty low,
as actual surveys demonstrate.
> Don't
> you believe in refactoring syntax toward the goal of re-usable
> software components formerly known as "modular programming"?
not in reuse-just-for-reuse, but in cases where it's clearly
seen that part can be reused, this of course make sense.
trick here is that only some low percent of code
actually make sense to be reused.
> release it as-is), and solicit other people to volunteer to "adopt"
> one or more of your software modules and perform whatever syntax
> refactoring would be desired that you don't have time to do
> yourself?
well, i see you point, but it looks a bit weird why i should
solicit people to "adopt" _my_ modules?
> This is where you really sound like you're boasting.
well, it was meant to be not boasting but just making fun..
> I was thinking
> of posting this followup, but this boasting is what convinced me
> that I really ought to challenge you to put your ability to present
> good information where your mouth is. Is that 1000 LoC/mo what
> you've already been doing (which I wasn't aware of) or only
> something you are *potentially* capable of doing when challenged to
> put your LoC where your mouth is? This is an actual question, not
> rhetorical, I honestly don't know whether you've released 1000
> LoC/mo or not in recent months/years.
no. i think some monthes i could be doing something of that order
of magnitude (i think more realistic would be 400-500 LoC), but in
other monthes i could release nothing. or even write nothing --
for example, recently i've spent a month setting up and configuring
a server.
> One more question to satisfy my curiosity: After you've written
> some software, how easy do you find it to acquire somebody as a
> beta tester to give you feedback on your software while it's still
> fresh in your mind and you're still actively working on it?
well, that depends. i can't say that's easy to me, but some stuff
i've done was tested a bit by friends and people i know. some even
was downloaded by thousands of people from internets,
and they sent me actual bug reports :). some didn't get much attention
though..
> From: "Alex Mizrahi" <············@gmail.com>
> ... bayesian classifier ... we wrote classifier ourselves. while
> it was written in quite a good and clean style and with some degree
> of flexibilty in mind, it was intergrated with guts of system that
> reads corpora of text, so if i'll post code here, i'm sure people
> will find it quite cryptic, and it will take considerable effort to
> make it doing what they want.
Why haven't you refactored the syntax to clearly separate the
specification of the Bayesian classifier algorithm per se from the
other elements of your application (specific access to corpora of
whatever your text was, specific interface to system guts,
user-interface, etc.)? With macros and first-class functions/closures,
it's possible to syntactically separate two or more aspects of
software without making it more than slightly less CPU-efficient.
You build an API for each aspect, then have a toplevel "script"
that configures each aspect and hooks them all together into an
application or tool.
That's a genuine question, not a demeaning remark. I expect an
answer such as one of these:
- I work only for money, and nobody has offered me money to
refactor the syntax of that software. Show me $70/hr and I'll
gladly do some work on that task.
- I would do it for free, but nobody showed any interest, and I
have better things to do with my unpaid time than do something
that is no value to me personally and of no interest to anyone
else.
- I don't have that skill, of analyzing software I wrote and
figuring out how to capture the essence of various aspects of it
in separate syntax as you say above. I wouldn't know where to
begin. I never write Model-View-Controller software because I
wouldn't have any idea how to refactor software per that model.
- You need to write up detailed specification of what kind of
refactoring you need before I'll understand your requirements
well enough to perform the requested refactoring. What you said
above is just not clear enough.
- I think it's a stupid suggestion. It's a waste of mental energy
to even consider clearly separating the syntax as you suggest.
- I considered it, but Lisp macros etc. aren't powerful enough.
Refactoring like that can be done only in other programming
languages that support such division of syntax at the core
language level. Trying to hack it in Lisp would be doomed fo
failure.
Maybe your answer isn't represented among my guesses above?
So please give *your* answer why it's not done already?
> > Usually what I do when I'm interested in some software, I'll
> > read the reports about it and then I try to contact the
> > authors...
> this is not supposed to be fast -- it can take weeks to find
> well burried information, communicate with authors, evaluate
> different options, try adapting it for system you have etc.
I agree. That's why I suggested a database and query engine for
(respectively) registering and locating people who have eagerly
volunteered to provide live customer support for existing software
they wrote or have "adopted". So if you find somebody in the
database willing to show you the documentation and answer simple
questions about it and update the FAQ with the new answers, you can
be confident it won't take months just to establish contact with them.
> Why haven't you refactored the syntax to clearly separate the
> specification of the Bayesian classifier algorithm per se from the
> other elements of your application (specific access to corpora of
> whatever your text was, specific interface to system guts,
> user-interface, etc.)?
because it is more work to do. as i've mentioned, bayesian
classifier was used only for testing and evaluation, so
adding abstractions would be over-engineering.
(also i'd like to note that bayesian stuff was not implemented by me
personally. i said _we implemented_ -- that means it was somebody
else.)
> - I would do it for free, but nobody showed any interest, and I
> have better things to do with my unpaid time than do something
> that is no value to me personally and of no interest to anyone
> else.
yep, i have lots of interesting stuff to do, and somehow i assumed
_some_ bayesian code exists in public access, it can be just
somewhat hard to find. so i haven't seen any value in polishing
and releasing a lib.
> - I work only for money, and nobody has offered me money to
> refactor the syntax of that software. Show me $70/hr and I'll
> gladly do some work on that task.
sure, for $70/hr i will do that :)
> begin. I never write Model-View-Controller software because I
> wouldn't have any idea how to refactor software per that model.
MVC has nothing to do with it. code just depends on specific data
representation (i.e. we represent document as hashtable
word->frequency and keep them in hashtable doc_id->doc_ht;
we map words to IDs to avoid expensive hash lookups in run time),
and it reuses some parts of application (i.e. some values useful
for a bayes algorithm were already calculated in other parts of
software).
so it's a problem of interface -- either we make interface
hard, assuming that user code implements parts we've missed.
or we make interface simple, but loose integration with our
application -- either we'll need to implement stuff twice,
or do some massive refactoring of application.
first option is silly (who needs library with weird interface?),
second requires considerable effort (to refactor all the stuff..).
so..
> I agree. That's why I suggested a database and query engine for
> (respectively) registering and locating people who have eagerly
> volunteered to provide live customer support for existing software
> they wrote or have "adopted".
> So if you find somebody in the
> database willing to show you the documentation and answer simple
> questions about it and update the FAQ with the new answers, you can
> be confident it won't take months just to establish contact with them.
actively maintained projects often have mailing lists where you can
find help.
for non-maintained projects those people who can do support
simply do not exist
On Jul 8, 7:28 pm, ··················@spamgourmet.com.remove (Robert
Maas, http://tinyurl.com/uh3t) wrote:
> I agree. That's why I suggested a database and query engine for
> (respectively) registering and locating people who have eagerly
> volunteered to provide live customer support for existing software
> they wrote or have "adopted". So if you find somebody in the
> database willing to show you the documentation and answer simple
> questions about it and update the FAQ with the new answers, you can
> be confident it won't take months just to establish contact with them.
I find this idea rather promising. At least it would be good to have a
site (or, maybe, some distributed system), which had:
1. A lisp bot, so people could post any code (and, you're right, that
it's essential, *categorize* and tag it, as well as annotate it's
status etc.)
2. Some kind of a profile system with a least a list of hacker's
posted code, his means of contact and customer-support options (as
you've mentioned). Besides there could be links to the libraries and
their maintainers as well (but here the situation is better -- many
libraries have their own web-sites or at least a page on cliki.net)
Moreover, such site can easily become an online community for lisp-
hackers, if you add blogs and discussions. As well some reputation
system might be used.
Vsevolod
V> 1. A lisp bot, so people could post any code (and, you're right, that
V> it's essential, *categorize* and tag it, as well as annotate it's
V> status etc.)
what do you mean by "a lisp bot"?
On Jul 12, 2:31 pm, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> V> 1. A lisp bot, so people could post any code (and, you're right, that
> V> it's essential, *categorize* and tag it, as well as annotate it's
> V> status etc.)
>
> what do you mean by "a lisp bot"?
paste.lisp.org
REM> I agree. That's why I suggested a database and query engine
REM> for (respectively) registering and locating people who have
REM> eagerly volunteered to provide live customer support for
REM> existing software they wrote or have "adopted". So if you find
REM> somebody in the database willing to show you the documentation
REM> and answer simple questions about it and update the FAQ with
REM> the new answers, you can be confident it won't take months
REM> just to establish contact with them.
Vsevolod <········@gmail.com> wrote:
Vs> I find this idea rather promising. At least it would be good to
Vs> have a site (or, maybe, some distributed system), which had:
Vs> 1. A lisp bot, so people could post any code (and, you're
Vs> right, that it's essential, *categorize* and tag it, as well as
Vs> annotate it's status etc.)
I have a grand scheme of setting up a "club" which requires
contribution before benefit. The very first step to join the "club"
is to pass a Turing test to prove you aren't a spambot. After that,
you bid on contracts to perform various useful tasks.
(What you bid is the amount of elapsed it take you to complete a
task, from first getting the contract to submitting the final
product to satisfy the contract. Lowest bidder gets first dibs at
a contract.)
Whenever you complete a contract, you then have "cash" (time)time
in your "pocket" (account), which you can later spend to purchase
services that somebody else spent time to create, or to repurchase
somebody else's futures contract. Yesterday I thought this out and
wrote up lots of tentative (first-draft) specs of such a system.
Let me know if you're interested in discussing the details and
possibly working with me to develop such a system.
Within that system, a club member could submit a file containing a
description of a software module that is available, and a club
member (usually somebody else) could propose "strong keywords" for
the topic (per the jargon of my circa 1973 proposal for
"Generalized Computer Dating" = GCD for clustering all
forms of data per topic, "weak" keywords are those which
occur naturally in text while "strong" keywords are
assigned specifically by people acting as "librarians"
to provide better keywords for better automated
clustering)
and various people could make bids on the future "price"
(correctness) of such keywords, and a query engine would use the
estimated correctness in that way to weight its algorithms towards
the high-estimated-value keywords and away from low-estimated-value
keywords, but then relevance feedback from users would determine
the "true" price hence the payoff for investors.
All/most contracts for services, such as assigning "strong"
keywords, would be awarded to the lowest bidder (the person who
said they could do the job in least time hence earn least
time-credit as a result of well-done work) as first person to get a
chance on the contract. If somebody does a shoddy job at any
contract, the person would get paid less than bid, and others could
bid on contracts to fix what was wrong with the the original work,
just like fixing bugs. (Like remember the people who got paid big
bucks to fix y2k bugs?)
So let me know if you want to discuss my specific ideas for the
strong-keyword-tagging system.
After weak keywords are used to do preliminary organization to make
the process of bidding on strong-keyword-assignment contracts
reasonably efficient, then strong keywords are assigned and used to
drive a search engine and clustering, those same strong keywords
could be used to make the process of defining a hierarchy of
"intentional datatypes" more efficiently, with the keyword
clustering doing most of the work of generating first-draft
hierarchies which then require only a small amount of manual tuning
to get the very best hierarchy. Note one nice thing about strong
keywords: They can be tagged words instead of just barebones
natural language words. For example, a keyword can be tagged to
show that it refers to the name of an intentional datatype, or to
show that one intentional datatype is a subtype of another, etc.
It's sort of like how HREFs in HTML documents tell where to find
sub-nodes and parent nodes without the entire Web of documents
**really** being in any actually tree structure.
So let me know if you want to discuss my ideas for intentional
datatypes as the primary paradigm for classifying/organizing
software modules.
Of course an additional useful services people can provide is
consumer feedback and expert evaluation of contributed software
modules, regardless of whether the modules are contributed as open
source, or confidential but source provided to anyone who pays for
them, or permanently confidental and residing on a server that
talks XML or SEXPR to support leased RJE/RPC/RMI service. Futures
market on feedback/critique can help reduce warfare where some
people badmouth perfectly good software from their competitors.
Vs> 2. Some kind of a profile system with a least a list of
Vs> hacker's posted code, his means of contact and customer-support
Vs> options (as you've mentioned).
I wish I could find others interested in a new e-mail system based
on pseudo-SMTP using HTTP with CGI/PHP instead of our current
spam-prone SMTP. The e-mail system would feed into an
event-prioritizing system so that your hot screen always shows you
the most urgent/important events you need to respond to. Then we
could provide expedited e-mail service without anyone suffering
spam. Anyone could run their own instance of a pseudo-SMTP client
on their own Web site using PHP or CGI, where they could establish
their own anti-spam measures however they choose, if they don't
like the anti-spam measures I set up on my server. And the
public-key encrypted&signed system of passing alerts from one
system to another, again by HTTP transactions, would be totally
flexible from the start as to who is allowed to signal events at
what maximum priority level.
(Like if you trust your girlfriend not to call emergency except
when it's really an emergency, and she has a device that she can
use to signal a dire emergency such as mugging or car ran off the
road or house on fire, then you would allow your personal alert
server to accept such high priority alerts from her and as a
result wake you out of a dead sleep at 3AM and sound a loud alarm
to get you out of the shower immediately and turn off your music
system to allow you to hear the alarm etc. But you wouldn't grant
such an alert level to your very best friend who has *other*
people to help him in an emergency, although you'd still like to
hear a milder alert sound in the shower just to know you should
finish up quickly, and you wouldn't grant any high alert
whatsoever to hardly anybody else you don't really trust.)
Of course some people wouldn't use our new e-mail system, so we'd
index their availability per older technologies, such as FAX
number, secretary's phone number, or even a reverse-tree. See
<http://shell.rawbw.com/~rem/WAP/projectIdeas.html> for some info
about various projects I'd like help with, including several I've
mentionned above. This Web site uses very very small Web pages
suitable for mobile InterNet over cellphones.
Vs> Besides there could be links to the libraries and their
Vs> maintainers as well (but here the situation is better -- many
Vs> libraries have their own web-sites or at least a page on
Vs> cliki.net)
For libraries with a well-written Web site, all they really need to
submit to my proposed system would be the URL together with a tag
identifying what kind of Web site it is (cliki or blog or regular
Wiki or handcrafted HTML or whatever). And the tag could be
assigned later if the original contributor is too busy to even
bother with that, or if submissions were harvested by spidering a
large Web site such as cliki.net. (The host name might serve as a
weak keyword in such cases to aid the tagging process.)
One really important thing in any meta-info site such as ours would
be to keep a list of everything we already have indexed, and
automatically compute a checklist of items we don't yet have
indexed within each large Web site, so that people don't waste time
re-submitting something we already know about. There could be a
hierarchial directory view of such a Web site, showing at each node
of the tree how many modules are located within that sub-tree and
how many of them are not yet indexed in our system.
> Moreover, such site can easily become an online community for
> lisp- hackers, if you add blogs and discussions.
I can't do that here because I'm on a Unix shell account with
virtually no free disk space. I would have only the very top level
of the index directly hosted on my shell account, with volunteers
"adopting" sub-trees of the overall hierarchy to be hosted on their
Web sites instead. At the moment I can't even set up a MySQL
database until I clear out 40 megabytes, the smallest DB size
allowed here.
> As well some reputation system might be used.
I think a future's market is the best way to handle this. Anybody
who does lots of stupid things simply loses their investment. Your
reputation is your accumulated wealth, not some words somebody
wrote that others may or may not believe. The **big** difference
between my market and traditional markets is that:
- People who are born rich can't pay cash to buy their way into the
market, and can't throw large amounts of money around to
manipulate the price of a futures commodity. It's all based on
labor rather than money, and only the most consistently good
labor (as judged by buyouts by other investors, and relevance
feedback by users, and automated acceptance tests by purchasers
of software) results in somebody building up a lot of time
credits.
- People who don't have money can join just the same as anybody
else, merely by investing their *time* to build up time-credits
in their account, so the rich cats can't lock out the poor
folks. It won't be like the current labor market, where
unemployed people can't get jobs because they don't have recent
work experience. Anyone with access to the net, even at the
public library or homeless drop-in center or Connect/NOVA, will
be able to offer to do work by bidding on a contract.
On Mon, 07 Jul 2008 12:20:24 +0300, "Alex Mizrahi"
<············@gmail.com> wrote:
>ok, let's leave web field for a while. i'm interested
>in computational linguistics. are there libs for it?
>i'm pretty sure there are some old ones from 1980',
>made by students, with questionable quality, but are
>there modern and good ones, with support for wide range of
>implementations? haven't seen such.. (have seen such for Python,
>though, and they say there are libs for C..)
>
The Linguistic Knowsledge Builder is implemented mostly in Common
Lisp. It's not a library or the the web 2.0 kind of NLP you seem to
looking for though. The StatNLP parts are handled by the usual
suspects (GIZA++, CMU LM, TADM, SVMlight etc) that are implemented in
C++.
I've been thinking about doing a GIZA++ reimplementation in a higher
order language for a while now. I think this would be very useful
since it is the cornerstone software for a lot of SMT related work and
it is the kind of C++ software with the temper and idiosyncracies I
guess we all know and love ;) Drop me a line if you're interested in a
project such as this or something similar.
On Mon, 07 Jul 2008 15:37:36 +0200, sinnatagg <andrely _ ifi uio no>
wrote:
>The Linguistic Knowsledge Builder is implemented mostly in Common
>Lisp. ...
Forgot the link to LKB...
http://wiki.delph-in.net/moin/LkbTop
-a
thus spoke Alex Mizrahi <············@gmail.com>:
>> So I'd like to propose a challenge for those, who hold to this
>> viewpoint, to bring up specific examples of situations, when a decent
>> library for some common task couldn't be found for Common Lisp.
>> I
>> think the answers could serve as a road-map for the potential library
>> developers for Lisp (and not just for my and, possibly, other people
>> enlightenment :)
> to start with, there is no good object persistence
> libraries IMHO. [...]
cl-perec's usable if a bit undocumented. Its MOP's pretty good for
introspecting on stuff.
> Drakma HTTP client is assumed to be of quite high quality, but
> unfortunately it's unusable for automated tasks like site monitoring
> and spidering. both this tasks require a timeout parameter (for both
> connection and read operations) -- otherwise remote site that is
> either down or evil will hang your process indefinitely.
Check out the patch I posted on drakma-devel a while ago if SBCL's what
you're using. Doing it portably with usocket conses too much, though :/
--
The great peril of our existence lies in the fact that our diet consists
entirely of souls. -- Inuit saying
On Jul 6, 11:35 pm, Vsevolod <········@gmail.com> wrote:
> I have noted, that people use to say, that Common Lisp environment
> lacks good library support. For me that doesn't seem obvious, as in my
> 3 years' experience with the language I've never encountered such
> situation.
>
> So I'd like to propose a challenge for those, who hold to this
> viewpoint, to bring up specific examples of situations, when a decent
> library for some common task couldn't be found for Common Lisp. I
> think the answers could serve as a road-map for the potential library
> developers for Lisp (and not just for my and, possibly, other people
> enlightenment :)
I don't disagree with you. But what we further need is an easily-
accessible library repository, with well-written and well-tested and
well-documented libraries, along with various useful metadata to help
everyone find the libraries that they want.
The facility/framework I'm looking forward to is CLornucopia (http://
trac.clozure.com/clornucopia) from Clozure. There isn't much written
up yet since it's not quite up-and-running. If you come to the
International Lisp Conference 2009 you'll hear more about it, and of
course it will be advertised on the web when it's ready. I'm
optimistic that this will vastly improve the situation.
> From: Dan Weinreb <····@alum.mit.edu>
> ... what we further need is an easily-accessible library
> repository, with well-written and well-tested and well-documented
> libraries, along with various useful metadata to help everyone find
> the libraries that they want.
A few weeks ago I started seriously thinking that "intentional
datatypes" is the right way to organize types of libraries, and
since then I've been coming more and more to the conclusion that
the idea really is the right way. Yesterday I even took the
extreme/radical view that the right way to teach computer science
and software development is by starting from just two primitives,
the single bit, and the tuple of objects which can be either bits
or tuples or a mix of some of each.
(And not yet posted, until right now, but even more radical:
Everything we do with digital data should be thought of intentional
datatypes. What we **really** have is analog circuits that have
high amplification and quenching at the extremes, resulting in two
deep energy wells separated by a high barrier except that barrier
is lowered and the terrain is tipped when we want to force the bit
to change to a newly-determined state. But our *intention* of such
a circuit is that it represent a logical bit of data.
And one level higher, we have digital circuitry configured per our
*intention* to make them perform as a CPU to process binary data
whose *intention* is to represent instructions, and to sequence
these instruction-execution steps in a Von Neumann sequential
manner with occasional jumps to new non-sequential addresses.
Then at higher levels we have places in memory where a high-level
programming language specifies the *intention* of what these
memory places are supposed to represent, such as 32-bit integers
and unsigned bytes and characters etc.
So the idea of "intention" lifts us level by level from quantum
mechanics and analog circuity through RAM and CPU instructions and
Von Neumann CPUs and low-level datatypes and high-level datatypes
etc. all the way up to what our data **really** means to us as
users of the data, such as a gorgyle with weapons trying to stop us
from climbing to the highest level of a video-action game.)
So back to serious rather than ultra-radical thinking: I believe
the right way to organize libraries is in regard to what
intentional datatype(s) they primarily deal with above and beyond
that datatypes already present and taken for granted in the
underlying programming language. The advantage of thinking about
intentional datatypes is that it's somewhat orthogonal to the
question of how to actually implement stuff. A particular
intentional datatype might be implemented formally as a Java or
CLOS class, or it might just be a re-interpretation of an ordinary
data structure such as a two-level list being treated as if it were
a matrix. In a sense I'm talking about the *semantics* of the data,
as defined by how it's used as well as what it means to us. I
believe "intentional datatype" better captures the specific
category we want to use (to index libraries) rather than the more
vague C.S. term "semantics" itself. Thus "intentional datatype"
implies categorizing things, whereas "semantics" tries to capture
too much of the deep subtle meaning without any emphasis on
*classifying* whatsoever. <cliche>Sometimes just a change in
wording, using a synonym, can have the effect of channeling our
minds into appropriate thought for the task at hand.</cliche>
So if most of you folks like my emphasis on intentional datatype
for classifying libraries, I'd be willing to host the very top
level of the classification scheme, if others of you will "adopt"
various specific categories of intentional datatype within my
overall classification system. I have very little available disk
space on my shell account, so I can't host much more than the top
level of organization, somewhat analagous to the very top menus of
{http://vlib.org/ http://www.dmoz.org/ http://dir.yahoo.com/
http://www.qango.com/ http://www.onekey.com/
http://www.SearchKing.com/ http://www.about.com/index.htm or
http://members.tripod.com/~MaasInfo/MaasInfo/MaasInfo.AllTopM.html
}
except that we're already restricting our coverage to just software
libraries. Still those all-topic trees might be useful for a
toplevel categorization of some of our intentional datatypes. For
example, there might be software to aid athletic fitness, where the
datatypes might deal with blood rate and respiration and the like,
or software to aid a lawyer, where the datatypes might deal with
cases and briefs and petitions and court schedules and witnesses
etc.
Note: I found many of those above links here:
<http://web.ukonline.co.uk/martin.nail/search.htm#subjecttrees>
Here's a nice general description of this kind of tree of information:
<http://navigators.com/search_tool_intro.html#Subject_trees>
Now if I were to seriously build my own hierarchial
index-to-all-info today, replacing the MaasInfo index I created
199[1..6], I might do the very top level like this:
Science/Nature - Knowledge about the natural world
around us, understanding of how it works, and research
to increase our knowledge and improve our
understanding.
Engineering/Invention - Application of our
understanding of science and nature to design and build useful
devices and systems, both physical devices and computer software.
Social/Politics/Legal/Military - Instutitions we set up
to improve the general welfare or to improve ourselves
individually, including forms of government, laws,
legal systems to enforce the laws and protect our
rights, and social instutitions to make life more worth
living.
Religion - Beliefs based on faith rather than directly
in conformance with science/nature, for example: "I
believe the World is shaped like a cube, and I have the
right to believe that despite the satellite photos
showing it's actually more like a sphere, and nobody is
going to interfere with my right to believe that the
cubic Earth is ruled by an immortal omnipotent
omniscient phoronid with an emotional framework similar
to a two-year-old child, who throws trantrums almost
every day, which is what thunderstorms really are."
Sports/Athletics/Fitness - Improving our physical
bodies, and our state of mind in regard to them, and
competing with other people or against objective
measures to test who has the best body and has
developed that body the most.
(other) - Any new topics that don't seem to fit within
any of the other topics as they are presently defined.
OT: <http://www.jefflindsay.com/my2k.shtml>
-
Nobody in their right mind likes spammers, nor their automated assistants.
To open an account here, you must demonstrate you're not one of them.
Please spend a few seconds to study the following text and find
where a word is missing:
"On Demand gives you more flexibility
so you can watch what you want when you want."
Now type the previous word, the missing word, and the next word,
three words in sequence, separated by whitespace
(10-20 characters total), into this TextField:
+--------------------+
| |
+--------------------+