Hi,
i went through the blog
http://i-need-closures.blogspot.com/2007/09/programming-collective-intelligence-in.html
where a python code was translated into common-lisp. i couldn't grok
the presented translation of the recommendations data, where the
movies strings are repeated. the first general representation that
came to my mind as a best fit was a two tables with a third joint
table to represents the m-m relation (this is in database term. ).
however doing a similar modeling in common-lisp is not natural (i
mean you are going to build a lot of tools to support mapping and will
not leverage any by-product built in facilities). so i did the
following.
(defparameter *films* (list "Lady in the Water"
"Snakes on a Plane"
"Just My Luck"
"Superman Returns"
"You, Me and Dupree"
"The Night Listener"))
(defparameter *critics* '())
(defun add-critic (name ratings)
(setf *critics* (acons name (list ratings) *critics*)))
(add-critic "Lisa Rose" '(2.5 3.5 3.0 3.5 2.5 3.0))
(add-critic "Gene Seymour" '(3.0 3.5 1.5 5.0 3.5 3.0))
(add-critic "Michael Phillips" '(2.5 3.0 nil 3.5 nil 4.0))
(add-critic "Claudia Puig" '(nil 3.5 3.0 4.0 2.5 4.5))
(add-critic "Mick LaSalle" '(3.0 4.0 2.0 3.0 2.0 3.0))
(add-critic "Jack Matthews" '(3.0 4.0 nil 5.0 3.5 3.0))
(add-critic "Toby" '(4.5 nil nil 4.0 1.0 nil))
(defun critic-ratings (name)
(second (assoc name *critics* :test #'string=)))
(defun critic-films (name)
(loop for index from 0 upto (length *films*)
for rating in (critic-ratings name)
when rating
collect (cons (nth index *films*) rating) into results
finally (return results)))
(defun film-rating (title critic)
(let ((film-index (position title *films* :test #'string=)))
(when film-index
(nth film-index (critic-ratings critic)))
---- this is only a snippet.
Now my questionS (yes multiple of them):
1- i'm not satisfied with my solution since i'm building many small
utilities to get at the data! is this natural in common lisp (not in
general) but in this specific example?
2- also i'm noticing that my representation will have spares (NILs)
if the data sample was large? is this a problem in the long run?
3- what is the best (or pragmatic) data type to represent the above
data (aka recommendation, preferences, etc)?
4- does "best" in the above question morph through the stages of
development ? (aka one representation was good then, its not in
another)
5- when playing with data, should i store it in a store (DB,
persistent store, file) then play with it, or should i store it like
the above example (using defparameter only) what is the best
practices?
6- when i modeled the above data, i started building small utilites to
gather different data. now what is the level to which i should keep
the size of these functions, should i make them as slim as possible,
or should i make them medium (merging the above critic-ratings
function into the other two) or should i build them based on the
vocabulary required ?
finally, i hope people will drop by and share there experience! (if
any samples, links or books) and helping me reducing the time i keep
procrastinating on such questions rather than continuing with real
coding
thanks in advance.
cmo-0
On Sep 14, 2:53 pm, cmo <·······@gmail.com> wrote:
> Hi,
>
> i went through the bloghttp://i-need-closures.blogspot.com/2007/09/programming-collective-in...
>
> where a python code was translated into common-lisp. i couldn't grok
> the presented translation of the recommendations data, where the
> movies strings are repeated. the first general representation that
> came to my mind as a best fit was a two tables with a third joint
> table to represents the m-m relation (this is in database term. ).
> however doing a similar modeling in common-lisp is not natural (i
> mean you are going to build a lot of tools to support mapping and will
> not leverage any by-product built in facilities). so i did the
> following.
>
> (defparameter *films* (list "Lady in the Water"
> "Snakes on a Plane"
> "Just My Luck"
> "Superman Returns"
> "You, Me and Dupree"
> "The Night Listener"))
For comparison, a Ruby version.
$films =
"Lady in the Water",
"Snakes on a Plane",
"Just My Luck",
"Superman Returns",
"You, Me and Dupree",
"The Night Listener"
>
> (defparameter *critics* '())
>
> (defun add-critic (name ratings)
> (setf *critics* (acons name (list ratings) *critics*)))
>
> (add-critic "Lisa Rose" '(2.5 3.5 3.0 3.5 2.5 3.0))
> (add-critic "Gene Seymour" '(3.0 3.5 1.5 5.0 3.5 3.0))
> (add-critic "Michael Phillips" '(2.5 3.0 nil 3.5 nil 4.0))
> (add-critic "Claudia Puig" '(nil 3.5 3.0 4.0 2.5 4.5))
> (add-critic "Mick LaSalle" '(3.0 4.0 2.0 3.0 2.0 3.0))
> (add-critic "Jack Matthews" '(3.0 4.0 nil 5.0 3.5 3.0))
> (add-critic "Toby" '(4.5 nil nil 4.0 1.0 nil))
$critics = []
def add_critic name, ratings
$critics << [ name,
ratings.split.map{|s| 'nil'==s ? nil : s.to_f} ]
end
add_critic "Lisa Rose" , "2.5 3.5 3.0 3.5 2.5 3.0"
add_critic "Gene Seymour" , "3.0 3.5 1.5 5.0 3.5 3.0"
add_critic "Michael Phillips" , "2.5 3.0 nil 3.5 nil 4.0"
add_critic "Claudia Puig" , "nil 3.5 3.0 4.0 2.5 4.5"
add_critic "Mick LaSalle" , "3.0 4.0 2.0 3.0 2.0 3.0"
add_critic "Jack Matthews" , "3.0 4.0 nil 5.0 3.5 3.0"
add_critic "Toby" , "4.5 nil nil 4.0 1.0 nil"
>
> (defun critic-ratings (name)
> (second (assoc name *critics* :test #'string=)))
def critic_ratings name
$critics.assoc( name ).last
end
>
> (defun critic-films (name)
> (loop for index from 0 upto (length *films*)
> for rating in (critic-ratings name)
> when rating
> collect (cons (nth index *films*) rating) into results
> finally (return results)))
def critic_films name
$films.zip( critic_ratings name ).select{ |f,r|
r }.map{|array| array.first }
end
>
> (defun film-rating (title critic)
> (let ((film-index (position title *films* :test #'string=)))
> (when film-index
> (nth film-index (critic-ratings critic)))
def film_rating title, critic
critic_ratings( critic )[ $films.index title ]
rescue
nil
end
On Sep 14, 11:10 pm, William James <·········@yahoo.com> wrote:
> On Sep 14, 2:53 pm, cmo <·······@gmail.com> wrote:
>
>
>
> > Hi,
>
> > i went through the bloghttp://i-need-closures.blogspot.com/2007/09/programming-collective-in...
>
> > where a python code was translated into common-lisp. i couldn't grok
> > the presented translation of the recommendations data, where the
> > movies strings are repeated. the first general representation that
> > came to my mind as a best fit was a two tables with a third joint
> > table to represents the m-m relation (this is in database term. ).
> > however doing a similar modeling in common-lisp is not natural (i
> > mean you are going to build a lot of tools to support mapping and will
> > not leverage any by-product built in facilities). so i did the
> > following.
>
> > (defparameter *films* (list "Lady in the Water"
> > "Snakes on a Plane"
> > "Just My Luck"
> > "Superman Returns"
> > "You, Me and Dupree"
> > "The Night Listener"))
>
> For comparison, a Ruby version.
>
> $films =
> "Lady in the Water",
> "Snakes on a Plane",
> "Just My Luck",
> "Superman Returns",
> "You, Me and Dupree",
> "The Night Listener"
>
>
>
> > (defparameter *critics* '())
>
> > (defun add-critic (name ratings)
> > (setf *critics* (acons name (list ratings) *critics*)))
>
> > (add-critic "Lisa Rose" '(2.5 3.5 3.0 3.5 2.5 3.0))
> > (add-critic "Gene Seymour" '(3.0 3.5 1.5 5.0 3.5 3.0))
> > (add-critic "Michael Phillips" '(2.5 3.0 nil 3.5 nil 4.0))
> > (add-critic "Claudia Puig" '(nil 3.5 3.0 4.0 2.5 4.5))
> > (add-critic "Mick LaSalle" '(3.0 4.0 2.0 3.0 2.0 3.0))
> > (add-critic "Jack Matthews" '(3.0 4.0 nil 5.0 3.5 3.0))
> > (add-critic "Toby" '(4.5 nil nil 4.0 1.0 nil))
>
> $critics = []
>
> def add_critic name, ratings
> $critics << [ name,
> ratings.split.map{|s| 'nil'==s ? nil : s.to_f} ]
> end
>
> add_critic "Lisa Rose" , "2.5 3.5 3.0 3.5 2.5 3.0"
> add_critic "Gene Seymour" , "3.0 3.5 1.5 5.0 3.5 3.0"
> add_critic "Michael Phillips" , "2.5 3.0 nil 3.5 nil 4.0"
> add_critic "Claudia Puig" , "nil 3.5 3.0 4.0 2.5 4.5"
> add_critic "Mick LaSalle" , "3.0 4.0 2.0 3.0 2.0 3.0"
> add_critic "Jack Matthews" , "3.0 4.0 nil 5.0 3.5 3.0"
> add_critic "Toby" , "4.5 nil nil 4.0 1.0 nil"
>
>
>
> > (defun critic-ratings (name)
> > (second (assoc name *critics* :test #'string=)))
>
> def critic_ratings name
> $critics.assoc( name ).last
> end
>
>
>
> > (defun critic-films (name)
> > (loop for index from 0 upto (length *films*)
> > for rating in (critic-ratings name)
> > when rating
> > collect (cons (nth index *films*) rating) into results
> > finally (return results)))
>
> def critic_films name
> $films.zip( critic_ratings name ).select{ |f,r|
> r }.map{|array| array.first }
> end
>
>
>
> > (defun film-rating (title critic)
> > (let ((film-index (position title *films* :test #'string=)))
> > (when film-index
> > (nth film-index (critic-ratings critic)))
>
> def film_rating title, critic
> critic_ratings( critic )[ $films.index title ]
> rescue
> nil
> end
For you:
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
I hope that will be enough for a week?! If you need
more, I'm happy to help out... If you don't need that
many, please give some to your friends.
On Sep 14, 9:53 pm, cmo <·······@gmail.com> wrote:
> Hi,
>
> i went through the bloghttp://i-need-closures.blogspot.com/2007/09/programming-collective-in...
>
> where a python code was translated into common-lisp. i couldn't grok
> the presented translation of the recommendations data, where the
> movies strings are repeated. the first general representation that
> came to my mind as a best fit was a two tables with a third joint
> table to represents the m-m relation (this is in database term. ).
> however doing a similar modeling in common-lisp is not natural (i
> mean you are going to build a lot of tools to support mapping and will
> not leverage any by-product built in facilities). so i did the
> following.
>
> (defparameter *films* (list "Lady in the Water"
> "Snakes on a Plane"
> "Just My Luck"
> "Superman Returns"
> "You, Me and Dupree"
> "The Night Listener"))
>
> (defparameter *critics* '())
>
> (defun add-critic (name ratings)
> (setf *critics* (acons name (list ratings) *critics*)))
>
> (add-critic "Lisa Rose" '(2.5 3.5 3.0 3.5 2.5 3.0))
> (add-critic "Gene Seymour" '(3.0 3.5 1.5 5.0 3.5 3.0))
> (add-critic "Michael Phillips" '(2.5 3.0 nil 3.5 nil 4.0))
> (add-critic "Claudia Puig" '(nil 3.5 3.0 4.0 2.5 4.5))
> (add-critic "Mick LaSalle" '(3.0 4.0 2.0 3.0 2.0 3.0))
> (add-critic "Jack Matthews" '(3.0 4.0 nil 5.0 3.5 3.0))
> (add-critic "Toby" '(4.5 nil nil 4.0 1.0 nil))
>
> (defun critic-ratings (name)
> (second (assoc name *critics* :test #'string=)))
>
> (defun critic-films (name)
> (loop for index from 0 upto (length *films*)
> for rating in (critic-ratings name)
> when rating
> collect (cons (nth index *films*) rating) into results
> finally (return results)))
Puh, that's not a good way to iterate of a list of films. NTH in a
loop
trying to access list elements looks wrong.
You can LOOP over two lists...
you don't need to COLLECT INTO results and FINALLY return results.
(LOOP ... COLLECT something ) is enough. The collected list will be
returned.
>
> (defun film-rating (title critic)
> (let ((film-index (position title *films* :test #'string=)))
> (when film-index
> (nth film-index (critic-ratings critic)))
>
> ---- this is only a snippet.
>
> Now my questionS (yes multiple of them):
>
> 1- i'm not satisfied with my solution since i'm building many small
> utilities to get at the data! is this natural in common lisp (not in
> general) but in this specific example?
There are many ways to do that.
* simple lists and some small functions.
* structures or classes for film and critic. create some instances
which get linked.
* small relational storage. Typically a simple Prolog-like tool will
do. Retrievals
are then Prolog-like queries. There are also other tools supporting
relational data.
* using a db interface and the data stored as data records
> 2- also i'm noticing that my representation will have spares (NILs)
> if the data sample was large? is this a problem in the long run?
Could be
> 3- what is the best (or pragmatic) data type to represent the above
> data (aka recommendation, preferences, etc)?
CLOS classes, hash-tables, ...
> 4- does "best" in the above question morph through the stages of
> development ? (aka one representation was good then, its not in
> another)
>
> 5- when playing with data, should i store it in a store (DB,
> persistent store, file) then play with it, or should i store it like
> the above example (using defparameter only) what is the best
> practices?
Depends what you have. A text file with lists you can read is fine.
Reading CVS files is simple. Read your data and create the objects.
> 6- when i modeled the above data, i started building small utilites to
> gather different data. now what is the level to which i should keep
> the size of these functions, should i make them as slim as possible,
> or should i make them medium (merging the above critic-ratings
> function into the other two) or should i build them based on the
> vocabulary required ?
Depends. There is no limit to the size of functions. Pragmatically
it is useful to keep them small. If they get too large (what ever
that is) then you can split them up or even have longish code
being generated by Lisp.
> finally, i hope people will drop by and share there experience! (if
> any samples, links or books) and helping me reducing the time i keep
> procrastinating on such questions rather than continuing with real
> coding
Lisp, Winston&Horn, 3rd edition: shows a primitive data storage with
some retrieval.
Practical Common Lisp, Seibel: has examples in that area.
Paradigms of AI Programming, Norvig: has a simple Prolog that can be
used for relational data and queries.
There is really no end to data representation in Lisp - there are so
many different attempts.
It mostly depends on what you need in your program:
* do you have lots of objects or only a few?
* is the domain complicated or simple?
* is it likely that the domain classes will change dramatically over
time?
* do you need very efficient retrieval?
* do you have simple or complex retrieval queries?
* do you need persistence? What kind?
...
>
> thanks in advance.
>
> cmo-0
On Sep 15, 12:40 am, ·······@corporate-world.lisp.de"
<······@corporate-world.lisp.de> wrote:
>
> * small relational storage. Typically a simple Prolog-like tool will
> do. Retrievals
> are then Prolog-like queries. There are also other tools supporting
> relational data.
>
Paul Graham's book "ANSI Common Lisp" contains a small chapter about
"Inference", in which he shows how to write a simple query tool.
Given the code in that chapter, we can play:
;; copy/paste *recommendations* parameter from
;; http://i-need-closures.blogspot.com/2007/09/programming-collective-intelligence-in.html
(defun recommendations-to-facts (recommendations)
(loop for (critic . ratings) in recommendations
appending (ratings-to-facts critic ratings)))
(defun ratings-to-facts (critic ratings)
(loop for (film . rating) in ratings
collecting (rating-to-fact critic film rating)))
(defun rating-to-fact (critic film rating)
`(<- (rating ,(intern critic) ,(intern film) ,rating)))
(eval `(progn ,@(recommendations-to-facts *recommendations*)))
(<- (critic ?c) (rating ?c ?f ?r))
(<- (film ?f) (rating ?c ?f ?r))
;; so now we can, say, print all films like this:
(with-answer (film ?f)
(format t "~A~%" ?f))
;; we want a function to return the set of query results, without
;; duplicates:
(defun bagof (query &optional vars)
(let ((vars (or vars (vars-in query))))
(loop for binds in (prove query)
collecting (mapcar (lambda (v) (binding v binds)) vars))))
(defun setof (query &optional vars)
(remove-duplicates (bagof query vars) :test #'equalp))
;; now we can get the set of all films like this:
(setof '(film ?f))
;; to get ratings of a critic:
(defun critic-ratings (name)
(mapcar #'first (bagof `(rating ,name ?f ?r) '(?r))))
;; we could definitely improve on this, e.g., by having a special
;; designator for an "ignored" logic variable
;; more query examples:
(defun critic-films (name)
(mapcar #'first (setof `(rating ,name ?f ?r) '(?f))))
(defun film-rating (title critic)
(first (first (setof `(rating ,critic ,title ?r)))))
(defun common-movies (person1 person2)
(mapcar #'first
(setof `(and (rating ,person1 ?f ?r1)
(rating ,person2 ?f ?r2))
'(?f))))
Ariel
P� Sun, 14 Sep 2008 21:53:10 +0200, skrev cmo <·······@gmail.com>:
You ask quite a lot of questions. Thinking it over I feel you might be
better off reading Practical Common Lisp Chapter 3
(http://gigamonkeys.com/book/practical-a-simple-database.html), after
which you might see the answer to some of these questions youself.
--------------
John Thingstad