From: ········@gmail.com
Subject: Lisp Dictionary organization
Date: 
Message-ID: <1134625396.341762.256290@g47g2000cwa.googlegroups.com>
Marc Battyani wrote:
>Once you are there you can see that I've added a few categories like
>"implementations", etc. as an example.

>Now to go on, the first step is to define the category tree. So let's do it
>(in a breadth first way rather than depth first at least for the first 1 or
>2 levels)

>So what's your ideas for the other root categories ?


I don't know how the fractal framework works, so here's some some code
in vanilla that might help get a discussion started about the
organization of a Lisp Dictionary based on Linkit.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;
;    12/14/05
;     Nick Allen
;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defpackage :foo
  (:use :cl))

(in-package :foo)

(defvar *link-hash-table* (make-hash-table :test 'string=)
  "this will be hashed with (cons description category/rank-alist)
keyed by url string")

(defvar *category-hash-table* (make-hash-table :test 'string=)
  "this will be hashed with (list associated-links) keyed by url
string")


(defun submit-link (url title category)
  (unless (gethash url *link-hash-table*)
    (setf (gethash url *link-hash-table*) (list title (cons category
1)))
    (setf (gethash category *category-hash-table*) (cons url (gethash
category *category-hash-table*)))
    (read-link url)))

(defun read-link (url)
  "returns (1) an alist of (cons category rank) (2) the description of
the link (3) the url"
  (let ((val (gethash url *link-hash-table*)))
    (when val
      (values (cdr val) (car val) url))))

(defun submit-category (string)
  (unless (nth-value 1 (gethash string *category-hash-table*))
    (setf (gethash string *category-hash-table*) nil)))

(defun read-category (string)
  "returns a list of all the links in a category"
  (gethash string *category-hash-table*))

(defun associate (url category)
  "pushes (cons category 1) to the url's category/rank-alist"
  (let* ((alist (read-link url))
	 (cons (assoc category alist :test 'string=)))
    (unless cons
      (setf (cdr (last alist)) (list (cons category 1)))
      (setf (gethash category *category-hash-table*)
	    (cons url (gethash category *category-hash-table*)))
      (read-link url))))

(defun up-association (url category)
  "increases the rank (cdr of that category's cons in the url's
category/rank-alist) of the urls association with 'category'"
  (let* ((alist (read-link url))
	 (cons (assoc category alist :test 'string=)))
    (when cons
      (incf (cdr cons)))
    (read-link url)))

(defun down-association (url category)
 "decreases the rank (cdr of that category's cons in the url's
category/rank-alist) of the urls association with 'category'"
  (let* ((alist (read-link url))
	 (cons (assoc category alist :test 'string=)))
    (when cons
      (decf (cdr cons)))
    (read-link url)))


(defun sort-by-category (list category)
  "sorts a list by it's rank in 'category', with no association
 with that category being the lowest rank."
  (stable-sort (copy-list list)
	#'(lambda (url1 url2)
	    (let ((n1 (cdr (assoc category (read-link url1) :test 'string=)))
		  (n2 (cdr (assoc category (read-link url2) :test 'string=))))
	      (or (and (numberp n1) (numberp n2) (>= n1 n2))
		  (and n1 (null n2))
		  (and (null n1) (null n2)))))))

(defun hedge-links (n)
  "sends all links that do not have one or more association
 ratings above N to live at the farm"
  (maphash #'(lambda (key val)
	       (let ((alist (cdr val)))
		 (dolist (cons alist (send-to-live-at-farm key))
		   (if (> (cdr cons) n)
		       (return "don't remove!")))))
	   *link-hash-table*))

(defun send-to-live-at-farm (url)
  "link disapears forever"
  (let ((alist (read-link url)))
    (dolist (cons alist (remhash url *link-hash-table*))
      (let ((category (car cons)))
	(setf (gethash category *category-hash-table*)
	      (remove url (gethash category *category-hash-table*) :test
'string=))))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

FOO> (submit-link "monaco.com" "Principality of Monaco" "country")
(("country" . 1))
"Principality of Monaco"
"monaco.com"

FOO> (submit-link "morocco.com" "Kingdom of Morocco" "country")
(("country" . 1))
"Kingdom of Morocco"
"morocco.com"

FOO> (submit-link "china.com" "People's Republic of China" "country")
(("country" . 1))
"People's Republic of China"
"china.com"

FOO> (submit-link "malta.com" "Republic of Malta" "country")
(("country" . 1))
"Republic of Malta"
"malta.com"

FOO> (submit-category "island")
NIL

FOO> (associate "malta.com" "island")
(("country" . 1) ("malta" . 1))
"Republic of Malta"
"malta.com"

FOO> (associate "china.com" "island")
(("country" . 1) ("malta" . 1))
"People's Republic of China"
"china.com"

FOO> (read-category "island")
("china.com" "malta.com")
T

FOO> (up-association "malta.com" "island")
((country . 1) ("island" . 2))
"Republic of Malta"
"malta.com"

FOO> (down-association "china.com" "island")
(("country . 1) ("island" . 0))
"People's Republic of China"
"china.com"

FOO> (read-category "island")
("china.com" "malta.com")

FOO> (sort-by-category * "island")
("malta.com" "china.com")

FOO> (add-category "size")
NIL

FOO> (associate "china.com" "size")
(("country" . 1) ("island" . 2) ("size" . 1))
"People's Republic of China"
"china.com"

FOO> (read-category "country")
("morocco.com" "malta.com" "china.com" "monaco.com")

FOO> (sort-by-category * "island")
("malta.com" "china.com" "monaco.com" "morocco.com")

FOO> (sort-by-category * "size")
("china.com" "morocco.com" "monaco.com" "malta.com") ;stable-sort
didn't work quite as well as I'd hoped

FOO> (down-association "monaco.com" "country") ;nothing against monaco,
just an example
(("country" . 0))
"Principality of Monaco"
"monaco.com

FOO> (hedge-links 0)
NIL

FOO> (read-category "country")
("morocco.com" "malta.com" "china.com")

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Thank being said, I don't think there's any reason "links" to
subcategories can't be included in categories to be ranked.

The linkit/reddit model might actually be an easy way for a large group
of people to contstruct and organize and manage a large structure.

regards

nick

From: ········@gmail.com
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <1134631356.927364.294290@g49g2000cwa.googlegroups.com>
>FOO> (associate "china.com" "island")
>(("country" . 1) ("malta" . 1))
>"People's Republic of China"
>"china.com"

should be (("country" . 1) ("island" . 0))

>FOO> (associate "china.com" "size")
>(("country" . 1) ("island" . 2) ("size" . 1))
>"People's Republic of China"
>"china.com"

should be (("country" . 1) ("island . 0) ("size" . 1)

I decided to change directions with the repl example after I had
already pasted it and made the changes by hand. doh!

nick
From: Marc Battyani
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <FLmdnbBo_442sjzeRVnyvg@giganews.com>
<········@gmail.com> wrote
> Marc Battyani wrote:
> >Once you are there you can see that I've added a few categories like
> >"implementations", etc. as an example.
>
> >Now to go on, the first step is to define the category tree. So let's do
it
> >(in a breadth first way rather than depth first at least for the first 1
or
> >2 levels)
>
> >So what's your ideas for the other root categories ?
>
>
> I don't know how the fractal framework works, so here's some some code
> in vanilla that might help get a discussion started about the
> organization of a Lisp Dictionary based on Linkit.

Thanks Nick.
I was not clear enough but the repository is not based on LinkIt!
LinkIt will be used to announce/rate/push what's is in the Common Lisp
Directory but it is not the Directory.
What I need now are "Category/Sub Category Names".

I've looked at your rating code. It's OK for LinkIt but the Directory is has
no rating itself. (The rating is left to LinkIt)

> Thank being said, I don't think there's any reason "links" to
> subcategories can't be included in categories to be ranked.
>
> The linkit/reddit model might actually be an easy way for a large group
> of people to contstruct and organize and manage a large structure.

For me the LinkIt model is used to rate the Directory content but is not the
content.

Marc
From: ········@gmail.com
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <1134724173.171608.234470@f14g2000cwb.googlegroups.com>
>I was not clear enough but the repository is not based on LinkIt!
>LinkIt will be used to announce/rate/push what's is in the Common Lisp
>Directory but it is not the Directory.
>What I need now are "Category/Sub Category Names".

[...]

>For me the LinkIt model is used to rate the Directory content but is not the
>content.

Yes, actualIy I got that ; - )

I should have used a better example. What I meant to suggest is that a
Linkit/Reddit model with category-oriented ranking, user submitted
subcategories, and rankable categories could serve as a voting
framework for a self-organizing lisp dictionary separate from the data
depository.

suppose the following links, all submitted by different people at
different times of the day

   (submit-link "http://sbcl-internals.cliki.net/index" "SBCL
Internals" "SBCL")
   (submit-link "http://www.sbcl.org/" "About SBCL" "SBCL")
   (submit-link "http://www.sbcl.org/manual/" "SBCL Manual" "Steel Bank
Common Lisp")
   (submit-link "http://www.lispworks.com/documentation/HyperSpec/"
"HyperSpec" "Documentation")
   (submit-link "http://www.franz.com/support/documentation/" "ACL
Documentation" "Documentation")

now we have three categories "filled" with links

    SBCL => SBCL Internals, About SBCL
    Steel Bank Common Lisp => SBCL Manual
    Documentation => HyperSpec, ACL Documentation

now if one user comes by that night he might (with a few clicks)

   (associate "sbcl-manual" "documentation")
   (associate "sbcl-manual" "sbcl")
   (down-associate "sbcl-manual" "Steel Bank Common Lisp")

now we have

    documentation => HyperSpec, ACL documentation, SBCL manual
    SBCL => SBCL internals, about SBCL, SBCL manual

and a deprecated

    Steel Bank Common Lisp => SBCL manual

now one more user comes by and

   (make-category "lisp implementations")
   (make-category "ACL")
   (associate "ACL documentation" "ACL")
   (submit-link *link-to-ACL-category* "lisp implementations")
   (submit-link *link-to-SBCL-category* "lisp implementations")

and now we have something like

    documentation => HyperSpec, ACL documentation, SBCL manual
    lisp implementations => ACL => ACL documentation
                            SBCL=> about SBCL, SBCL internals, SBCL
manual

and some deprecated other forms that may or may not get voted off the
current category

nick
From: Christophe Rhodes
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <sq1x0d6r6s.fsf@cam.ac.uk>
Matthias <··@spam.please> writes:

> It says that folders/categories-subcategories don't work very well for
> big user groups.  The reason is that folders encode a tree, but the
> "belongs to" relationship should actually be a graph ("SBCL manual"
> belongs to "SBCL" and to "manuals").  This explains (to me) why I
> never use dictionaries to find something on the web.  I always use a
> search engine.
>
> Tags solve this problem completely.

A side point really, because I would also recommend against a purely
hierarchical view, but just so that no-one gets the wrong idea: tags
do not solve this problem completely, but they do provide a more
flexible view of what's going on.  (problems with just tags: tags have
different meanings to different individuals; some means of partial
assignment to categories turns out to be useful as well.)

Christophe
From: Marc Battyani
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <zNydnXTM-vFUQz_enZ2dnUVZ8qSdnZ2d@giganews.com>
"Christophe Rhodes" <·····@cam.ac.uk> wrote
> Matthias <··@spam.please> writes:
>
> > It says that folders/categories-subcategories don't work very well for
> > big user groups.  The reason is that folders encode a tree, but the
> > "belongs to" relationship should actually be a graph ("SBCL manual"
> > belongs to "SBCL" and to "manuals").  This explains (to me) why I
> > never use dictionaries to find something on the web.  I always use a
> > search engine.
> >
> > Tags solve this problem completely.
>
> A side point really, because I would also recommend against a purely
> hierarchical view, but just so that no-one gets the wrong idea: tags
> do not solve this problem completely, but they do provide a more
> flexible view of what's going on.  (problems with just tags: tags have
> different meanings to different individuals; some means of partial
> assignment to categories turns out to be useful as well.)

As use a combinaison of all that:

The categories are organized in a tree.
Each category also have related categories (side edges) making it a DAG.
The directory items have any number of categories so that there are several
paths to find them.

I've not considered yet if the "related" edges should be bidirectional.

Comments, improvements ?

Marc
From: Geoffrey Summerhayes
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <jjEof.3750$El.362440@news20.bellglobal.com>
"Marc Battyani" <·············@fractalconcept.com> wrote in message 
·····································@giganews.com...
>
>
> The categories are organized in a tree.
> Each category also have related categories (side edges) making it a DAG.
> The directory items have any number of categories so that there are 
> several
> paths to find them.
>
> I've not considered yet if the "related" edges should be bidirectional.
>

Maybe a 'new' or 'recent changes'?

--
Geoff 
From: Marc Battyani
Subject: Re: Lisp Dictionary organization
Date: 
Message-ID: <9YCdndCN4LjAuj7eRVny1Q@giganews.com>
"Geoffrey Summerhayes" <·············@hotmail.com> wrote
>
> "Marc Battyani" <·············@fractalconcept.com> wrote in message
> >
> >
> > The categories are organized in a tree.
> > Each category also have related categories (side edges) making it a DAG.
> > The directory items have any number of categories so that there are
> > several
> > paths to find them.
> >
> > I've not considered yet if the "related" edges should be bidirectional.
> >
>
> Maybe a 'new' or 'recent changes'?

The 'new' content will be dumped to the LinkIt 'new' section so that it can
be viewed, rated and promoted (or not ;-) to the hot and top sections.
I will also add RSS feeds at every category level.

Marc