Peter Norvig wrote a simple spelling corrector in 20 lines of Python 2.5;
anyone want to take a crack at a Lisp version? I did
translate it to Ruby as a comparison (I'll provide that version below in
case having two versions is helpful, but the Python version is more
readable IMO), but the Lisp version is going to take me much longer due to
my lack of proficiency, so it would be great to see an elegant Lisp
version.
Here's a link to Norvig's page: http://www.norvig.com/spell-correct.html
That page includes a link to a text file that I saved locally as
holmes.txt: http://www.norvig.com/holmes.txt
Note: I wrapped a few of the longer lines for posting.
def words text
text.downcase.scan(/[a-z]+/)
end
def train features
model = Hash.new(1)
features.each {|f| model[f] += 1 }
return model
end
NWORDS = train(words(File.new('holmes.txt').read))
LETTERS = 'abcdefghijklmnopqrstuvwxyz'
def edits1 word
n = word.length
deletion = (0...n).collect {|i| word[0...i]+word[i+1..-1] }
transposition = (0...n-1).collect {
|i| word[0...i]+word[i+1,1]+word[i,1]+word[i+2..-1] }
alteration = []
n.times {|i| LETTERS.each_byte {
|l| alteration << word[0...i]+l.chr+word[i+1..-1] } }
insertion = []
(n+1).times {|i| LETTERS.each_byte {
|l| insertion << word[0...i]+l.chr+word[i..-1] } }
result = deletion + transposition + alteration + insertion
result.empty? ? nil : result
end
def known_edits2 word
result = []
edits1(word).each {|e1| edits1(e1).each {
|e2| result << e2 if NWORDS.has_key?(e2) }}
result.empty? ? nil : result
end
def known words
result = words.find_all {|w| NWORDS.has_key?(w) }
result.empty? ? nil : result
end
def correct word
(known([word]) or known(edits1(word)) or known_edits2(word) or
[word]).max {|a,b| NWORDS[a] <=> NWORDS[b] }
end
Brian Adkins
http://lojic.com/blog/
Brian Adkins <·················@gmail.com> writes:
> Peter Norvig wrote a simple spelling corrector in 20 lines of Python 2.5;
> anyone want to take a crack at a Lisp version? I did
> translate it to Ruby as a comparison (I'll provide that version below in
> case having two versions is helpful, but the Python version is more
> readable IMO), but the Lisp version is going to take me much longer due to
> my lack of proficiency, so it would be great to see an elegant Lisp
> version.
>
> Here's a link to Norvig's page: http://www.norvig.com/spell-correct.html
>
> That page includes a link to a text file that I saved locally as
> holmes.txt: http://www.norvig.com/holmes.txt
My stab at it, translated from the Python, follows. Note that it got
a tad long because of how concise Python is about list comprehensions
and possibly because I'm not at all fluent in Python and not a true CL
guru.
(require :cl-ppcre)
(defun words (text)
(cl-ppcre:all-matches-as-strings "[a-z]+" (string-downcase text)))
(defun train (features)
(let ((model (make-hash-table :test 'equal)))
(mapc #'(lambda (feature)
(unless (gethash feature model)
(setf (gethash feature model) 0))
(incf (gethash feature model)))
features)
model))
(defun read-file (filename)
(let (lines)
(with-open-file (s filename)
(do ((line (read-line s nil nil) (read-line s nil nil)))
((null line))
(push line lines)
(push #(#\Newline) lines)))
(apply 'concatenate 'string (nreverse lines))))
(defparameter *nwords* (train (words (read-file "holmes.txt"))))
(defun edits1 (word)
(let ((length (length word))
edits)
; Deletion
(dotimes (i length)
(pushnew (concatenate 'string
(subseq word 0 i)
(subseq word (1+ i)))
edits
:test 'string=))
; Transposition
(dotimes (i (1- length))
(pushnew (concatenate 'string
(subseq word 0 i)
(subseq word (1+ i) (+ 2 i))
(subseq word i (1+ i))
(subseq word (+ 2 i)))
edits
:test 'string=))
; Alteration
(dotimes (i length)
(dotimes (j 26)
(pushnew (concatenate 'string
(subseq word 0 i)
(subseq "abcdefghijklmnopqrstuvwxyz" j (1+ j))
(subseq word (1+ i)))
edits
:test 'string=)))
; Insertion
(dotimes (i length)
(dotimes (j 26)
(pushnew (concatenate 'string
(subseq word 0 i)
(subseq "abcdefghijklmnopqrstuvwxyz" j (1+ j))
(subseq word i))
edits
:test 'string=)))
edits))
(defun known-edits2 (word)
(mapcan #'(lambda (e1)
(and (gethash e1 *nwords*) (edits1 e1)))
(edits1 word)))
(defun known (words)
(mapcan #'(lambda (word)
(and (gethash word *nwords*) (list word)))
words))
(defun correct (word)
(if (gethash word *nwords*)
word
(let ((words (concatenate 'list
(list word)
(known (list word))
(edits1 word)
(known-edits2 word))))
(first
(stable-sort words
'>
:key #'(lambda (w) (gethash w *nwords* 0)))))))
Well, why not? [Of course, the Python is practically illegible line-
noise to me, so it's possible the algorithm has drifted . . .]
--Chhi'mèd Künzang
(in-package cl-user)
(defparameter *file* "~/tmp/asdf.txt")
(defconstant +letters+ "abcdefghijklmnopqrstuvwxyz")
(defun strcat (&rest args) (apply #'concatenate 'string (mapcar
#'string args)))
(defun words (text)
(flet ((transition (a b) (and (notevery #'alpha-char-p (list a b))
(some #'alpha-char-p (list a b)))))
(let ((transitions (loop for last = #\. then c for c across text
for i from 0
when (transition last c) collect i)))
(loop for (start end) on transitions by #'cddr
collect (subseq text start end)))))
(defun train (features)
(let ((model (make-hash-table :test 'equal)))
(dolist (feature features)
(let ((found (gethash feature model)))
(setf (gethash feature model) (if found (1+ found) 1))))
model))
(defparameter *n-words* (train (words (with-output-to-string (out)
(with-open-file (in
*file* :direction :input)
(loop for line = (read-line
in nil nil)
while line do (write-
line line out)))))))
(defun edits-1 (word)
(flet ((subw (start &optional end) (subseq word start end)))
(let* ((n (length word))
(deletion (loop for i below n collect (strcat (subw 0 i)
(subw (1+ i)))))
(transposition (loop for i below (1- n)
collect (strcat (subw 0 i) (char word
(1+ i)) (char word i)
(subw (+ i 2)))))
(alteration (loop for char across +letters+
append (loop for i below n
collect (strcat (subw 0 i)
char (subw (1+ i))))))
(insertion (loop for char across +letters+
append (loop for i below (1+ n)
collect (strcat (subw 0 i)
char (subw i))))))
(append deletion transposition alteration insertion))))
(defun lookup (word) (gethash word *n-words*))
(defun known (words) (remove-if-not #'lookup words))
(defun known-edits-2 (word) (known (mapcan #'edits-1 (edits-1 word))))
(defun correct (word)
(car (sort (or (known (list word))
(known (edits-1 word))
(known-edits-2 word)) #'> :key #'lookup)))
On Apr 10, 12:09 am, ········@gmail.com wrote:
> Well, why not? [Of course, the Python is practically illegible line-
> noise to me, so it's possible the algorithm has drifted . . .]
Plus I'm an idiot with a wide editor buffer,
turned loose on Google Groups.
Let's try that again. -ck
---
(in-package cl-user)
(defparameter *file* "~/tmp/asdf.txt")
(defconstant +letters+ "abcdefghijklmnopqrstuvwxyz")
(defun strcat (&rest args)
(apply #'concatenate 'string (mapcar #'string args)))
(defun words (text)
(flet ((transition (a b) (and (notevery #'alpha-char-p (list a b))
(some #'alpha-char-p (list a b)))))
(let ((transitions (loop for last = #\. then c
for c across text
for i from 0
when (transition last c) collect i)))
(loop for (start end) on transitions by #'cddr
collect (subseq text start end)))))
(defun train (features)
(let ((model (make-hash-table :test 'equal)))
(dolist (feature features)
(let ((found (gethash feature model)))
(setf (gethash feature model) (if found (1+ found) 1))))
model))
(defparameter *n-words*
(train (words (with-output-to-string (out)
(with-open-file (in *file* :direction :input)
(loop for line = (read-line in nil nil)
while line do (write-line line out)))))))
(defun edits-1 (word)
(flet ((subw (start &optional end) (subseq word start end)))
(let* ((n (length word))
(deletion (loop for i below n
collect (strcat (subw 0 i) (subw (1+ i)))))
(transposition (loop for i below (1- n)
collect (strcat (subw 0 i)
(char word (1+ i))
(char word i)
(subw (+ i 2)))))
(alteration (loop for char across +letters+
append (loop for i below n
collect (strcat (subw 0 i)
char
(subw (1+
i))))))
(insertion (loop for char across +letters+
append (loop for i below (1+ n)
collect (strcat (subw 0 i)
char
(subw i))))))
(append deletion transposition alteration insertion))))
(defun lookup (word) (gethash word *n-words*))
(defun known (words) (remove-if-not #'lookup words))
(defun known-edits-2 (word) (known (mapcan #'edits-1 (edits-1 word))))
(defun correct (word)
(car (sort (or (known (list word))
(known (edits-1 word))
(known-edits-2 word)) #'> :key #'lookup)))
Brian Adkins <·················@gmail.com> writes:
> Peter Norvig wrote a simple spelling corrector in 20 lines of Python 2.5;
> anyone want to take a crack at a Lisp version?
Here's my version. For some reason, I decided to (over-)optimize for
line count. Not sure if that's what you had in mind. Same line count
as the Python version but fits in 80 columns! :-)
(defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
(in-package #:spell-correct)
(defun words (file)
(iter (for line in-file file using #'read-line)
(nconcing (all-matches-as-strings "[a-z]+" (string-downcase line)))))
(defun train (words &aux (ht (make-hash-table :test 'equalp)))
(dolist (w words ht) (incf (gethash w ht 0))))
(defvar *n-words* (train (words #p"/tmp/holmes.txt")))
(defun edits (w &aux (n (length w)))
(flet ((cat (&rest args) (apply #'concatenate 'string (mapcar #'string args)))
(s (start &optional end) (subseq w start end)))
(nconc (iter (for i below n) (collect (cat (s 0 i) (s (1+ i)))))
(iter (for i below (1- n))
(collect (cat (s 0 i) (char w (1+ i)) (char w i) (s (+ i 2)))))
(iter (for ch in-string w)
(dotimes (i n) (collect (cat (s 0 i) ch (s (1+ i)))))
(dotimes (i (1+ n)) (collect (cat (s 0 i) ch (s i))))))))
(defun correct (word)
(flet ((known (ws) (remove-if-not (lambda (w) (gethash w *n-words*)) ws)))
(iter (for w in (or (known (list word)) (known (edits word))
(known (mapcan #'edits (edits word))) (list word)))
(finding w maximizing (gethash w *n-words*)))))
--
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/
On Tue, 10 Apr 2007 10:01:00 +0100, Lu�s Oliveira wrote:
> Brian Adkins <·················@gmail.com> writes:
>> Peter Norvig wrote a simple spelling corrector in 20 lines of Python 2.5;
>> anyone want to take a crack at a Lisp version?
>
> Here's my version. For some reason, I decided to (over-)optimize for
> line count. Not sure if that's what you had in mind. Same line count
> as the Python version but fits in 80 columns! :-)
Well, it wasn't meant to be a golf exercise, but it's nice to see a more
concise version.
>
> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
> (in-package #:spell-correct)
I get the following error:
[1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
*** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
installed, and I do have the cl-ppcre debian package installed. So I
checked out the following page to see what else I need to do to get
cl-ppcre to work:
http://weitz.de/cl-ppcre/#install
I tried:
cd /usr/share/common-lisp/source/cl-ppcre
clisp
[1]> (mk:compile-system "cl-ppcre")
*** - READ from #<INPUT CONCATENATED-STREAM #<INPUT STRING-INPUT-STREAM> #<IO TERMINAL-STREAM>>: there is no package with
name "MK"
Ok, I guess I don't have MK installed. The page above also says "(or the
equivalent one for asdf)", but I don't know what the equivalent one for
asdf is.
No problem, I'll try (load "load.lisp")
[1]> (load "load.lisp")
;; Loading file load.lisp ...
*** - UNIX error 13 (EACCES): Permission denied
Ok, at this point, I think asking for help might be warranted :) What is
the easiest way to get cl-ppcre working?
>
> (defun words (file)
> (iter (for line in-file file using #'read-line)
> (nconcing (all-matches-as-strings "[a-z]+" (string-downcase
> line)))))
>
> (defun train (words &aux (ht (make-hash-table :test 'equalp)))
> (dolist (w words ht) (incf (gethash w ht 0))))
>
> (defvar *n-words* (train (words #p"/tmp/holmes.txt")))
>
> (defun edits (w &aux (n (length w)))
> (flet ((cat (&rest args) (apply #'concatenate 'string (mapcar #'string
> args)))
> (s (start &optional end) (subseq w start end)))
> (nconc (iter (for i below n) (collect (cat (s 0 i) (s (1+ i)))))
> (iter (for i below (1- n))
> (collect (cat (s 0 i) (char w (1+ i)) (char w i) (s (+
> i 2)))))
> (iter (for ch in-string w)
> (dotimes (i n) (collect (cat (s 0 i) ch (s (1+ i)))))
> (dotimes (i (1+ n)) (collect (cat (s 0 i) ch (s
> i))))))))
>
> (defun correct (word)
> (flet ((known (ws) (remove-if-not (lambda (w) (gethash w *n-words*))
> ws)))
> (iter (for w in (or (known (list word)) (known (edits word))
> (known (mapcan #'edits (edits word))) (list
> word)))
> (finding w maximizing (gethash w *n-words*)))))
Hi,
> Ok, at this point, I think asking for help might be warranted :) What is
> the easiest way to get cl-ppcre working?
(clc:clc-require :cl-ppcre)
CLC is short for the Common Lisp Controller, which is what Debian and
Gentoo use for installing and loading Common Lisp systems. You can
also use ,load-system cl-ppcre [RET] in SLIME or the ASDF loading
command (asdf:oos 'asdf:load-op :cl-ppcre), as the Common Lisp
Controller is based on ASDF.
Personally, I use (require :cl-ppcre), but that's possible only
because ASDF is hooked into SBCL's REQUIRE mechanism. It won't work
on CMUCL or CLISP. (Actually, I usually define my own ASDF systems
and have them depend on others, which causes ASDF to load them
automatically, and then REQUIRE those systems of mine.)
Bye,
Matthias
On Tue, 10 Apr 2007 08:13:48 -0700, Matthias Benkard wrote:
> Hi,
>
>> Ok, at this point, I think asking for help might be warranted :) What is
>> the easiest way to get cl-ppcre working?
>
> (clc:clc-require :cl-ppcre)
>
> CLC is short for the Common Lisp Controller, which is what Debian and
> Gentoo use for installing and loading Common Lisp systems. You can
> also use ,load-system cl-ppcre [RET] in SLIME or the ASDF loading
> command (asdf:oos 'asdf:load-op :cl-ppcre), as the Common Lisp
> Controller is based on ASDF.
Thanks - very helpful. Although I don't foresee developing on a non-debian
system in the future, I expect that using (asdf:oos 'asdf:load-op
:cl-ppcre) would be better because it's more portable. Is that an
accurate assessment?
>
> Personally, I use (require :cl-ppcre), but that's possible only because
> ASDF is hooked into SBCL's REQUIRE mechanism. It won't work on CMUCL or
> CLISP. (Actually, I usually define my own ASDF systems and have them
> depend on others, which causes ASDF to load them automatically, and then
> REQUIRE those systems of mine.)
>
> Bye,
> Matthias
Hi,
> I expect that using (asdf:oos 'asdf:load-op
> :cl-ppcre) would be better because it's more portable. Is that an
> accurate assessment?
I think that defining your own ASDF system and having it explicitly
depend on the systems it, well, depends on, is the most general _and_
convenient approach. :)
It's not that hard either. Just create a .asd file in the directory
your Lisp files sit in and link to it from ~/.clc/systems/, e.g.:
····@wirselkraut:~/Project-Bla123% cat bla123.asd
(defsystem "bla123"
:description "Bla bla bla."
:version "0.1"
:author "Brian Adkins <·····@bla-mail.org>"
:licence "Affero General Public License, version 1 or higher"
:depends-on (#:cl-ppcre #:iterate #:xmls)
:components ((:file "package")
(:file "bla-server"))
:serial t)
····@wirselkraut:~/Project-Bla123% mkdir -p ~/.clc/systems
····@wirselkraut:~/Project-Bla123% cd ~/.clc/systems
····@wirselkraut:~/.clc/systems% ln -s ../../Project-Bla123/bla123.asd
>From then on, you can load your system and all of its dependencies via
(clc:clc-require :bla123), and if you package it as a .tar.gz file, it
should even be ASDF-installable right away (including on non-Debian
systems).
That said, if you absolutely, positively want your Lisp project to
consist of a single file only, yeah, I guess using the generic ASDF
command is a better aproach than using clc:clc-require. You can still
use the latter in the REPL, of course. I do that all the time, as I
find it far easier to type than that ASDF:OOS stuff. :)
Bye, Matthias
On 10 Apr 2007 08:13:48 -0700, <··········@gmail.com> wrote:
>
> Personally, I use (require :cl-ppcre), but that's possible only
> because ASDF is hooked into SBCL's REQUIRE mechanism. It won't work
> on CMUCL
It works fine on CMUCL once you connect pre-existing bits in
REQUIRE. There used to be HOWTO info on cliki, it's only a couple
lines difference as I recall. I believe that ASDF is builtin to SBCL,
which is why it runs out-of-the-box.
--
There are no average Common Lisp programmers
Reply-To: email is ignored.
--
Posted via a free Usenet account from http://www.teranews.com
>>>>> "GP" == GP lisper <········@CloudDancer.com> writes:
GP> On 10 Apr 2007 08:13:48 -0700, <··········@gmail.com> wrote:
>>
>> Personally, I use (require :cl-ppcre), but that's possible only
>> because ASDF is hooked into SBCL's REQUIRE mechanism. It won't work
>> on CMUCL
GP> It works fine on CMUCL once you connect pre-existing bits in
GP> REQUIRE. There used to be HOWTO info on cliki, it's only a couple
GP> lines difference as I recall. I believe that ASDF is builtin to SBCL,
Here is the link: http://www.cliki.net/asdf
For this to work with cmucl, all that needs to be done is one of the
following:
1. Someone add the necessary bits to asdf
2. Add the bit of code to your .cmucl-init.lisp or something.
I just do option 2, myself.
Ray
Brian Adkins <·················@gmail.com> writes:
> I get the following error:
>
> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>
> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>
> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
> installed, and I do have the cl-ppcre debian package installed.
If you have the Ubuntu package, follow the Ubuntu instructions to get
it to work. They're likely not specific to cl-ppcre, but generic to
any Ubuntu Lisp package. (I don't know where those instructions are,
sorry.)
> So I checked out the following page to see what else I need to do to
> get cl-ppcre to work:
>
> http://weitz.de/cl-ppcre/#install
This page is for people who install cl-ppcre from the source, not from
an OS vendor package.
Zach
On Tue, 10 Apr 2007 10:45:24 -0400, Zach Beane wrote:
> Brian Adkins <·················@gmail.com> writes:
>
>> I get the following error:
>>
>> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>>
>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>>
>> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
>> installed, and I do have the cl-ppcre debian package installed.
>
> If you have the Ubuntu package, follow the Ubuntu instructions to get
> it to work. They're likely not specific to cl-ppcre, but generic to
> any Ubuntu Lisp package. (I don't know where those instructions are,
> sorry.)
Thanks for the response, but for every other Ubuntu package I've
installed, it goes something like this:
sudo apt-get install cl-ppcre
and that's it. I've never had to refer to any Ubuntu instructions because
the point of the package manager is to do *everything* required to install
the package. It will even prompt the user for configuration info if
necessary which is rare.
>
>> So I checked out the following page to see what else I need to do to
>> get cl-ppcre to work:
>>
>> http://weitz.de/cl-ppcre/#install
>
> This page is for people who install cl-ppcre from the source, not from
> an OS vendor package.
Well, the page does state the following:
"If you're on Debian, you should probably use the cl-ppcre Debian package
which is available thanks to Peter van Eynde and Kevin Rosenberg."
Ubuntu is a debian distro, so since the page sad "you should probably", I
did.
The source for cl-ppcre seems to have been installed just fine:
·····@airstream:/usr/share/common-lisp/source/cl-ppcre$ ls
api.lisp cl-ppcre-test.asd lexer.lisp optimize.lisp ppcre-tests.lisp scanner.lisp
closures.lisp convert.lisp lispworks-defsystem.lisp packages.lisp regex-class.lisp specials.lisp
cl-ppcre.asd errors.lisp load.lisp parser.lisp repetition-closures.lisp util.lisp
I think it's just a matter of telling clisp that it exists. I have no idea
why I got the "permission denied" error since all the files are readable.
Is (load "load.lisp") trying to write to the directory or something?
Anyone know how to load it with asdf ?
>
> Zach
Brian Adkins <·················@gmail.com> writes:
> > If you have the Ubuntu package, follow the Ubuntu instructions to get
> > it to work. They're likely not specific to cl-ppcre, but generic to
> > any Ubuntu Lisp package. (I don't know where those instructions are,
> > sorry.)
>
> Thanks for the response, but for every other Ubuntu package I've
> installed, it goes something like this:
>
> sudo apt-get install cl-ppcre
>
> and that's it.
That's not sufficient to get cl-ppcre (or any other Lisp package)
loaded into a freshly started Lisp session.
> I think it's just a matter of telling clisp that it exists.
You need to tell clisp to load it.
> I have no idea why I got the "permission denied" error since all the
> files are readable. Is (load "load.lisp") trying to write to the
> directory or something?
It's trying to compile it, which will create fasl files.
> Anyone know how to load it with asdf ?
You should refer to the Debian/Ubuntu directions for loading a
Debian/Ubuntu Lisp package.
Zach
Brian Adkins <·················@gmail.com> writes:
> Anyone know how to load it with asdf ?
(asdf:oos 'asdf:load-op :cl-ppcre)
--
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/
On Tue, 10 Apr 2007 11:11:34 -0400, Brian Adkins wrote:
> On Tue, 10 Apr 2007 10:45:24 -0400, Zach Beane wrote:
>
>> Brian Adkins <·················@gmail.com> writes:
>>
>>> I get the following error:
>>>
>>> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>>>
>>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>>>
>>> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
>>> installed, and I do have the cl-ppcre debian package installed.
>>
>> If you have the Ubuntu package, follow the Ubuntu instructions to get
>> it to work. They're likely not specific to cl-ppcre, but generic to
>> any Ubuntu Lisp package. (I don't know where those instructions are,
>> sorry.)
>
> Thanks for the response, but for every other Ubuntu package I've
> installed, it goes something like this:
>
> sudo apt-get install cl-ppcre
>
> and that's it. I've never had to refer to any Ubuntu instructions because
> the point of the package manager is to do *everything* required to install
> the package. It will even prompt the user for configuration info if
> necessary which is rare.
>
>>
>>> So I checked out the following page to see what else I need to do to
>>> get cl-ppcre to work:
>>>
>>> http://weitz.de/cl-ppcre/#install
>>
>> This page is for people who install cl-ppcre from the source, not from
>> an OS vendor package.
>
> Well, the page does state the following:
>
> "If you're on Debian, you should probably use the cl-ppcre Debian package
> which is available thanks to Peter van Eynde and Kevin Rosenberg."
>
> Ubuntu is a debian distro, so since the page sad "you should probably", I
> did.
>
> The source for cl-ppcre seems to have been installed just fine:
>
> ·····@airstream:/usr/share/common-lisp/source/cl-ppcre$ ls
> api.lisp cl-ppcre-test.asd lexer.lisp optimize.lisp ppcre-tests.lisp scanner.lisp
> closures.lisp convert.lisp lispworks-defsystem.lisp packages.lisp regex-class.lisp specials.lisp
> cl-ppcre.asd errors.lisp load.lisp parser.lisp repetition-closures.lisp util.lisp
>
> I think it's just a matter of telling clisp that it exists. I have no idea
> why I got the "permission denied" error since all the files are readable.
> Is (load "load.lisp") trying to write to the directory or something?
>
> Anyone know how to load it with asdf ?
Erm, like:
(asdf:oos 'asdf:load-op :cl-ppcre)
HTH Ralf Mattes
>>
>> Zach
On Tue, 10 Apr 2007 17:15:29 +0200, Ralf Mattes wrote:
> On Tue, 10 Apr 2007 11:11:34 -0400, Brian Adkins wrote:
>
>> On Tue, 10 Apr 2007 10:45:24 -0400, Zach Beane wrote:
>>
>>> Brian Adkins <·················@gmail.com> writes:
>>>
>>>> I get the following error:
>>>>
>>>> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>>>>
>>>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>>>>
>>>> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
>>>> installed, and I do have the cl-ppcre debian package installed.
>>>
>>> If you have the Ubuntu package, follow the Ubuntu instructions to get
>>> it to work. They're likely not specific to cl-ppcre, but generic to
>>> any Ubuntu Lisp package. (I don't know where those instructions are,
>>> sorry.)
>>
>> Thanks for the response, but for every other Ubuntu package I've
>> installed, it goes something like this:
>>
>> sudo apt-get install cl-ppcre
>>
>> and that's it. I've never had to refer to any Ubuntu instructions because
>> the point of the package manager is to do *everything* required to install
>> the package. It will even prompt the user for configuration info if
>> necessary which is rare.
>>
>>>
>>>> So I checked out the following page to see what else I need to do to
>>>> get cl-ppcre to work:
>>>>
>>>> http://weitz.de/cl-ppcre/#install
>>>
>>> This page is for people who install cl-ppcre from the source, not from
>>> an OS vendor package.
>>
>> Well, the page does state the following:
>>
>> "If you're on Debian, you should probably use the cl-ppcre Debian package
>> which is available thanks to Peter van Eynde and Kevin Rosenberg."
>>
>> Ubuntu is a debian distro, so since the page sad "you should probably", I
>> did.
>>
>> The source for cl-ppcre seems to have been installed just fine:
>>
>> ·····@airstream:/usr/share/common-lisp/source/cl-ppcre$ ls
>> api.lisp cl-ppcre-test.asd lexer.lisp optimize.lisp ppcre-tests.lisp scanner.lisp
>> closures.lisp convert.lisp lispworks-defsystem.lisp packages.lisp regex-class.lisp specials.lisp
>> cl-ppcre.asd errors.lisp load.lisp parser.lisp repetition-closures.lisp util.lisp
>>
>> I think it's just a matter of telling clisp that it exists. I have no idea
>> why I got the "permission denied" error since all the files are readable.
>> Is (load "load.lisp") trying to write to the directory or something?
>>
>> Anyone know how to load it with asdf ?
>
> Erm, like:
>
> (asdf:oos 'asdf:load-op :cl-ppcre)
Perfect, that did the trick - thanks. Yes, I know this is basic stuff, but
try to remember back to when you knew essentially nothing about Lisp :)
>
> HTH Ralf Mattes
>
>
>>> Zach
From: Edi Weitz
Subject: Re: How to Write a Spelling Corrector
Date:
Message-ID: <u1wisckoj.fsf@agharta.de>
On Tue, 10 Apr 2007 11:11:34 -0400, Brian Adkins <·················@gmail.com> wrote:
> On Tue, 10 Apr 2007 10:45:24 -0400, Zach Beane wrote:
>
>> Brian Adkins <·················@gmail.com> writes:
>>
>>> I get the following error:
>>>
>>> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>>>
>>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>>>
>>> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the
>>> package installed, and I do have the cl-ppcre debian package
>>> installed.
That the Debian package is installed doesn't mean that the library has
been loaded into your Lisp image. Do you want your Lisp image to
automatically load each and every Lisp library that happens to be on
your hard disk each time it starts? Does the Perl interpreter
automatically load the DBI module each time it starts once you've
installed the corresponding Debian package?
Also note that a Debian "package" and a Common Lisp "package" are
different things.
http://www.lispworks.com/documentation/HyperSpec/Body/11_.htm
> Thanks for the response, but for every other Ubuntu package I've
> installed, it goes something like this:
>
> sudo apt-get install cl-ppcre
>
> and that's it. I've never had to refer to any Ubuntu instructions
> because the point of the package manager is to do *everything*
> required to install the package.
Good for you. But it seems that this time you /do/ need to look at
the documentation.
> I think it's just a matter of telling clisp that it exists.
It's mainly a matter of telling CLISP that you want it to be loaded.
> Anyone know how to load it with asdf ?
See its README file for example.
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
On Tue, 10 Apr 2007 17:29:16 +0200, Edi Weitz wrote:
> On Tue, 10 Apr 2007 11:11:34 -0400, Brian Adkins <·················@gmail.com> wrote:
>
>> On Tue, 10 Apr 2007 10:45:24 -0400, Zach Beane wrote:
>>
>>> Brian Adkins <·················@gmail.com> writes:
>>>
>>>> I get the following error:
>>>>
>>>> [1]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
>>>>
>>>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
>>>>
>>>> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the
>>>> package installed, and I do have the cl-ppcre debian package
>>>> installed.
>
> That the Debian package is installed doesn't mean that the library has
> been loaded into your Lisp image. Do you want your Lisp image to
> automatically load each and every Lisp library that happens to be on
> your hard disk each time it starts? Does the Perl interpreter
> automatically load the DBI module each time it starts once you've
> installed the corresponding Debian package?
I understand that the equivalent of 'require' or 'import' is necessary, I
just didn't know what it was. I've updated the cliki with a note about
how to load the package. Yes, it's better to teach someone how to fish
than to give them a fish - unless they're really hungry, then maybe eating
a fish will allow them to fish for the rest of the day ;)
>
> Also note that a Debian "package" and a Common Lisp "package" are
> different things.
>
> http://www.lispworks.com/documentation/HyperSpec/Body/11_.htm
>
>> Thanks for the response, but for every other Ubuntu package I've
>> installed, it goes something like this:
>>
>> sudo apt-get install cl-ppcre
>>
>> and that's it. I've never had to refer to any Ubuntu instructions
>> because the point of the package manager is to do *everything* required
>> to install the package.
>
> Good for you. But it seems that this time you /do/ need to look at the
> documentation.
My point wasn't that I didn't need to look at any docs, but that it wasn't
an Ubuntu thing, but a Lisp thing. In this case, asdf was the missing
piece. I'm going through ACL and PCL, but I haven't learned much about
Lisp systems yet, the only reason I jumped ahead in this case was to try
out Lu�s' program, so I didn't want to do a lot of background research
just to do that.
>
>> I think it's just a matter of telling clisp that it exists.
>
> It's mainly a matter of telling CLISP that you want it to be loaded.
>
>> Anyone know how to load it with asdf ?
>
> See its README file for example.
Oops, the cat's out of the bag already...
>
> Cheers,
> Edi.
From: Holger Schauer
Subject: Re: How to Write a Spelling Corrector
Date:
Message-ID: <yxzabxg6yt6.fsf@gmx.de>
On 4969 September 1993, Brian Adkins wrote:
> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "CL-PPCRE"
> I'm running Ubuntu 6.10 Linux, so I checked to make sure I had the package
> installed, and I do have the cl-ppcre debian package installed.
Issue (asdf:oos 'asdf:load-op 'cl-ppcre) and probably also
(asdf:oos 'asdf:load-op 'iterate) at the command line of your
lisp. Try reading the doc for ASDF in /usr/share/doc/cl-asdf/.
Holger
--
--- http://hillview.bugwriter.net/ ---
"Linux stands for the 'Linguistic Institute of Nocturnal
Undertakers on XEmacs'."
-- Anonzymous Coward on slashdot
On Tue, 10 Apr 2007 16:40:58 +0200, Brian Adkins
<·················@gmail.com> wrote:
Ok to sum it up..
(asdf:oos asdf:load-op 'cl-ppcre)
(asdf:oos asdf:load-op 'iterate)
(load (compile #p"spell-correct"))
(make-package :test-spell :use (:spell-correct))
(use-package :test-spell)
(correct "word")
...
(use-package :cl-user)
(delete-package :test-spell) ; cleans up the namespace
or something like that is a typical workflow to test some code
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
On Tue, 10 Apr 2007 10:01:00 +0100, Lu�s Oliveira wrote:
> Brian Adkins <·················@gmail.com> writes:
>> Peter Norvig wrote a simple spelling corrector in 20 lines of Python 2.5;
>> anyone want to take a crack at a Lisp version?
>
> Here's my version. For some reason, I decided to (over-)optimize for
> line count. Not sure if that's what you had in mind. Same line count
> as the Python version but fits in 80 columns! :-)
>
> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
> (in-package #:spell-correct)
Thanks to the helpful folks on this thread, I got cl-ppcre working. Now I
get this:
[4]> (defpackage #:spell-correct (:use #:cl #:cl-ppcre #:iterate))
*** - SYSTEM::%FIND-PACKAGE: There is no package with name "ITERATE"
Ok, so I need to find the ITERATE package, back to cliki.net for yet
another library :( I don't recall what Python has, but Ruby has gems which
are a very convenient package management system. Since I can't expect a
debian package for every useful CL library, is asdf-install the commonly
accepted way to install lisp libraries?
For now, I just downloaded the tarball, untarred and created a softlink to
iterate.asd. Everything loads fine, but now I get:
; loading system definition from /usr/share/common-lisp/systems/iterate.asd into #<PACKAGE ASDF0>
;; Loading file /usr/share/common-lisp/systems/iterate.asd ...
; registering #<SYSTEM :ITERATE #x2069282E> as ITERATE
; registering #<SYSTEM :ITERATE-PG #x20698DDE> as ITERATE-PG
; registering #<SYSTEM :ITERATE-TESTS #x2069A4FE> as ITERATE-TESTS
WARNING: The generic function #<STANDARD-GENERIC-FUNCTION PERFORM> is being modified, but has already been called.
;; Loaded file /usr/share/common-lisp/systems/iterate.asd
;; Loading file /var/cache/common-lisp-controller/1000/clisp/iterate-1.4.3/package.fas ...
;; Loaded file /var/cache/common-lisp-controller/1000/clisp/iterate-1.4.3/package.fas
;; Loading file /var/cache/common-lisp-controller/1000/clisp/iterate-1.4.3/iterate.fas ...
;; Loaded file /var/cache/common-lisp-controller/1000/clisp/iterate-1.4.3/iterate.fas
0 errors, 0 warnings
*** - EVAL: undefined function ITER
No missing package this time. I would presume that the ITERATE package I
just loaded would be responsible for providing the ITER function.
A quick check of the iterate source directory finds nothing:
find . -name \*.lisp | xargs grep -i "defun iter"
I do find a macro definition:
find . -name \*.lisp | xargs grep -i "defmacro iter"
./iterate.lisp:(defmacro iterate (&body body)
./iterate.lisp:(defmacro iter (&body body &environment env)
So it appears that clisp doesn't know about the iter macro even though
the iterate package loaded successfully. Is this a namespace problem?
>
> (defun words (file)
> (iter (for line in-file file using #'read-line)
> (nconcing (all-matches-as-strings "[a-z]+" (string-downcase line)))))
>
> (defun train (words &aux (ht (make-hash-table :test 'equalp)))
> (dolist (w words ht) (incf (gethash w ht 0))))
>
> (defvar *n-words* (train (words #p"/tmp/holmes.txt")))
>
> (defun edits (w &aux (n (length w)))
> (flet ((cat (&rest args) (apply #'concatenate 'string (mapcar #'string args)))
> (s (start &optional end) (subseq w start end)))
> (nconc (iter (for i below n) (collect (cat (s 0 i) (s (1+ i)))))
> (iter (for i below (1- n))
> (collect (cat (s 0 i) (char w (1+ i)) (char w i) (s (+ i 2)))))
> (iter (for ch in-string w)
> (dotimes (i n) (collect (cat (s 0 i) ch (s (1+ i)))))
> (dotimes (i (1+ n)) (collect (cat (s 0 i) ch (s i))))))))
>
> (defun correct (word)
> (flet ((known (ws) (remove-if-not (lambda (w) (gethash w *n-words*)) ws)))
> (iter (for w in (or (known (list word)) (known (edits word))
> (known (mapcan #'edits (edits word))) (list word)))
> (finding w maximizing (gethash w *n-words*)))))
Brian Adkins <·················@gmail.com> writes:
> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "ITERATE"
>
> Ok, so I need to find the ITERATE package, back to cliki.net for yet
> another library :( I don't recall what Python has, but Ruby has gems which
> are a very convenient package management system. Since I can't expect a
> debian package for every useful CL library,
There is a cl-iterate package.
> is asdf-install the commonly accepted way to install lisp libraries?
When asdf-install works, it's nice. The other option is to simply
follow download & install instructions by hand.
Zach
On Tue, 10 Apr 2007 12:09:49 -0400, Zach Beane wrote:
> Brian Adkins <·················@gmail.com> writes:
>
>
>> *** - SYSTEM::%FIND-PACKAGE: There is no package with name "ITERATE"
>>
>> Ok, so I need to find the ITERATE package, back to cliki.net for yet
>> another library :( I don't recall what Python has, but Ruby has gems which
>> are a very convenient package management system. Since I can't expect a
>> debian package for every useful CL library,
>
> There is a cl-iterate package.
Thanks. Just in case I screwed up the install, I blew mine away and
installed the cl-iterate Ubuntu package with same results. Then I
discovered a couple of important lines slipped through the crack when I
copied/pasted Lu�s' code :( Sorry for the false alarm.
However, when I restored the lines, I now get:
*** - UNIX error 40 (ELOOP): Too many levels of symbolic links
Ok, I guess Lisp doesn't like Unix softlinks (I had created a
softlink in /tmp for holmes.txt). I edited the file to use a regular file
and I'm able to (load "spell.lisp"). After discovering that I have to
reissue the (in-package #:spell-correct) statement in the REPL (even
though it's in spell.lisp), I finally get the following:
[2]> (in-package #:spell-correct)
#<PACKAGE SPELL-CORRECT>
SPELL-CORRECT[9]> (correct "speling")
"spelling"
Success! Not only that, but my copy if SICP just arrived via UPS - happy
day :)
Thanks to all for the help.
>
>> is asdf-install the commonly accepted way to install lisp libraries?
>
> When asdf-install works, it's nice. The other option is to simply follow
> download & install instructions by hand.
>
> Zach
Hi,
> However, when I restored the lines, I now get:
>
> *** - UNIX error 40 (ELOOP): Too many levels of symbolic links
>
> Ok, I guess Lisp doesn't like Unix softlinks
The error message makes me suspect that you might have linked the link
to itself, which is somewhat like an inifinitely recursive function
call.
> After discovering that I have to
> reissue the (in-package #:spell-correct) statement in the REPL (even
> though it's in spell.lisp)
Of course. IN-PACKAGE in a file usually doesn't apply to the REPL,
and that's a good thing, because it means that reloading a file that
you just modified doesn't suddenly affect the REPL in weird ways.
Oh, and by the way, in Debian, it's usually considered bad style to
put stuff into /usr (not /usr/local) yourself without using dpkg file
diversions. That includes /usr/share/common-lisp/systems. You should
keep /usr free from your manual downloads and let your GNU/Linux
distribution manage everything therein. If you want to install .asd
files locally, put a symlink into ~/.clc/systems/ (with ~ being your
home directory; oh, and don't be put off if the ~/.clc/systems/
directory doesn't exist yet, just create it).
Bye, Matthias
On Tue, 10 Apr 2007 10:22:26 -0700, Matthias Benkard wrote:
> Hi,
>
>> However, when I restored the lines, I now get:
>>
>> *** - UNIX error 40 (ELOOP): Too many levels of symbolic links
>>
>> Ok, I guess Lisp doesn't like Unix softlinks
>
> The error message makes me suspect that you might have linked the link
> to itself, which is somewhat like an inifinitely recursive function
> call.
>
>> After discovering that I have to
>> reissue the (in-package #:spell-correct) statement in the REPL (even
>> though it's in spell.lisp)
>
> Of course. IN-PACKAGE in a file usually doesn't apply to the REPL,
> and that's a good thing, because it means that reloading a file that
> you just modified doesn't suddenly affect the REPL in weird ways.
>
> Oh, and by the way, in Debian, it's usually considered bad style to
> put stuff into /usr (not /usr/local) yourself without using dpkg file
> diversions. That includes /usr/share/common-lisp/systems. You should
> keep /usr free from your manual downloads and let your GNU/Linux
> distribution manage everything therein. If you want to install .asd
> files locally, put a symlink into ~/.clc/systems/ (with ~ being your
> home directory; oh, and don't be put off if the ~/.clc/systems/
> directory doesn't exist yet, just create it).
>
> Bye, Matthias
Thanks for the tips Matthias.
On Tue, 10 Apr 2007 12:58:35 -0400, Brian Adkins wrote:
> On Tue, 10 Apr 2007 12:09:49 -0400, Zach Beane wrote:
>
>> Brian Adkins <·················@gmail.com> writes:
>>
> However, when I restored the lines, I now get:
>
> *** - UNIX error 40 (ELOOP): Too many levels of symbolic links
>
> Ok, I guess Lisp doesn't like Unix softlinks (I had created a
> softlink in /tmp for holmes.txt). I edited the file to use a regular file
> and I'm able to (load "spell.lisp"). After discovering that I have to
> reissue the (in-package #:spell-correct) statement in the REPL (even
> though it's in spell.lisp), I finally get the following:
My bad:
·····@airstream:~/temp$ ln -s holmes.txt /tmp/holmes.txt
should've been:
·····@airstream:~/temp$ ln -s /home/brian/temp/holmes.txt /tmp/holmes.txt
The "too many levels" error was an OS error, not a Lisp problem. ln didn't
expand the current directory so I ended up with a link to itself in the
/tmp directory.
On Tue, 10 Apr 2007 11:01:00 +0200, Lu�s Oliveira
<·············@deadspam.com> wrote:
Thanks for bringing my attention to iterate.
I took the trouble of downloading, installing, reading and playing around
with it.
This is a really powerful loop construct! Also it is more logical than
loop.
(for in ...) (for in-sequence ...) (for in-file ... using ...) etc
I think it will replace loop in my programs from now on.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
"John Thingstad" <··············@chello.no> writes:
> Thanks for bringing my attention to iterate. I took the trouble of
> downloading, installing, reading and playing around with it.
I converted its documentation to texinfo a while ago which you can now
find here <http://common-lisp.net/project/iterate/doc/index.html> and in
other formats here <http://common-lisp.net/~loliveira/tmp/iterate-manual/>.
Browsing the HTML docs works much better for me than PDF so I hope you
find it useful too.
> This is a really powerful loop construct! Also it is more logical than
> loop. (for in ...) (for in-sequence ...) (for in-file ... using ...)
> etc I think it will replace loop in my programs from now on.
My favourites are for...in-file and finding...maximizing. The
extensibility is pretty cool too.
--
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/