Hi,
I (probaby, naively) wrote escape (for HTML output) as
;(defun escape (s)
; "escape special characters in HTML"
; (apply #'concatenate 'string
; (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
; (coerce s 'list))))
;
and it worked, and worked fast enough for my a few hundred kilobytes
long HTML files.
However, LispWorks complains about too many arguments to apply, so for
LispWorks, I had to re-write it as:
;(defun escape (s)
; "escape special characters in HTML"
; (reduce #'(lambda (x y) (concatenate 'string x y))
; (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
; (coerce s 'list)) :initial-value ""))
which is several times slower than the first variant in all other
implementations.
Is there something in the standard about the limit of the length of the
list passed to apply?
David
············@gmail.com wrote:
> Is there something in the standard about the limit of the length of the
> list passed to apply?
Constant Variable CALL-ARGUMENTS-LIMIT
Constant Value:
An integer not smaller than 50 and at least as great as the value of
lambda-parameters-limit, the exact magnitude of which is implementation-dependent.
Description:
The upper exclusive bound on the number of arguments that may be passed
to a function.
Examples: None.
See Also:
lambda-parameters-limit, multiple-values-limit
Notes: None.
Paul
From: Thomas A. Russ
Subject: Re: limit on the number of arguments to apply?
Date:
Message-ID: <ymism36yn90.fsf@sevak.isi.edu>
·············@gmail.com" <············@gmail.com> writes:
·············@gmail.com" <············@gmail.com> writes:
>
> Hi,
>
> I (probaby, naively) wrote escape (for HTML output) as
>
> ;(defun escape (s)
> ; "escape special characters in HTML"
> ; (apply #'concatenate 'string
> ; (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
> ; (coerce s 'list))))
> ;
...
>
> Is there something in the standard about the limit of the length of the
> list passed to apply?
There is a constant CALL-ARGUMENTS-LIMIT which wil give you the limit
for any given implementation. The standard requires it to be at least
50, but beyond that it is implementation dependent.
Some examples:
ACL (Unix): 16384
CMUCL 536870911 !!!!!
MCL 8192
OpenMCL 8192
LispWorks (Mac) 300
So there is a large variation.
Doing something with a string stream is likely to be the simplest
solution and it will have generally reasonable performance.
It also has the advantage that you don't need to create the list of
characters to iterate over. Personally, I'm a loop fan so I would tend
to write this as something like:
(with-output-to-string (string-stream)
(loop for c across input-string
as translation = (cdr (assoc c ESCTAB))
do (if translation
(write-string transation string-stream)
(write-char c :stream string-stream))))
If you want to get fancier for performance reasons, it can often be
a net win to actually traverse the string twice. Once checking to
see if you actually need to do any translations at all and then again
to translate.
For even more fun, you can step an index across the string and make
the decision on a dynamic basis:
(defun html-escape-char (char)
;; Return the html escape sequence for CHAR if it exists
(cdr (assoc char *ESCTAB*)))
(defun html-escape-string-to-stream (input start end stream)
;; Processes string INPUT from location START below END,
;; writing the escape-sequence onto STREAM.
;; Executed for side effects only.
(loop for index from start below end
as escape-sequence = (html-escape-char (char input index))
when escape-sequence
do (when (> index start)
(write-string (subseq input start index) stream))
(write-string escape-sequence stream)
;; We reset our starting point here and continue
;; processing the string.
(setq start index)
;; Don't forget to include the suffix in case the last
;; character doesn't get escaped.
finally (when (> index start)
(write-string (subseq input start end) stream))))
(defun html-escape-string (input)
;; Plan: We loop through the string INPUT and if it is necessary
;; to escape characters, we create a string stream and use it to
;; assemble the output string containing escape sequences.
(let ((end (length input)))
(loop for index from 0 below end
as escape-sequence = (html-escape-char (char input index))
when escape-sequence
do (return-from html-escape-string
(with-output-to-string (string-stream)
;; Copy unmodified prefix
(when (> index 0)
(write-string (subseq input 0 index) string-stream))
(write-string escape-sequence string-stream)
(html-escape-string-to-stream input (1+ index) end string-stream))))
;; If we get to here, there are no characters that need escaping,
;; so we just return the input unchanged.
input))
--
Thomas A. Russ, USC/Information Sciences Institute
From: Duane Rettig
Subject: Re: limit on the number of arguments to apply?
Date:
Message-ID: <46501yf4a.fsf@franz.com>
···@sevak.isi.edu (Thomas A. Russ) writes:
> ·············@gmail.com" <············@gmail.com> writes:
> > Is there something in the standard about the limit of the length of the
> > list passed to apply?
>
> There is a constant CALL-ARGUMENTS-LIMIT which wil give you the limit
> for any given implementation. The standard requires it to be at least
> 50, but beyond that it is implementation dependent.
>
> Some examples:
> ACL (Unix): 16384
> CMUCL 536870911 !!!!!
Hmmm ...
> MCL 8192
> OpenMCL 8192
> LispWorks (Mac) 300
In reality it's just a number, and gives the user a feel for how
many arguments can be guaranteed to be passed successfully. An
arbitrary test for the limit would be extremely unfriendly, unless
the reason for the limit were in fact that the number-of-arguments
register were of a particular size (our own limit in Allegro CL
is purely historical; it comes from the 68k lisp, where we used
a 2-byte argument count register - in reality there could be a huge
number more arguments sent to a function).
However, given the ballpark number that this constant represents,
the same call may work when done at top level, but fail when done
20 nestings deep of the same function call. The size of the stack
and the nature of its growth will also affect the effectiveness of
this indicator.
I wonder if anyone has actually tried to make a call with 500 million
arguments. I wonder if any operating system at all would support
such a stack size...
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
>>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
Duane> ···@sevak.isi.edu (Thomas A. Russ) writes:
>> ·············@gmail.com" <············@gmail.com> writes:
>> > Is there something in the standard about the limit of the length of the
>> > list passed to apply?
>>
>> There is a constant CALL-ARGUMENTS-LIMIT which wil give you the limit
>> for any given implementation. The standard requires it to be at least
>> 50, but beyond that it is implementation dependent.
>>
>> Some examples:
>> ACL (Unix): 16384
>> CMUCL 536870911 !!!!!
Duane> Hmmm ...
I think that's really a lie because the Lisp stack isn't big enough to
hold 512 million args. The default control stack is about 128 MB. I
guess that would work out to be about 128 million (or was that 64
million?) args.
And I think there are some internal code issues where some platforms
can't access more than 8000 or 16000 args because of the size of the
offset field in some instructions.
Also, on CMUCL/x86 with its threads, the stack is much smaller than
128 MB, so the limit is lower too.
Ray
From: Thomas A. Russ
Subject: Re: limit on the number of arguments to apply?
Date:
Message-ID: <ymioedtznfh.fsf@sevak.isi.edu>
Raymond Toy <···········@ericsson.com> writes:
>
> >>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
>
> Duane> ···@sevak.isi.edu (Thomas A. Russ) writes:
> >>
> >> There is a constant CALL-ARGUMENTS-LIMIT which wil give you the limit
> >> for any given implementation. The standard requires it to be at least
> >> 50, but beyond that it is implementation dependent.
> >>
> >> Some examples:
> >> ACL (Unix): 16384
> >> CMUCL 536870911 !!!!!
>
> Duane> Hmmm ...
>
> I think that's really a lie because the Lisp stack isn't big enough to
> hold 512 million args. The default control stack is about 128 MB. I
> guess that would work out to be about 128 million (or was that 64
> million?) args.
>
> And I think there are some internal code issues where some platforms
> can't access more than 8000 or 16000 args because of the size of the
> offset field in some instructions.
>
> Also, on CMUCL/x86 with its threads, the stack is much smaller than
> 128 MB, so the limit is lower too.
I don't know if it makes any difference, but the CMUCL in question was
18c on a Sparc/Solaris system. I don't generally run it, though, but
since it was lying around, I thought I would ask it as well.
>
> Ray
--
Thomas A. Russ, USC/Information Sciences Institute
Raymond Toy <···········@ericsson.com> writes:
> >>>>> "Duane" == Duane Rettig <·····@franz.com> writes:
>
> Duane> ···@sevak.isi.edu (Thomas A. Russ) writes:
> >> ·············@gmail.com" <············@gmail.com> writes:
>
> >> > Is there something in the standard about the limit of the length of the
> >> > list passed to apply?
> >>
> >> There is a constant CALL-ARGUMENTS-LIMIT which wil give you the limit
> >> for any given implementation. The standard requires it to be at least
> >> 50, but beyond that it is implementation dependent.
> >>
> >> Some examples:
> >> ACL (Unix): 16384
> >> CMUCL 536870911 !!!!!
>
> Duane> Hmmm ...
>
> I think that's really a lie because the Lisp stack isn't big enough to
> hold 512 million args. The default control stack is about 128 MB. I
> guess that would work out to be about 128 million (or was that 64
> million?) args.
>
> And I think there are some internal code issues where some platforms
> can't access more than 8000 or 16000 args because of the size of the
> offset field in some instructions.
>
> Also, on CMUCL/x86 with its threads, the stack is much smaller than
> 128 MB, so the limit is lower too.
It might be not a lie, because the implementation is not obliged to
copy the parameters on the stack:
(defun f (&rest arguments)
(do-something-with-them arguments))
(apply (function f) list-of-500e6-items)
Unfortunately, the specification of CALL-ARGUMENT-LIMIT is quite vague:
The upper exclusive bound on the number of [15]arguments that may be
passed to a [16]function.
Must an implementation guarantee that any call with less than
CALL-ARGUMENT-LIMIT arguments will succeed?
Or does it mean only that any call with more than CALL-ARGUMENT-LIMIT
won't succeed? (But some calls with less than CALL-ARGUMENT-LIMIT
arguments may fail too).
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
·············@gmail.com" <············@gmail.com> writes:
> Hi,
>
> I (probaby, naively) wrote escape (for HTML output) as
>
> ;(defun escape (s)
> ; "escape special characters in HTML"
> ; (apply #'concatenate 'string
> ; (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
> ; (coerce s 'list))))
> ;
>
> and it worked, and worked fast enough for my a few hundred kilobytes
> long HTML files.
>
> However, LispWorks complains about too many arguments to apply, so for
> LispWorks, I had to re-write it as:
>
> ;(defun escape (s)
> ; "escape special characters in HTML"
> ; (reduce #'(lambda (x y) (concatenate 'string x y))
> ; (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
> ; (coerce s 'list)) :initial-value ""))
>
> which is several times slower than the first variant in all other
> implementations.
>
> Is there something in the standard about the limit of the length of the
> list passed to apply?
Yes: CALL-ARGUMENTS-LIMIT
(defun concat-strings (strings)
(let ((result (make-array (reduce (function +) strings :key (function length))
:element-type 'character))
(start 0))
(dolist (string strings result)
(replace result string :start1 start)
(incf start (length string)))))
(defun escape (s)
"escape special characters in HTML"
(concat-strings
(mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
(coerce s 'list))))
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
> (defun concat-strings (strings)
> (let ((result (make-array (reduce (function +) strings :key
(function length))
> :element-type 'character))
> (start 0))
> (dolist (string strings result)
> (replace result string :start1 start)
> (incf start (length string)))))
>
>
> (defun escape (s)
> "escape special characters in HTML"
> (concat-strings
> (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
> (coerce s 'list))))
>
This is the level of detail I don't want to put into my program; or I
would not coerce a string as a list, but iterate over it instead; that
would be faster and would not cons a few thousand times, of course.
Writing to a string works, thank you.
David
·············@gmail.com" <············@gmail.com> writes:
> > (defun concat-strings (strings)
> > (let ((result (make-array (reduce (function +) strings :key
> (function length))
> > :element-type 'character))
> > (start 0))
> > (dolist (string strings result)
> > (replace result string :start1 start)
> > (incf start (length string)))))
> >
> >
> > (defun escape (s)
> > "escape special characters in HTML"
> > (concat-strings
> > (mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
> > (coerce s 'list))))
> >
>
> This is the level of detail I don't want to put into my program; or I
> would not coerce a string as a list, but iterate over it instead; that
> would be faster and would not cons a few thousand times, of course.
>
> Writing to a string works, thank you.
I can understand that you don't want to enter into the details.
That's why I think you should not write:
(defun escape (s)
(with-output-to-string (stream)
(loop for char in s
for esc = (cdr assoc char ESCTAB)
if (null esc)
(write-char char stream)
else
(loop for esc-char in esc
do (write-char esc-char stream)))))
but rather:
(defun escape (s)
"escape special characters in HTML"
(concat-strings ;; See, no details!
(mapcar (lambda (c) (or (cdr (assoc c ESCTAB)) (string c)))
(coerce s 'list))))
Now, if you want to write:
(defun concat-strings (strings)
(with-output-to-string (stream)
(dolist (string strings) (princ string stream))))
instead of:
(defun concat-strings (strings)
(let ((result (make-array (reduce (function +) strings :key (function length))
:element-type 'character))
(start 0))
(dolist (string strings result)
(replace result string :start1 start)
(incf start (length string)))))
you're welcome.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Nobody can fix the economy. Nobody can be trusted with their finger
on the button. Nobody's perfect. VOTE FOR NOBODY.
> Now, if you want to write:
>
> (defun concat-strings (strings)
> (with-output-to-string (stream)
> (dolist (string strings) (princ string stream))))
>
> instead of:
>
> (defun concat-strings (strings)
> (let ((result (make-array (reduce (function +) strings :key
(function length))
> :element-type 'character))
> (start 0))
> (dolist (string strings result)
> (replace result string :start1 start)
> (incf start (length string)))))
>
> you're welcome.
Yes, that's what I wrote, and it worked, but finally realized that
since the efficiency is an issue, I don't want to go through a list at
all, and rewrote it as a loop over a string. Less conses, faster
execution. Probably worth the effort.
Ingvar <······@hexapodia.net> writes:
> You're (probably) better off with something like:
>
> (defun escape (s)
> (with-output-to-string (stream)
> (loop for char in s
> for esc = (cdr assoc char ESCTAB)
> if (null esc)
> (write-char char stream)
> else
> (loop for esc-char in esc
> do (write-char esc-char stream)))))
And I ask: How does WITH-OUTPUT-TO-STRING know how many characters to
allocate for the output string? How many times will WRITE-CHAR
allocate a new string copying the characters written up to now?
--
__Pascal Bourguignon__ http://www.informatimago.com/