Hi,
I currently have some code that needs to write an output file for use
on windows systems that the users can edit with notepad (yes, that
awful thing that doesn't understand <lf> line endings).
I want to use essentially something like:
(with-open-file (o "myfile.txt" :direction :output)
(dotimes (i 2)
(format o "This is line ~D.~%~%" (1+ i))))
to get the output:
This is line 1.
This is line 2.
The problem is that the above output looks like (in Notepad):
This is line 1.**This is line 2. (where the "*" is a <lf> escape
character)
I figured I could write a macro to send <cr><lf> instead of just <lf>,
perhaps something like this:
(defmacro format-crlf (mystream &rest args)
(let (mystring newstring)
(setq mystring (apply 'format nil args))
(setq newstring (string-replace mystring
(format nil "~A" #\Newline)
(format nil "~A~A" #\Return #\Newline)))
`(format ,mystream ,newstring)))
This seems to work for calls such as
(with-open-file (o "myfile.txt" :direction :output)
(dotimes (i 2)
(format-crlf "This is line ~A~%~%" (1+ i))))
But if the format string is itself a variable, such as
(setq myfmtstr "This is line ~A~%~%")
then the code
(with-open-file (o "myfile.txt" :direction :output)
(dotimes (i 2)
(format-crlf myfmtstr (1+ i))))
generates an error that the function myfmtstr is not defined.
How can I modify the macro (or re-do it completely) so that it will do
everything that "format" will do, but will put the <cr><lf> line
endings in place.
(by the way, the lisp implementation we're required to use is an old
version of KCL, and doesn't have any options for changing the line
endings).
Thanks in advance for any help you can provide!
bufie
From: Kent M Pitman
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <uir8nq60k.fsf@nhplace.com>
bufie <·····@spamneggs.com> writes:
> (defmacro format-crlf (mystream &rest args)
> (let (mystring newstring)
> (setq mystring (apply 'format nil args))
> (setq newstring (string-replace mystring
> (format nil "~A" #\Newline)
> (format nil "~A~A" #\Return #\Newline)))
> `(format ,mystream ,newstring)))
> ...
> How can I modify the macro (or re-do it completely) so that it will do
> everything that "format" will do, but will put the <cr><lf> line
> endings in place.
(if (stringp mystring)
(setq newstring (string-replace mystring
(format nil "~A" #\Newline)
(format nil "~A~A" #\Return #\Newline)))
(setq newstring `(string-replace ,mystring
(format nil "~A" #\Newline)
(format nil "~A~A" #\Return #\Newline))))
You could consider modularizing that better by factoring out the
string-replace call into a separate function, but that's the basic idea.
Oh, and either way, you should have a #. in front of all of those calls
to (format nil ...). There's no reason you need to be redoing that
FORMAT call every time--it will return the same answer every time it's
called. At runtime, you should be passing a constant string. Or you can
factor that out into a defconstant initialized by a call to FORMAT and
it will have the same effect.
Incidentally, another way to do this is to write this as a function
rather than as a macro, and then write a compiler-macro to optimize
the common case of a literal string argument. I'll let someone else
who has more time volunteer example code.
From: Ari Johnson
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <m2d4yv4tzd.fsf@hermes.theari.com>
Kent M Pitman <······@nhplace.com> writes:
> bufie <·····@spamneggs.com> writes:
>
>> (defmacro format-crlf (mystream &rest args)
>> (let (mystring newstring)
>> (setq mystring (apply 'format nil args))
>> (setq newstring (string-replace mystring
>> (format nil "~A" #\Newline)
>> (format nil "~A~A" #\Return #\Newline)))
>> `(format ,mystream ,newstring)))
>> ...
>> How can I modify the macro (or re-do it completely) so that it will do
>> everything that "format" will do, but will put the <cr><lf> line
>> endings in place.
>
> (if (stringp mystring)
> (setq newstring (string-replace mystring
> (format nil "~A" #\Newline)
> (format nil "~A~A" #\Return #\Newline)))
> (setq newstring `(string-replace ,mystring
> (format nil "~A" #\Newline)
> (format nil "~A~A" #\Return #\Newline))))
>
> You could consider modularizing that better by factoring out the
> string-replace call into a separate function, but that's the basic idea.
>
> Oh, and either way, you should have a #. in front of all of those calls
> to (format nil ...). There's no reason you need to be redoing that
> FORMAT call every time--it will return the same answer every time it's
> called. At runtime, you should be passing a constant string. Or you can
> factor that out into a defconstant initialized by a call to FORMAT and
> it will have the same effect.
>
> Incidentally, another way to do this is to write this as a function
> rather than as a macro, and then write a compiler-macro to optimize
> the common case of a literal string argument. I'll let someone else
> who has more time volunteer example code.
I don't have more time, but I am sick of what I was doing so I will
give it a shot. Note that I have limited experience with compiler
macros, at best, but this one seems fairly straightforward.
(defun %fix-newlines (string)
(string-replace string
#.(string #\Newline)
#.(concatenate 'string
(string #\Return)
(string #\Newline))))
(defun fix-newlines (string)
(%fix-newlines string))
(define-compiler-macro fix-newlines (&whole form string)
(if (and (constantp string) (stringp string))
(%fix-newlines string)
form))
bufie wrote:
> I currently have some code that needs to write an output file for use
> on windows systems that the users can edit with notepad (yes, that
> awful thing that doesn't understand <lf> line endings).
...
> I figured I could write a macro to send <cr><lf> instead of just <lf>,
> perhaps something like this:
...
> How can I modify the macro (or re-do it completely) so that it will do
> everything that "format" will do, but will put the <cr><lf> line
> endings in place.
Preferred solution:
Have your users install a decent text editor. e.g. SciTE
http://www.scintilla.org/SciTE.html
(Assuming that having them install a decent OS is out of the question. ;)
Alternate solution:
(defun endl-crlf (string)
"Replace all the ~%'s in string with CRLFs."
(let ((s string))
(do ((pos (search "~%" s)
(search "~%" s)))
((null pos) s)
(setf (char s pos) #\Return
(char s (1+ pos)) #\Newline))))
(defmacro ms-format (destination control-string &rest args)
"Wrap format so that ~% becomes CRLF."
(if (stringp control-string)
`(format ,destination ,(endl-crlf control-string) ,@args)
`(format ,destination (endl-crlf ,control-string) ,@args)))
- Daniel
On Jul 13, 11:28 pm, D Herring <········@at.tentpost.dot.com> wrote:
> Preferred solution:
> Have your users install a decent text editor. e.g. SciTEhttp://www.scintilla.org/SciTE.html
> (Assuming that having them install a decent OS is out of the question. ;)
Excellent answer! Ah, if only it were that easy! :)
> Alternate solution:
> (defun endl-crlf (string)
> "Replace all the ~%'s in string with CRLFs."
> (let ((s string))
> (do ((pos (search "~%" s)
> (search "~%" s)))
> ((null pos) s)
> (setf (char s pos) #\Return
> (char s (1+ pos)) #\Newline))))
>
> (defmacro ms-format (destination control-string &rest args)
> "Wrap format so that ~% becomes CRLF."
> (if (stringp control-string)
> `(format ,destination ,(endl-crlf control-string) ,@args)
> `(format ,destination (endl-crlf ,control-string) ,@args)))
This seems to do exactly what I need, and is nice and clean. I had
played around with something similar, but was trying to do the
substitution *after* the expansion, which didn't work very well.
Replacing the "~%" is a *much* better solution!
Thanks, Daniel!
bufie
From: Richard M Kreuter
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <87abtyjsmr.fsf@tan-ru.localdomain>
bufie <·····@spamneggs.com> writes:
> On Jul 13, 11:28 pm, D Herring <········@at.tentpost.dot.com> wrote:
>
>> Alternate solution:
>> (defun endl-crlf (string)
>> "Replace all the ~%'s in string with CRLFs."
>> (let ((s string))
>> (do ((pos (search "~%" s)
>> (search "~%" s)))
>> ((null pos) s)
>> (setf (char s pos) #\Return
>> (char s (1+ pos)) #\Newline))))
>>
>> (defmacro ms-format (destination control-string &rest args)
>> "Wrap format so that ~% becomes CRLF."
>> (if (stringp control-string)
>> `(format ,destination ,(endl-crlf control-string) ,@args)
>> `(format ,destination (endl-crlf ,control-string) ,@args)))
>
> This seems to do exactly what I need, and is nice and clean. I had
> played around with something similar, but was trying to do the
> substitution *after* the expansion, which didn't work very well.
> Replacing the "~%" is a *much* better solution!
I only skimmed this thread quickly, but the above doesn't solve the
general problem of ensuring that each linefeed in the output is
preceded by a newline:
* ~% can take a prefix parameter, thereby producing several newlines
in succession.
* Another FORMAT directive, ~&, which produces a newline in case the
stream is not known to be at the beginning of a line. ~& also takes
a prefix parameter.
* The pretty printer can, of course, produce Newlines fairly freely.
In particular, ~<··@> performs paragraph filling.
* The ~? directive and its variants treat subsequent arguments as
control strings, and so frobbing only the second argument doesn't
suffice, in general.
* Finally, the objects to be printed can contain linefeeds, or else
have linefeeds as part of their printed representations. You didn't
mention what to do about these cases; I'd assume you actually want
all Linefeeds to be preceded by Returns.
Given the (above and perhaps more), it's likely that you'll get closer
to the desired result with your original strategy of accumulating the
formatted string and then replacing all the linefeeds, rather than by
trying to munge the control string. There's no reason to do this with
a macro, however: an ordinary function will do.
--
RmK
On Jul 14, 8:41 pm, Richard M Kreuter <·······@progn.net> wrote:
> Given the (above and perhaps more), it's likely that you'll get closer
> to the desired result with your original strategy of accumulating the
> formatted string and then replacing all the linefeeds, rather than by
> trying to munge the control string. There's no reason to do this with
> a macro, however: an ordinary function will do.
Thanks for the reply, Richard.
You make a good point -- in my usage, there is certainly a possibility
that the objects to be printed may contain linefeeds as well. The
reason I wanted to use a macro was so that the string-replace would be
done at compile-time where possible rather than at run time to make
thing more efficient...
I know very little about all of the power of "format" other than
relatively basic stuff (which is probably quite apparent!). If you
have any other suggestions on how to implement this -- again,
preferably as a macro. The function method is very straightforward,
and I had it working just fine (with the exception that I didn't
consider the case where the string being printed already had a crlf in
it -- then it would end up as crcrlf... but that would be an easy
fix...
Thanks again!
bufie
From: Richard M Kreuter
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <87y7hii4g7.fsf@tan-ru.localdomain>
bufie <·····@spamneggs.com> writes:
> You make a good point -- in my usage, there is certainly a
> possibility that the objects to be printed may contain linefeeds as
> well. The reason I wanted to use a macro was so that the
> string-replace would be done at compile-time where possible rather
> than at run time to make thing more efficient...
If it's starting to be clear why you'll get the wrong results when any
of the printed objects contain linefeeds, then it should become
apparent why a macro is only of limited use here. Consider:
(let ((x (coerce '(#\a #\b #\linefeed #\c #\d) 'string)))
(format-macro t "~A~%" x))
The FORMAT-MACRO macro function receives the symbol T, the string
"~A~%", and the symbol X as arguments. Knowing nothing about X, the
macro function can't accomplish the whole job; you'll still need to
transcribe things at run time. Notice also that if you replace any
linefeeds during macroexpansion, then you'll have to be extra careful
not to repeat the process at run time.
The only case where you can compute the string at compile-time is when
all the arguments other than the destination are constants. Probably
this isn't worth optimizing for, but if you do want to do so, the
right tool is to use a compiler macro:
http://www.lisp.org/HyperSpec/Body/sec_3-2-2-1.html
Since it's not very likely that you'll be invoking the function with
constant arguments very often, I wouldn't worry too much about trying
to perform the computation at compile-time, except perhaps as an
exercise.
> I know very little about all of the power of "format" other than
> relatively basic stuff (which is probably quite apparent!).
Just because you only know a little bit of FORMAT now doesn't mean you
won't learn more later. If you proceed with a macro that only
produces the right results for a common but restricted subset of
control strings and arguments, what will you do as your FORMAT-fu
improves?
--
RmK
On Jul 15, 12:08 am, Richard M Kreuter <·······@progn.net> wrote:
> bufie <·····@spamneggs.com> writes:
> > You make a good point -- in my usage, there is certainly a
> > possibility that the objects to be printed may contain linefeeds as
> > well. The reason I wanted to use a macro was so that the
> > string-replace would be done at compile-time where possible rather
> > than at run time to make thing more efficient...
...
> Since it's not very likely that you'll be invoking the function with
> constant arguments very often, I wouldn't worry too much about trying
> to perform the computation at compile-time, except perhaps as an
> exercise.
That's a good point. So then, if I were to just do it as a function,
what's the best way to ensure that if the string I'm processing ends
up with crlf, then the replacement would be crcrlf... I thought this
would be trivial, but in looking at it, I'm not so sure...
The string-replace function would would essentially need to be
modified to check to see if the <lf> that it's going to replace is
following a a <cr>, and if so, don't replace it.
The case where the format string was modified was nice because the "~
%" provided two char spots to insert the <cr> and <lf> chars.
However, if we work on the expanded string, the <lf> only takes one
spot. What is the most efficient way to accomplish this "2 for 1"
substitution? It seems that since it's a specific case, we should be
able to have a replacement function that's more efficient than using
string-replace.
Of course, if efficiency is completely disregarded, a second call to
string-replace would make the original code (modified to work as a
function) have the desired result:
(defun format-crlf (mystream &rest args)
(let (mystring newstring)
(setq mystring (apply 'format nil args))
(setq newstring (string-replace mystring
(format nil "~A" #\Newline)
(format nil "~A~A" #\Return #\Linefeed)))
(setq newstring (string-replace newstring
(format nil "~A~A~A" #\Return #\Return #
\Linefeed)
(format nil "~A~A" #\Return #\Linefeed)))
(format mystream newstring)))
but more efficient would definitely be better... A simple 2-for-1
substitution seems like it should be very simple, but I'm just not
coming up with it...
Other ideas?
Thanks again!
bufie
From: Richard M Kreuter
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <87ir8lidwm.fsf@tan-ru.localdomain>
bufie <·····@spamneggs.com> writes:
> On Jul 15, 12:08 am, Richard M Kreuter <·······@progn.net> wrote:
> [I]f I were to just do it as a function, what's the best way to
> ensure that if the string I'm processing ends up with crlf, then the
> replacement would be crcrlf... I thought this would be trivial, but
> in looking at it, I'm not so sure...
Some regular expression libraries support fixed-length "lookbehinds".
Using CL-PPCRE, you might say:
(defparameter *crlf* (coerce '(#\Return #\Linefeed) 'string))
(let ((source-string (coerce '(#\a #\Return #\b #\Linefeed #\c
#\Return #\Linefeed #\d)
'string)))
(ppcre:regex-replace "(?<!\\r)\\n" source-string *crlf*))
I personally don't like regular expressions very much, so I'd probably
write the transformation out explicitly:
(defun replace-linefeeds (string)
(with-output-to-string (string-stream)
(loop
with just-emitted-return = nil
for character across string
if (and (char= character #\Linefeed)
(not just-emitted-return))
do (write-char #\Return string-stream)
(setf just-emitted-return t)
if (char= character #\Return)
do (write-char character string-stream)
(setf just-emitted-return t)
else
do (write-char character string-stream)
(setf just-emitted-return nil))))
--
RmK
bufie wrote:
> On Jul 15, 12:08 am, Richard M Kreuter <·······@progn.net> wrote:
>> bufie <·····@spamneggs.com> writes:
>>> You make a good point -- in my usage, there is certainly a
>>> possibility that the objects to be printed may contain linefeeds as
>>> well. The reason I wanted to use a macro was so that the
>>> string-replace would be done at compile-time where possible rather
>>> than at run time to make thing more efficient...
> ...
>> Since it's not very likely that you'll be invoking the function with
>> constant arguments very often, I wouldn't worry too much about trying
>> to perform the computation at compile-time, except perhaps as an
>> exercise.
>
> That's a good point. So then, if I were to just do it as a function,
> what's the best way to ensure that if the string I'm processing ends
> up with crlf, then the replacement would be crcrlf... I thought this
> would be trivial, but in looking at it, I'm not so sure...
Here's another try.
(defun lf->crlf (string)
"Make sure all the linefeeds in string follow carriage returns."
(let ((s "") ; collect the lines with CRLF
(c0 0)) ; start of the current line
(do ((c1 (position #\Linefeed string)
(position #\Linefeed string :start (1+ c1))))
((null c1) (concatenate 'string s (subseq string c0)))
(if (= c0 c1)
; first char is lf -> not following cr
(setf s (concatenate 'string
s
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1))
; check for cr -- do nothing if its already there
(unless (eql #\Return (char string (1- c1)))
(setf s (concatenate 'string
s
(subseq string c0 (1- c1))
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1)))))))
- Daniel
D Herring <········@at.tentpost.dot.com> writes:
> bufie wrote:
>> I currently have some code that needs to write an output file for use
>> on windows systems that the users can edit with notepad (yes, that
>> awful thing that doesn't understand <lf> line endings).
> ...
>> I figured I could write a macro to send <cr><lf> instead of just <lf>,
>> perhaps something like this:
> ...
>> How can I modify the macro (or re-do it completely) so that it will do
>> everything that "format" will do, but will put the <cr><lf> line
>> endings in place.
>
> Preferred solution:
> Have your users install a decent text editor. e.g. SciTE
> http://www.scintilla.org/SciTE.html
> (Assuming that having them install a decent OS is out of the question. ;)
>
>
> Alternate solution:
> (defun endl-crlf (string)
> "Replace all the ~%'s in string with CRLFs."
> (let ((s string))
> (do ((pos (search "~%" s)
> (search "~%" s)))
> ((null pos) s)
> (setf (char s pos) #\Return
> (char s (1+ pos)) #\Newline))))
s/Newline/Linefeed/
> (defmacro ms-format (destination control-string &rest args)
> "Wrap format so that ~% becomes CRLF."
> (if (stringp control-string)
> `(format ,destination ,(endl-crlf control-string) ,@args)
> `(format ,destination (endl-crlf ,control-string) ,@args)))
>
>
> - Daniel
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
On Jul 14, 1:17 am, Pascal Bourguignon <····@informatimago.com> wrote:
> > (defun endl-crlf (string)
> > "Replace all the ~%'s in string with CRLFs."
> > (let ((s string))
> > (do ((pos (search "~%" s)
> > (search "~%" s)))
> > ((null pos) s)
> > (setf (char s pos) #\Return
> > (char s (1+ pos)) #\Newline))))
> s/Newline/Linefeed/
as this is a lisp group, I was surprised to see perl (I think!)
syntax.
so you're saying that the above should use #\Linefeed instead of
#\Newline? what is the difference? on my implementation, both
#\Linefeed and #\Newline return a char-int of 10. Are there other
systems
where they are different?
bufie <·····@spamneggs.com> writes:
> On Jul 14, 1:17 am, Pascal Bourguignon <····@informatimago.com> wrote:
>
>> > (defun endl-crlf (string)
>> > "Replace all the ~%'s in string with CRLFs."
>> > (let ((s string))
>> > (do ((pos (search "~%" s)
>> > (search "~%" s)))
>> > ((null pos) s)
>> > (setf (char s pos) #\Return
>> > (char s (1+ pos)) #\Newline))))
>
>> s/Newline/Linefeed/
>
> as this is a lisp group, I was surprised to see perl (I think!)
> syntax.
>
> so you're saying that the above should use #\Linefeed instead of
> #\Newline? what is the difference?
The difference is that on a different implementation (or possibly just
a different OS), #\newline may be converted to 13, or 13 10.
> on my implementation, both #\Linefeed and #\Newline return a
> char-int of 10. Are there other systems where they are different?
Yes.
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
On Jul 14, 8:40 am, Pascal Bourguignon <····@informatimago.com> wrote:
> > so you're saying that the above should use #\Linefeed instead of
> > #\Newline? what is the difference?
>
> The difference is that on a different implementation (or possibly just
> a different OS), #\newline may be converted to 13, or 13 10.
Ok, that makes sense...
> > on my implementation, both #\Linefeed and #\Newline return a
> > char-int of 10. Are there other systems where they are different?
>
> Yes.
Thanks for the clarification -- I didn't know this!
bufie
bufie <·····@spamneggs.com> writes:
> I currently have some code that needs to write an output file for use
> on windows systems that the users can edit with notepad (yes, that
> awful thing that doesn't understand <lf> line endings).
>
> I want to use essentially something like:
>
> (with-open-file (o "myfile.txt" :direction :output)
> (dotimes (i 2)
> (format o "This is line ~D.~%~%" (1+ i))))
My first advice would be to use an (unfortunately implementation
specific) external format such as:
(with-open-file (o "myfile.txt"
:direction :output
:external-format #+clisp (ext:make-encoding
:charset charset:windows-1252
:line-terminator :dos)
#-clisp (error "How do you specify an ~
external format encoding WINDOWS-1252 with CR-LF line ~
terminators in ~A" (lisp-implementation-type)))
(dotimes (i 2)
(format o "This is line ~D.~2%" (1+ i))))
> I figured I could write a macro to send <cr><lf> instead of just <lf>,
> perhaps something like this:
>
> (defmacro format-crlf (mystream &rest args)
You don't need a macro for this.
> (let (mystring newstring)
> (setq mystring (apply 'format nil args))
> (setq newstring (string-replace mystring
> (format nil "~A" #\Newline)
> (format nil "~A~A" #\Return #\Newline)))
> `(format ,mystream ,newstring)))
This is wrong, because #\newline may be converted to either #\return,
#\linefeed or #(#\return #\linefeed). At the very least, you should
write:
(format nil "~C~C" #\return #\linefeed)
But since the presence of #\return and #\linefeed is implementation
dependant, you should really guard it with:
#+#.(cl:if (cl:ignore-errors (cl:read-from-string "(#\\Return #\\Linefeed)"))
'(:and) '(:or))
(format nil "~C~C" #\return #\linefeed)
#-#.(cl:if (cl:ignore-errors (cl:read-from-string "(#\\Return #\\Linefeed)"))
'(:and) '(:or))
(error "There are no #\\Return or #\\Linefeed in ~A"
(lisp-implementation-type))
Therefore, to handle properly the second case, I would advise to treat
this output for what it is: a binary format:
(with-open-file (o "myfile.txt"
:direction :output
:element-type '(unsigned-byte 8))
(let ((ascii:*newline* :crlf))
(dotimes (i 2)
(ascii:ascii-format o "This is line ~D.~2%" (1+ i)))))
;; See: http://darcs.informatimago.com/public/lisp/common-lisp/ascii.lisp
;; and: http://www.informatimago.com/develop/lisp
Now, if the file you write is not encoded in ASCII but in WINDOWS-1252
or something else, you'll have to convert your non ASCII strings
explicitely, and the problem is that encoding/decoding operators are
implementation specific or inexistant.
(with-open-file (o "myfile.txt"
:direction :output
:element-type '(unsigned-byte 8))
(dotimes (i 2)
(write-sequence
(encode-in-windows-1252 (format nil "Th�s �s line ~D." (1+ i))) o)
(write-sequence (vector ascii:cr ascii:lf) o)))
Up to you to implement encode-in-windows-1252. If you're lucky, your
implementation will allow you to do it easily:
(defun encode-in-windows-1252 (string)
(assert (not (find #\newline string)))
;; Note, AFAIK there's now way to specify the line-terminator encoding
;; to sbcl, so we don't treat newlines in this function.
#+clisp (ext:convert-string-to-bytes charset:windows-1252)
#+sbcl (sb-ext:string-to-octets string :external-format :windows-1252)
#-(or clisp sbcl)
(error "How do I convert strings to windows-1252 byte sequences in ~A?"
(lisp-implementation-type)))
Of course, you can hide all this in a function notepad-format and a
macro with-notepad-file:
(with-notepad-file (o "myfile.txt" :direction :output)
(dotimes (i 2)
(notepad-format o "Th�s �s line ~D.~%" (1+ i))))
And perhaps notepad expects utf-8 instead of windows-1252, I don't
really know anything about MS-Windows...
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
From: John Thingstad
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <f7e9ll$7iq$1@news.get.no>
> (by the way, the lisp implementation we're required to use is an old
> version of KCL, and doesn't have any options for changing the line
> endings).
>
> Thanks in advance for any help you can provide!
>
> bufie
>
How about:
(defun convert-linefeed (string)
(with-output-to-string (out)
(loop for char across string)
(case char
(#\Newline (write-char #\Return out) (write-char #\Linefeed out))
(otherwise (write-char char out))))))
On Jul 15, 5:14 pm, "John Thingstad" <··············@chello.no> wrote:
> > (by the way, the lisp implementation we're required to use is an old
> > version of KCL, and doesn't have any options for changing the line
> > endings).
> How about:
>
> (defun convert-linefeed (string)
> (with-output-to-string (out)
> (loop for char across string)
> (case char
> (#\Newline (write-char #\Return out) (write-char #\Linefeed out))
> (otherwise (write-char char out))))))
This and the previous reply both looked promising, but it appears that
our implementation doesn't allow the use of "across" in a loop! So I
had to explore further...
Our lisp implementation does provide a "string-split" function that
takes two strings as its args (the string to split and the "match"
string), which essentially breaks the source string before and after
each occurrence of the "match" string (deleting each occurrence of the
match string), and returns a list of the resulting substrings.
I used that to come up with the following:
(defun lf2crlf (mystring)
(let ((newstr "")
(crlf (coerce '(#\Return #\Linefeed) 'string ))
(substrs (string-split mystring #\Newline)))
;; just grab first substring by itself
(setq newstr (pop substrs))
;; if it ends with a #\Return, strip it off
(when (eql #\Return (char newstr (1- (length newstr))))
(setq newstr (subseq newstr 0 (1- (length newstr)))))
;; process each substring
(dolist (s substrs)
;; if this substring ends with #\Return, strip it off
(when (eql #\Return (char s (1- (length s))))
(setq s (subseq s 0 (1- (length s)))))
;; now add crlf and this substring to newstr
(setq newstr (concatenate 'string newstr crlf s)))
;; and return newstr
newstr))
This seems to work just fine, and seems that it should be much more
efficient than repeated calls to string-replace and format... (is
this correct?)
Then for the whole solution, we could have a function that looks like:
(defun ms-format (mystream &rest args)
(let (mystring newstr
(crlf (coerce '(#\Return #\Linefeed) 'string ))
substrs)
;; generate the string resulting from the format command
(setq mystring (apply 'format nil args))
;; split mystring at #\Newline characters into substrings
(setq substrs (string-split mystring #\Newline))
(setq newstr (pop substrs))
;; if it ends with a #\Return, strip it off
(when (eql #\Return (char newstr (1- (length newstr))))
(setq newstr (subseq newstr 0 (1- (length newstr)))))
;; process each substring
(dolist (s substrs)
;; if this substring ends with #\Return, strip it off
(when (eql #\Return (char s (1- (length s))))
(setq s (subseq s 0 (1- (length s)))))
;; now add crlf and this substring to newstr
(setq newstr (concatenate 'string newstr crlf s)))
(format mystream newstr)))
Does anyone see problems with the above?
thanks!
bufie
From: John Thingstad
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <f7g57i$1n4$1@news.get.no>
"bufie" <·····@spamneggs.com> skrev i melding
·····························@j4g2000prf.googlegroups.com...
> On Jul 15, 5:14 pm, "John Thingstad" <··············@chello.no> wrote:
>> > (by the way, the lisp implementation we're required to use is an old
>> > version of KCL, and doesn't have any options for changing the line
>> > endings).
>
>> How about:
>>
>> (defun convert-linefeed (string)
>> (with-output-to-string (out)
>> (loop for char across string)
>> (case char
>> (#\Newline (write-char #\Return out) (write-char #\Linefeed
>> out))
>> (otherwise (write-char char out))))))
A better way is to grab the iterate package
http://common-lisp.net/project/iterate/
The code is now:
(defun convert-linefeed (text)
(with-output-to-string (out)
(iter (for char in-string text)
(case char
(\#Linefeed (write-char #\Return out) (write-char #\Linefeed
out))
(otherwise (write-char char out)))))
On Jul 16, 10:10 am, "John Thingstad" <··············@chello.no>
wrote:
> "bufie" <·····@spamneggs.com> skrev i ····································@j4g2000prf.googlegroups.com...
>
> > On Jul 15, 5:14 pm, "John Thingstad" <··············@chello.no> wrote:
> >> > (by the way, the lisp implementation we're required to use is an old
> >> > version of KCL, and doesn't have any options for changing the line
> >> > endings).
>
> >> How about:
>
> >> (defun convert-linefeed (string)
> >> (with-output-to-string (out)
> >> (loop for char across string)
> >> (case char
> >> (#\Newline (write-char #\Return out) (write-char #\Linefeed
> >> out))
> >> (otherwise (write-char char out))))))
>
> A better way is to grab the iterate package http://common-lisp.net/project/iterate/
>
> The code is now:
>
> (defun convert-linefeed (text)
> (with-output-to-string (out)
> (iter (for char in-string text)
> (case char
> (\#Linefeed (write-char #\Return out) (write-char #\Linefeed out))
> (otherwise (write-char char out)))))
This is very nice, indeed. However, it doesn't seem to handle the
case where the string being converted already has a crlf -- then it'll
end up as crcrlf with the above, right? How can we use this idea but
eliminate the double cr?
From: Duane Rettig
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <o07ip03yi7.fsf@gemini.franz.com>
bufie <·····@spamneggs.com> writes:
> On Jul 16, 10:10 am, "John Thingstad" <··············@chello.no>
> wrote:
>> "bufie" <·····@spamneggs.com> skrev i ····································@j4g2000prf.googlegroups.com...
>>
>> > On Jul 15, 5:14 pm, "John Thingstad" <··············@chello.no> wrote:
>> >> > (by the way, the lisp implementation we're required to use is an old
>> >> > version of KCL, and doesn't have any options for changing the line
>> >> > endings).
>>
>> >> How about:
>>
>> >> (defun convert-linefeed (string)
>> >> (with-output-to-string (out)
>> >> (loop for char across string)
>> >> (case char
>> >> (#\Newline (write-char #\Return out) (write-char #\Linefeed
>> >> out))
>> >> (otherwise (write-char char out))))))
>>
>> A better way is to grab the iterate package http://common-lisp.net/project/iterate/
>>
>> The code is now:
>>
>> (defun convert-linefeed (text)
>> (with-output-to-string (out)
>> (iter (for char in-string text)
>> (case char
>> (\#Linefeed (write-char #\Return out) (write-char #\Linefeed out))
>> (otherwise (write-char char out)))))
>
> This is very nice, indeed. However, it doesn't seem to handle the
> case where the string being converted already has a crlf -- then it'll
> end up as crcrlf with the above, right? How can we use this idea but
> eliminate the double cr?
You might consider reading this:
http://www.franz.com/support/documentation/8.0/doc/iacl.htm#composed-efs-3
and this:
http://www.franz.com/support/documentation/8.0/doc/operators/excl/crlf-base-ef.htm
You'd want to use it either directly on a stream, or using a call to
with-output-to-buffer:
http://www.franz.com/support/documentation/8.0/doc/operators/excl/with-output-to-buffer.htm
but since your original goal was to write the string out to a file, it
would be much simpler to do the job directly:
(with-open-file (o "myfile.txt" :direction :output
:external-format '(:e-crlf :latin1))
(dotimes (i 2)
(format o "This is line ~D.~%~%" (1+ i))))
Of course, you've limited your usage of CL implementations. But this
is how Allegro CL users can do it.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Dnia 16.07.2007 bufie <·····@spamneggs.com> napisa�/a:
> On Jul 16, 10:10 am, "John Thingstad" <··············@chello.no>
> wrote:
>> "bufie" <·····@spamneggs.com> skrev i ····································@j4g2000prf.googlegroups.com...
>>
>> > On Jul 15, 5:14 pm, "John Thingstad" <··············@chello.no> wrote:
>> >> > (by the way, the lisp implementation we're required to use is an old
>> >> > version of KCL, and doesn't have any options for changing the line
>> >> > endings).
>>
>> >> How about:
>>
>> >> (defun convert-linefeed (string)
>> >> (with-output-to-string (out)
>> >> (loop for char across string)
>> >> (case char
>> >> (#\Newline (write-char #\Return out) (write-char #\Linefeed
>> >> out))
>> >> (otherwise (write-char char out))))))
>>
>> A better way is to grab the iterate package http://common-lisp.net/project/iterate/
>>
>> The code is now:
>>
>> (defun convert-linefeed (text)
>> (with-output-to-string (out)
>> (iter (for char in-string text)
>> (case char
>> (\#Linefeed (write-char #\Return out) (write-char #\Linefeed out))
>> (otherwise (write-char char out)))))
>
> This is very nice, indeed. However, it doesn't seem to handle the
> case where the string being converted already has a crlf -- then it'll
> end up as crcrlf with the above, right? How can we use this idea but
> eliminate the double cr?
Perhaps something along these lines?
(defun convert-linefeed (text)
(with-output-to-string (out)
(iter (for char in-string text)
(for pchar previous char)
(when (and (eql char #\Linefeed) (not (eql pchar #\Return)))
(write-char #\Return out))
(write-char char out))))
--
Daniel 'Nathell' Janus, GG #1631668, ············@nathell.korpus.pl
create_initial_thread(initial_function);
lose("CATS. CATS ARE NICE.\n");
-- Steel Bank Common Lisp, sbcl/runtime/runtime.c:425
From: Thomas A. Russ
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <ymiwswznbyt.fsf@sevak.isi.edu>
bufie <·····@spamneggs.com> writes:
> I figured I could write a macro to send <cr><lf> instead of just <lf>,
> perhaps something like this:
>
> (defmacro format-crlf (mystream &rest args)
> (let (mystring newstring)
> (setq mystring (apply 'format nil args))
> (setq newstring (string-replace mystring
> (format nil "~A" #\Newline)
> (format nil "~A~A" #\Return #\Newline)))
> `(format ,mystream ,newstring)))
On a pedantic note, and to assure even more cross-platform
compatibility, you should really use #\Return #\Linefeed for the second
set of format strings.
That will help you if you ever run on something like a Mac, where the
newline character is <cr>, and it will also save you if you run this
code under windows where in some systems #\newline will translate to
<cr><lf>. In that case, you will end up with a no-op replacement, but
at least it will do the right thing.
--
Thomas A. Russ, USC/Information Sciences Institute
I was able to come up with a nice function (using the great
suggestions here) that will take a string and convert the LF to CRLF,
but I'm still unable to get the whole thing working the way I need it
to work.
I thought I would be able to use just the above function in place of
(format o ...), but when I pass in the stream variable, the function
throws an error since it doesn't know what the stream variable is. So
I think I need to use a macro in that location (which can, of course,
call a function).
Is the previous statement true, or can I pass the stream variable to a
function? i.e. if I have:
(with-open-file (o ....)
(my-output-func o "blah blah"))
How can I make that work?
Secondly, when I tried to use a macro to call my conversion function
(named lf2crlf), any variables were not getting replaced. i.e., if I
had:
(setq foo "test")
;; simplest case
(defmacro ms-format (dest fmtstr &rest args)
`(format ,dest (lf2crlf (format nil ,fmtstr ,@args))))
(with-open-file (o ....)
(ms-format o "foo = ~A" foo))
Then what gets written to the file is "foo = foo" rather than "foo =
test"
this also happens in the (what I thought was trivial) case where I
define:
(defmacro ms-format (dest fmtstr &rest args)
`(format ,dest ,fmtstr ,@args))
which I thought should just exactly do what a call to (format o ...)
would do, but doesn't expand the variables...
What's going wrong here? How can I get the variables expanded
properly?
Sorry to be clueless on this...
bufie
From: Thomas Russ
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <ymisl7dwi30.fsf@tubular.isi.edu>
bufie <·····@spamneggs.com> writes:
> I was able to come up with a nice function (using the great
> suggestions here) that will take a string and convert the LF to CRLF,
> but I'm still unable to get the whole thing working the way I need it
> to work.
>
> I thought I would be able to use just the above function in place of
> (format o ...), but when I pass in the stream variable, the function
> throws an error since it doesn't know what the stream variable is. So
> I think I need to use a macro in that location (which can, of course,
> call a function).
I'm not sure you really need to do this. Can you show the code that
is causing the problem?
> Is the previous statement true, or can I pass the stream variable to a
> function? i.e. if I have:
Sure you can. You can pass stream variables to functions.
> (with-open-file (o ....)
> (my-output-func o "blah blah"))
This should work just fine.
> How can I make that work?
>
> Secondly, when I tried to use a macro to call my conversion function
> (named lf2crlf), any variables were not getting replaced. i.e., if I
> had:
>
> (setq foo "test")
>
> ;; simplest case
> (defmacro ms-format (dest fmtstr &rest args)
> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args))))
This hardly seems like anything that needs a macro. It would appear
that one could just write a function that would do everything you
need:
(defun f-format (dest fmtstr &rest args)
(format dest (lf2crlf (apply #'format nil fmtstr args))))
> (with-open-file (o ....)
> (ms-format o "foo = ~A" foo))
>
> Then what gets written to the file is "foo = foo" rather than "foo =
> test"
>
> this also happens in the (what I thought was trivial) case where I
> define:
>
> (defmacro ms-format (dest fmtstr &rest args)
> `(format ,dest ,fmtstr ,@args))
Odd. Something else must be going wrong, since when I try this in my
Lisp, everything works as expected. Simple test case:
(let ((foo "test"))
(ms-format t "foo = ~A" foo))
foo = test
Which is exactly what one would expect. This also works:
(let ((stream *standard-output*)
(foo "test2"))
(ms-format stream "foo = ~A" foo))
foo = test2
> which I thought should just exactly do what a call to (format o ...)
> would do, but doesn't expand the variables...
>
> What's going wrong here? How can I get the variables expanded
> properly?
>
> Sorry to be clueless on this...
Don't know what is going wrong in your case. You could always try
using MACROEXPAND-1 on your code and see what gets produced:
(macroexpand-1 '(ms-format stream fmt arg1 arg2 arg3))
this often helps debug macros, but yours seem simple enough to not
require this.
Thomas Russ wrote:
> bufie <·····@spamneggs.com> writes:
>> ;; simplest case
>> (defmacro ms-format (dest fmtstr &rest args)
>> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args))))
>
> This hardly seems like anything that needs a macro. It would appear
> that one could just write a function that would do everything you
> need:
>
> (defun f-format (dest fmtstr &rest args)
> (format dest (lf2crlf (apply #'format nil fmtstr args))))
I was quite sure I had tried something like this before, and it threw an
error on the "dest" parameter, but it worked just great now, so that's good!
One of the reasons I wanted to use a macro instead of a function was so
that the compiler could optimize static strings at compile time, and
then the lf2crlf would only get called at run-time when necessary.
I had tried many variations, and couldn't get any of them to work.
However, I just tried this:
(defmacro ms-format (dest fmtstr &rest args)
(let (mystring newstring)
(if (= (length args) 0)
(progn
(setq mystring (apply 'format nil fmtstr args))
(setq newstring (lf2crlf mystring))
`(format ,dest ,newstring))
`(format ,dest (lf2crlf (format nil ,fmtstr ,@args))))))
which I believe can be condensed to:
(defmacro ms-format (dest fmtstr &rest args)
(if (= (length args) 0)
`(format ,dest ,(lf2crlf (format nil fmtstr args)))
`(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
which now seems to work -- I must have been doing something different
before...
Do you see any problem with the above macro? And does it seem like a
reasonable reason for using a macro? (the instances where we need the
dos-style line endings probably have about 20%-30% static text, so it
seemed appropriate).
Thanks for your input on this!
> (defmacro ms-format (dest fmtstr &rest args)
> (if (= (length args) 0)
> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
>
> which now seems to work -- I must have been doing something different
> before...
I spoke too quickly. this works when running interpreted, but doesn't
work when I compile it. The problem appears to be in the conversion
function, which is as follows:
(defun lf2crlf (string)
"Make sure all the linefeeds in string follow carriage returns."
(let ((s "") ; collect the lines with CRLF
(c0 0)) ; start of the current line
(do ((c1 (position #\Linefeed string)
(position #\Linefeed string :start (1+ c1))))
((null c1) (concatenate 'string s (subseq string c0)))
(if (= c0 c1)
; first char is lf -> not following cr
(setf s (concatenate
'string
s
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1))
; check for cr -- do nothing if its already there
(unless (eql #\Return (char string (1- c1)))
(setf s (concatenate
'string
s
(subseq string c0 c1)
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1)))))))
I inserted a bunch of debugging statements into the above code, and the
substitution doesn't appear to be taking place -- the length of the
returned string s is the same as the length of the initial string
"string" that is passed in, even though the relevant sections of code to
replace the lf with crlf *are* being called.
Any idea why this substitution doesn't work when compiled?
bufie <·····@spamneggs.com> writes:
>> (defmacro ms-format (dest fmtstr &rest args)
>> (if (= (length args) 0)
>> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
>> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
>>
>> which now seems to work -- I must have been doing something
>> different before...
>
> I spoke too quickly. this works when running interpreted, but doesn't
> work when I compile it. The problem appears to be in the conversion
> function, which is as follows:
>
> (defun lf2crlf (string)
> "Make sure all the linefeeds in string follow carriage returns."
> (let ((s "") ; collect the lines with CRLF
> (c0 0)) ; start of the current line
> (do ((c1 (position #\Linefeed string)
> (position #\Linefeed string :start (1+ c1))))
> ((null c1) (concatenate 'string s (subseq string c0)))
> (if (= c0 c1)
> ; first char is lf -> not following cr
> (setf s (concatenate
> 'string
> s
> #.(format nil "~A~A" #\Return #\Linefeed))
> c0 (1+ c1))
> ; check for cr -- do nothing if its already there
> (unless (eql #\Return (char string (1- c1)))
> (setf s (concatenate
> 'string
> s
> (subseq string c0 c1)
> #.(format nil "~A~A" #\Return #\Linefeed))
> c0 (1+ c1)))))))
Don't use (if test1
then
(unless test2
body))
or other such combinations of if when unless.
Use cond!
(cond
(test1 then)
((not test2) body))
Otherwhise, your function works correctly:
* (compile 'lf2crlf)
LF2CRLF
NIL
NIL
* (map 'vector 'char-code (lf2crlf (format nil "abc~Cdef~Cghi~C~Cklm" #\return #\linefeed #\return #\linefeed)))
#(97 98 99 13 100 101 102 13 10 103 104 105 13 10 107 108 109)
*
--
__Pascal Bourguignon__ http://www.informatimago.com/
CAUTION: The mass of this product contains the energy equivalent of
85 million tons of TNT per net ounce of weight.
Pascal Bourguignon wrote:
> bufie <·····@spamneggs.com> writes:
>
> Don't use (if test1
> then
> (unless test2
> body))
>
> Use cond!
>
> (cond
> (test1 then)
> ((not test2) body))
Thanks for pointing this out -- much cleaner! (I was borrowing code!)
I also found that the previous code worked if I pre-defined the crlf
combination and then used it in the concatenate statement instead of the
format line, and that all works (mostly!)
However, now the problem is that when I compile, the static strings are
not being inserted in the compiled code correctly, though all the
strings being built at run-time work just fine. Here is the latest code:
(defun lf2crlf (string)
"Make sure all the linefeeds in string follow carriage returns."
(let ((s "") ; collect the lines with CRLF
(c0 0) ; start of the current line
(crlf (coerce '(#\Return #\Linefeed) 'string )))
(do ((c1 (position #\Linefeed string)
(position #\Linefeed string :start (1+ c1))))
((null c1) (concatenate 'string s (subseq string c0)))
(cond ((= c0 c1)
;; first char is lf -> not following cr
(setf s (concatenate 'string s crlf)
c0 (1+ c1)))
;; check for cr -- do nothing if its already there
((not (eql #\Return (char string (1- c1))))
(setf s (concatenate 'string s (subseq string c0 c1) crlf)
c0 (1+ c1)))))))
(defmacro ms-format (dest fmtstr &rest args)
(if (= (length args) 0)
`(format ,dest ,(lf2crlf (format nil fmtstr args)))
`(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
If I put the debugging statements back in and watch the compilation, I
can see the static strings being adjusted, and their lengths increase by
the expected number of chars (i.e. the new CR char for each LF).
However, the strings seem to be "re-optimized" by the compiler to remove
the CR!
Has anyone heard of this type of behavior before?
If I can't get this figured out, it looks like the "static strings at
compile time" will need to go away, and I'll just need to build all of
them at run time...
From: Thomas Russ
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <ymimyxkw2kg.fsf@tubular.isi.edu>
bufie <·····@spamneggs.com> writes:
> However, now the problem is that when I compile, the static strings
> are not being inserted in the compiled code correctly, though all the
> strings being built at run-time work just fine. Here is the latest
> code:
[snip]
>
> If I put the debugging statements back in and watch the compilation, I
> can see the static strings being adjusted, and their lengths increase
> by the expected number of chars (i.e. the new CR char for each
> LF). However, the strings seem to be "re-optimized" by the compiler to
> remove the CR!
>
> Has anyone heard of this type of behavior before?
>
> If I can't get this figured out, it looks like the "static strings at
> compile time" will need to go away, and I'll just need to build all of
> them at run time...
This seems really odd, and potentially a bug in your lisp
implementation. I would be somewhat put-out if my lisp systems
started munging the line endings in the string constants in my source
code.
What lisp implementation are you using?
Just out of curiousity, does this also happen if you just put in
literal strings that have CRLF combinations in them?
Thomas Russ wrote:
> bufie <·····@spamneggs.com> writes:
>
>> However, now the problem is that when I compile, the static strings
>> are not being inserted in the compiled code correctly, though all the
>> strings being built at run-time work just fine. Here is the latest
>> code:
> [snip]
>> If I put the debugging statements back in and watch the compilation, I
>> can see the static strings being adjusted, and their lengths increase
>> by the expected number of chars (i.e. the new CR char for each
>> LF). However, the strings seem to be "re-optimized" by the compiler to
>> remove the CR!
>>
>> Has anyone heard of this type of behavior before?
>>
>> If I can't get this figured out, it looks like the "static strings at
>> compile time" will need to go away, and I'll just need to build all of
>> them at run time...
>
> This seems really odd, and potentially a bug in your lisp
> implementation. I would be somewhat put-out if my lisp systems
> started munging the line endings in the string constants in my source
> code.
>
> What lisp implementation are you using?
It's an embedded implementation based on an old version of Kyoto Common
Lisp.
> Just out of curiousity, does this also happen if you just put in
> literal strings that have CRLF combinations in them?
I actually had the same thought, and put in a couple of static strings
with crlf, and they came through just fine, so there seems to be
something else wrong with the implementation of the macro... I'll see
if there's any possibility of getting this fixed...
bufie <·····@spamneggs.com> writes:
> However, now the problem is that when I compile, the static strings
> are not being inserted in the compiled code correctly, though all the
> strings being built at run-time work just fine.
Perhaps you should use compiler macros, instead of simple macros.
Here is a small example:
------------(f.lisp)-----------------------
(defun f (str arg)
(format nil (string-upcase str) arg))
(define-compiler-macro f (&whole form str arg)
(if (stringp str)
`(format nil ,(string-upcase str) ,arg)
form))
(defun g (str)
(list
(f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
; called at compilation-time
(f str 2))) ; this won't be expanded and f will be called.
--------------------------------------------
C/USER[13]> (load (compile-file "/tmp/f.lisp"))
;; Compiling file /tmp/f.lisp ...
;; Wrote file /tmp/f.fas
0 errors, 0 warnings
;; Loading file /tmp/f.fas ...
;; Loaded file /tmp/f.fas
T
C/USER[14]> (disassemble 'g)
Disassembly of function G
(CONST 0) = "ABC "
(CONST 1) = 1
(CONST 2) = 2
(CONST 3) = F
1 required argument
0 optional arguments
No rest parameter
No keyword parameters
16 byte-code instructions:
0 (PUSH-UNBOUND 2)
2 (CALLS2&PUSH 118) ; MAKE-STRING-OUTPUT-STREAM
4 (CONST&PUSH 0) ; "ABC " <<== string already upcased
5 (LOAD&PUSH 1)
6 (PUSH-UNBOUND 2)
8 (CALLS1 139) ; WRITE-STRING <<== inlining of format
10 (CONST&PUSH 1) ; 1
11 (LOAD&PUSH 1)
12 (CALLS1 134) ; PRINC <<== inlining of format
14 (LOAD&PUSH 0)
15 (CALLS2&STORE 119 0) ; GET-OUTPUT-STREAM-STRING
18 (LOAD&PUSH 2)
19 (CONST&PUSH 2) ; 2
20 (CALL2&PUSH 3) ; F <<== call of (f str 2)
22 (LIST 2)
24 (SKIP&RET 2)
NIL
C/USER[15]>
--
__Pascal Bourguignon__ http://www.informatimago.com/
WARNING: This product attracts every other piece of matter in the
universe, including the products of other manufacturers, with a
force proportional to the product of the masses and inversely
proportional to the distance between them.
Pascal Bourguignon wrote:
> bufie <·····@spamneggs.com> writes:
>> However, now the problem is that when I compile, the static strings
>> are not being inserted in the compiled code correctly, though all the
>> strings being built at run-time work just fine.
>
> Perhaps you should use compiler macros, instead of simple macros.
>
> Here is a small example:
>
> ------------(f.lisp)-----------------------
> (defun f (str arg)
> (format nil (string-upcase str) arg))
>
> (define-compiler-macro f (&whole form str arg)
> (if (stringp str)
> `(format nil ,(string-upcase str) ,arg)
> form))
>
>
> (defun g (str)
> (list
> (f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
> ; called at compilation-time
> (f str 2))) ; this won't be expanded and f will be called.
>
> --------------------------------------------
OK, I've never tried compiler macros before, but I'll give it a shot...
So if I define a function ms-format as follows:
(defun ms-format (dest fmtstr &rest args)
(format dest (lf2crlf (apply #'format nil fmtstr args))))
and then a compiler macro:
(define-compiler-macro ms-format (&whole form dest fmtstr args)
(if (= (length args) 0)
`(format ,dest ,(lf2crlf (format nil fmtstr args)))
form))
the compiler throws up many errors on this, including that too few
arguments are supplied to the defmacro-lambda-list.
I thought that this would work (based on the examples I have seen). Any
ideas why it doesn't?
From: John Thingstad
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <op.tv05qcstpqzri1@pandora.upc.no>
P� Wed, 25 Jul 2007 18:36:51 +0200, skrev bufie <·····@spamneggs.com>:
> Pascal Bourguignon wrote:
>> bufie <·····@spamneggs.com> writes:
>>> However, now the problem is that when I compile, the static strings
>>> are not being inserted in the compiled code correctly, though all the
>>> strings being built at run-time work just fine.
>> Perhaps you should use compiler macros, instead of simple macros.
>> Here is a small example:
>> ------------(f.lisp)-----------------------
>> (defun f (str arg)
>> (format nil (string-upcase str) arg))
>> (define-compiler-macro f (&whole form str arg)
>> (if (stringp str)
>> `(format nil ,(string-upcase str) ,arg)
>> form))
>> (defun g (str)
>> (list
>> (f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
>> ; called at compilation-time
>> (f str 2))) ; this won't be expanded and f will be called.
>> --------------------------------------------
>
> OK, I've never tried compiler macros before, but I'll give it a shot...
>
> So if I define a function ms-format as follows:
>
> (defun ms-format (dest fmtstr &rest args)
> (format dest (lf2crlf (apply #'format nil fmtstr args))))
>
> and then a compiler macro:
>
> (define-compiler-macro ms-format (&whole form dest fmtstr args)
> (if (= (length args) 0)
> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
> form))
>
> the compiler throws up many errors on this, including that too few
> arguments are supplied to the defmacro-lambda-list.
>
> I thought that this would work (based on the examples I have seen). Any
> ideas why it doesn't?
Here is a example define-compiler-macro. A printf function for lisp a
while back
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/9bfc9c9f9a789dea/d5dcc0d9137f95bb?lnk=gst&q=printf+group%3Acomp.lang.lisp+author%3ATilton&rnum=1&hl=en#d5dcc0d9137f95bb
--
Sendt med Operas revolusjonerende e-postprogram: http://www.opera.com/mail/
John Thingstad wrote:
> På Wed, 25 Jul 2007 18:36:51 +0200, skrev bufie <·····@spamneggs.com>:
>
>> Pascal Bourguignon wrote:
>>> bufie <·····@spamneggs.com> writes:
>>>> However, now the problem is that when I compile, the static strings
>>>> are not being inserted in the compiled code correctly, though all the
>>>> strings being built at run-time work just fine.
>>> Perhaps you should use compiler macros, instead of simple macros.
>>> Here is a small example:
>>> ------------(f.lisp)-----------------------
>>> (defun f (str arg)
>>> (format nil (string-upcase str) arg))
>>> (define-compiler-macro f (&whole form str arg)
>>> (if (stringp str)
>>> `(format nil ,(string-upcase str) ,arg)
>>> form))
>>> (defun g (str)
>>> (list
>>> (f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
>>> ; called at compilation-time
>>> (f str 2))) ; this won't be expanded and f will be called.
>>> --------------------------------------------
>>
>> OK, I've never tried compiler macros before, but I'll give it a shot...
>>
>> So if I define a function ms-format as follows:
>>
>> (defun ms-format (dest fmtstr &rest args)
>> (format dest (lf2crlf (apply #'format nil fmtstr args))))
>>
>> and then a compiler macro:
>>
>> (define-compiler-macro ms-format (&whole form dest fmtstr args)
>> (if (= (length args) 0)
>> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
>> form))
>>
>> the compiler throws up many errors on this, including that too few
>> arguments are supplied to the defmacro-lambda-list.
>>
>> I thought that this would work (based on the examples I have seen).
>> Any ideas why it doesn't?
>
> Here is a example define-compiler-macro. A printf function for lisp a
> while back
>
> http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/9bfc9c9f9a789dea/d5dcc0d9137f95bb?lnk=gst&q=printf+group%3Acomp.lang.lisp+author%3ATilton&rnum=1&hl=en#d5dcc0d9137f95bb
Thanks, John. Unfortunately, even after skimming through those
messages, I still don't see the problem in my construct (which it seems
should be almost trivial)...
It shouldn't be causing this much trouble! :)
thanks in advance for any additional guidance and suggestions!
From: John Thingstad
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <op.tv06rzw4pqzri1@pandora.upc.no>
P� Wed, 25 Jul 2007 19:17:48 +0200, skrev bufie <·····@spamneggs.com>:
>
> Thanks, John. Unfortunately, even after skimming through those
> messages, I still don't see the problem in my construct (which it seems
> should be almost trivial)...
>
> It shouldn't be causing this much trouble! :)
>
> thanks in advance for any additional guidance and suggestions!
(define-compiler-macro printf->format (&whole whole format)
(if (stringp format)
(printf->format format)
whole))
The point is if the string is known at compile time call the function then
otherwise call it at runtime.
I thought the whole trick might be of some help..
John Thingstad wrote:
> På Wed, 25 Jul 2007 19:17:48 +0200, skrev bufie <·····@spamneggs.com>:
>
>> Thanks, John. Unfortunately, even after skimming through those
>> messages, I still don't see the problem in my construct (which it
>> seems should be almost trivial)...
>>
>> It shouldn't be causing this much trouble! :)
>>
>> thanks in advance for any additional guidance and suggestions!
>
> (define-compiler-macro printf->format (&whole whole format)
> (if (stringp format)
> (printf->format format)
> whole))
>
> The point is if the string is known at compile time call the function
> then otherwise call it at runtime.
> I thought the whole trick might be of some help..
Right -- that's what I thought I was doing with checking to see if tere
were any args passed to the ms-format macro. If not, then it's a static
string, so expand it at compile time. Otherwise expand it at runtime.
Unfortunately, it seems not to be working as I expected it to...
bufie <·····@spamneggs.com> writes:
> Pascal Bourguignon wrote:
>> bufie <·····@spamneggs.com> writes:
>>> However, now the problem is that when I compile, the static strings
>>> are not being inserted in the compiled code correctly, though all the
>>> strings being built at run-time work just fine.
>>
>> Perhaps you should use compiler macros, instead of simple macros.
>>
>> Here is a small example:
>>
>> ------------(f.lisp)-----------------------
>> (defun f (str arg)
>> (format nil (string-upcase str) arg))
>>
>> (define-compiler-macro f (&whole form str arg)
>> (if (stringp str)
>> `(format nil ,(string-upcase str) ,arg)
>> form))
>>
>>
>> (defun g (str)
>> (list
>> (f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
>> ; called at compilation-time
>> (f str 2))) ; this won't be expanded and f will be called.
>>
>> --------------------------------------------
>
> OK, I've never tried compiler macros before, but I'll give it a shot...
>
> So if I define a function ms-format as follows:
>
> (defun ms-format (dest fmtstr &rest args)
> (format dest (lf2crlf (apply #'format nil fmtstr args))))
>
> and then a compiler macro:
>
> (define-compiler-macro ms-format (&whole form dest fmtstr args)
> (if (= (length args) 0)
> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
> form))
>
> the compiler throws up many errors on this, including that too few
> arguments are supplied to the defmacro-lambda-list.
>
> I thought that this would work (based on the examples I have seen).
> Any ideas why it doesn't?
Well you may get "too few arguments" on function calls, but since you
don't show us how you use it, what actual function calls and error
message you have, we can hardly help here.
You should probably keep the argument list between the defun and
define-compiler-macro "congruents":
(define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
Instead of writing (= (length args) 0)
you could write: (zerop (length args))
And instead of writing (zerop (length args))
you should rather write: (null args)
(if (null args)
If there is no args then there is no need to pass them to format. (It
could still be usefull to pass an empty list of arguments to apply,
but since you removed apply, you should remove the last empty argument
when it's empty.
(lf2crlf (format nil fmtstr))
Now, what happens if you try to format "~~A"?
(format nil fmtstr) will return "~A".
lf2crlf will leave it as is and return "~A",
and (format ,dest "~A") will break because it lacks an argument.
So write:
`(format ,dest "~A" ,(lf2crlf (format nil fmtstr)))
This was ok:
form)))
(define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
(if (null args)
`(format ,dest "~A" ,(lf2crlf (format nil fmtstr)))
form)))
Now, you could also check if DEST is T or NIL, or even (QUOTE T) or
(QUOTE NIL) and use PRINC or nothing instead of (format ,dest "~A" ...):
(define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
(when (and (listp fmtstr)
(eq 'quote (first fmtstr))
(stringp (second fmtstr))
(null (cddr fmtstr)))
(setf fmtstr (second fmtstr)))
(cond
((or args (not (stringp fmtstr)))
form)
((member dest '(nil (quote nil)) :test (function equal))
;; return a string
(lf2crlf (format nil fmtstr)))
(t
;; use princ
`(princ ,(lf2crlf (format nil fmtstr))
,(if (member dest '(t (quote t)) :test (function equal))
'*standard-output*
dest)))))
Remember that compiler macros are executed at compilation time,
therefore the values bound to the arguments are the actual forms
written in the function call.
(ms-format nil "abc") ; should call only LF2CRLF at compilation time.
(ms-format 'nil '"abc") ; "
(ms-format t "abc") ; should call LF2CRLF at C-time and PRINC at R-time.
(ms-format 't '"abc") ; "
(ms-format (eql x x) "abc") ; "
(ms-format (eql x x) (string-downcase "ABC")) ; "
--
__Pascal Bourguignon__ http://www.informatimago.com/
"You can tell the Lisp programmers. They have pockets full of punch
cards with close parentheses on them." --> http://tinyurl.com/8ubpf
Pascal Bourguignon wrote:
> bufie <·····@spamneggs.com> writes:
>
>> Pascal Bourguignon wrote:
>>> bufie <·····@spamneggs.com> writes:
>>>> However, now the problem is that when I compile, the static strings
>>>> are not being inserted in the compiled code correctly, though all the
>>>> strings being built at run-time work just fine.
>>> Perhaps you should use compiler macros, instead of simple macros.
>>>
>>> Here is a small example:
>>>
>>> ------------(f.lisp)-----------------------
>>> (defun f (str arg)
>>> (format nil (string-upcase str) arg))
>>>
>>> (define-compiler-macro f (&whole form str arg)
>>> (if (stringp str)
>>> `(format nil ,(string-upcase str) ,arg)
>>> form))
>>>
>>>
>>> (defun g (str)
>>> (list
>>> (f "abc ~A" 1) ; this will get expanded, with STRING-UPCASE
>>> ; called at compilation-time
>>> (f str 2))) ; this won't be expanded and f will be called.
>>>
>>> --------------------------------------------
>> OK, I've never tried compiler macros before, but I'll give it a shot...
>>
>> So if I define a function ms-format as follows:
>>
>> (defun ms-format (dest fmtstr &rest args)
>> (format dest (lf2crlf (apply #'format nil fmtstr args))))
>>
>> and then a compiler macro:
>>
>> (define-compiler-macro ms-format (&whole form dest fmtstr args)
>> (if (= (length args) 0)
>> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
>> form))
>>
>> the compiler throws up many errors on this, including that too few
>> arguments are supplied to the defmacro-lambda-list.
>>
>> I thought that this would work (based on the examples I have seen).
>> Any ideas why it doesn't?
>
> Well you may get "too few arguments" on function calls, but since you
> don't show us how you use it, what actual function calls and error
> message you have, we can hardly help here.
Actually, I thought I had shown how I was using it as part of a
(with-open-file (o ....)
(ms-format o "my static text")
(ms-format o "my dynamic text ~A" myvar))
with ms-format defined as either a macro or a function... when I define
it as a compiler macro as above, that's when I get the "too few
arguments" message.
> You should probably keep the argument list between the defun and
> define-compiler-macro "congruents":
>
> (define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
Good catch -- I thought I had, but missed &rest. However, even with
that, the simple case still generated the same errors.
> Instead of writing (= (length args) 0)
> you could write: (zerop (length args))
>
> And instead of writing (zerop (length args))
> you should rather write: (null args)
>
> (if (null args)
Thanks, I had forgotten about (null ).
> If there is no args then there is no need to pass them to format. (It
> could still be usefull to pass an empty list of arguments to apply,
> but since you removed apply, you should remove the last empty argument
> when it's empty.
>
> (lf2crlf (format nil fmtstr))
>
> Now, what happens if you try to format "~~A"?
> (format nil fmtstr) will return "~A".
> lf2crlf will leave it as is and return "~A",
> and (format ,dest "~A") will break because it lacks an argument.
> So write:
>
> `(format ,dest "~A" ,(lf2crlf (format nil fmtstr)))
>
> This was ok:
>
> form)))
>
> (define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
> (if (null args)
> `(format ,dest "~A" ,(lf2crlf (format nil fmtstr)))
> form)))
Even with this simple compiler macro, the compilation process throws an
error with the message "Lisp Error: Frame Stack Overflow"
[ other helpful example info deleted ]
This is just very strange -- the fact that the previous simple macro
code seemed to work except that the compiled code didn't keep the
expansion on the static strings is very odd. I tried defining some
static strings with crlf in them, and they remained intact through
compilation, so the compiler isn't automatically converting them (which
it shouldn't, of course, so that's good).
I'm thinking that it might be better just to go with ms-format defined
as a function rather than a macro or compiler macro, but now it's
becoming more of a personal challenge to get this resolved!
Any other thoughts?
bufie <·····@spamneggs.com> writes:
> [...]
> Even with this simple compiler macro, the compilation process throws
> an error with the message "Lisp Error: Frame Stack Overflow"
> [...]
> Any other thoughts?
Well, it works here both with clisp and sbcl...
There is no recursive call, so the stack overflow may be due to too
little memory or a bug in that compiler.
C/USER[37]> (load (compile-file "/tmp/m.lisp"))
;; Compiling file /tmp/m.lisp ...
;; Wrote file /tmp/m.fas
0 errors, 0 warnings
;; Loading file /tmp/m.fas ...
;; Loaded file /tmp/m.fas
T
C/USER[38]> (cat "/tmp/m.lisp")
(defun lf2crlf (string)
"Make sure all the linefeeds in string follow carriage returns."
(let ((s "") ; collect the lines with CRLF
(c0 0)) ; start of the current line
(do ((c1 (position #\Linefeed string)
(position #\Linefeed string :start (1+ c1))))
((null c1) (concatenate 'string s (subseq string c0)))
(if (= c0 c1)
; first char is lf -> not following cr
(setf s (concatenate
'string
s
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1))
; check for cr -- do nothing if its already there
(unless (eql #\Return (char string (1- c1)))
(setf s (concatenate
'string
s
(subseq string c0 c1)
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1)))))))
(defun ms-format (dest fmtstr &rest args)
(format dest (lf2crlf (apply #'format nil fmtstr args))))
(define-compiler-macro ms-format (&whole form dest fmtstr &rest args)
(when (and (listp fmtstr)
(eq 'quote (first fmtstr))
(stringp (second fmtstr))
(null (cddr fmtstr)))
(setf fmtstr (second fmtstr)))
(cond
((or args (not (stringp fmtstr)))
form)
((member dest '(nil (quote nil)) :test (function equal))
;; return a string
(lf2crlf (format nil fmtstr)))
(t
;; use princ
`(princ ,(lf2crlf (format nil fmtstr))
,(if (member dest '(t (quote t)) :test (function equal))
'*standard-output*
dest)))))
(with-open-file (o "/tmp/o" :direction :output :if-does-not-exist :create :if-exists :supersede)
(ms-format o "my static text")
(ms-format o "my dynamic text ~A" 42))
C/USER[39]>
--
__Pascal Bourguignon__ http://www.informatimago.com/
IMPORTANT NOTICE TO PURCHASERS: The entire physical universe,
including this product, may one day collapse back into an
infinitesimally small space. Should another universe subsequently
re-emerge, the existence of this product in that universe cannot be
guaranteed.
> (defmacro ms-format (dest fmtstr &rest args)
> (if (= (length args) 0)
> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
>
> which now seems to work -- I must have been doing something different
> before...
I spoke too quickly. this works when running interpreted, but doesn't
work when I compile it. The problem appears to be in the conversion
function, which is as follows:
(defun lf2crlf (string)
"Make sure all the linefeeds in string follow carriage returns."
(let ((s "") ; collect the lines with CRLF
(c0 0)) ; start of the current line
(do ((c1 (position #\Linefeed string)
(position #\Linefeed string :start (1+ c1))))
((null c1) (concatenate 'string s (subseq string c0)))
(if (= c0 c1)
; first char is lf -> not following cr
(setf s (concatenate
'string
s
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1))
; check for cr -- do nothing if its already there
(unless (eql #\Return (char string (1- c1)))
(setf s (concatenate
'string
s
(subseq string c0 c1)
#.(format nil "~A~A" #\Return #\Linefeed))
c0 (1+ c1)))))))
I inserted a bunch of debugging statements into the above code, and the
substitution doesn't appear to be taking place -- the length of the
returned string s is the same as the length of the initial string
"string" that is passed in, even though the relevant sections of code to
replace the lf with crlf *are* being called.
Any idea why this substitution doesn't work when compiled?
From: Thomas Russ
Subject: Re: macro equivalent to "format" but with changed line endings
Date:
Message-ID: <ymihcnsw2eb.fsf@tubular.isi.edu>
bufie <·····@spamneggs.com> writes:
> Thomas Russ wrote:
>
> One of the reasons I wanted to use a macro instead of a function was
> so that the compiler could optimize static strings at compile time,
> and then the lf2crlf would only get called at run-time when necessary.
>
...
> (defmacro ms-format (dest fmtstr &rest args)
> (if (= (length args) 0)
> `(format ,dest ,(lf2crlf (format nil fmtstr args)))
> `(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
>
> which now seems to work -- I must have been doing something different
> before...
>
> Do you see any problem with the above macro? And does it seem like a
> reasonable reason for using a macro? (the instances where we need the
> dos-style line endings probably have about 20%-30% static text, so it
> seemed appropriate).
This could be reasonable. It really depends on how much run-time is
actually spent on this operation, in order to figure out if it is
really needed.
By the way, if you can impose some additional restrictions on the
values of DEST, then you may be able to optimize it a bit more. As
long as DEST is a real stream and not one of the special format items
like NIL or T, then you could write the string using WRITE-STRING,
which should be faster than format for a constant string.
Thomas Russ wrote:
> By the way, if you can impose some additional restrictions on the
> values of DEST, then you may be able to optimize it a bit more. As
> long as DEST is a real stream and not one of the special format items
> like NIL or T, then you could write the string using WRITE-STRING,
> which should be faster than format for a constant string.
This sounded like a very good possibility to try to solve the macro
problem, so I implemented another function to convert any "~%" to crlf,
and changed the original macro to:
(defmacro ms-format (dest fmtstr &rest args)
(if (null args)
`(write-string ,(tildepct2crlf fmtstr) ,dest )
`(format ,dest (lf2crlf (format nil ,fmtstr ,@args)))))
I added debugging calls as before, and could see the static strings
being created during the compilation (as before), but the resulting
output was still just the lf as I'd seen both with straight (format o
"....~%") as well as with the other macro/function.
So something strange is definitely going on.
For the time being, I suppose using ms-format as a function rather than
a macro is the best approach, but using write-string instead to reduce
overhead...
Strange, indeed....