Removing Smart Quotes in PDF files
(code also available at http://douglasedmunds.com/z_lisp)
I took an interest in learning lisp and found
several lisp books available on the web
in PDF format. I became very frustrated because
the code samples in many of them are set as
'smart quotes' instead of the apostrophes
and backquotes required by the lisp listener.
Changing them by hand is tedious and error-prone.
Another alternative is to copy the code
into a text editor, then run
search and replace twice, once for the left quotes
and again for the right quotes, and then
paste the finished product into lisp. Equally
tedious and error-prone.
To help me with this little problem,
I wrote a little desktop GUI program
that changes smart right quotes ( ' ) to apostrophes ( ' )
and smart left quotes ( ' ) to backquotes ( ` )
using just the clipboard.
With this tool, I just do this instead:
Copy (CTRL-C) from PDF
Click GUI-BUTTON
Paste (CTRL-V) into LISP
You are welcome to the program.
You'll need to use REBOL/view to run the code.
It's free on the rebol.com website.
There are Win/MAC/Linux versions.
Note: I have only tested this code on a
XP-based PC with REBOL/view v.1.3.2
-- Doug Edmunds
-- May 4, 2006
Code for the GUI program:
; start of code for fix-quotes.r
REBOL [
Title: "Replace Smart Quotes"
Date: 4-May-2006
Author: "Doug Edmunds"
Version: 1.1.0
;replaces smart right quotes with apostrophe
;replaces smart left quotes with backquote (under tilde)
; so '(a b c) is equal to '(a b c).
; becomes `(a b c) is equal to '(a b c).
]
; set some variables
findval_1: "'"
replaceval_1: "'"
findval_2:"'"
replaceval_2: "`"
; get clipboard contents when the gui opens
clipin: read/lines clipboard://
; store a copy
clipout: copy clipin
; modify clipout with the changes
replace/all clipout findval_1 replaceval_1
replace/all clipout findval_2 replaceval_2
; create a GUI named "out"
out: layout [
across
h2 "Replace smart quotes in clipboard" return
h3 "Change ' to ' and ' to `" return
button 96 "Get" [
clipin: read/lines clipboard://
f1/text: form clipin
clipout: copy clipin
replace/all clipout findval_1 replaceval_1
replace/all clipout findval_2 replaceval_2
show out
]
button 96 "Change" [
;change the clipboard contents
write clipboard:// clipout
clipin: read/lines clipboard://
f1/text: form clipin
show out
]
button "Get and Change" [
clipin: read/lines clipboard://
clipout: copy clipin
replace/all clipout findval_1 replaceval_1
replace/all clipout findval_2 replaceval_2
write clipboard:// clipout
clipin: read/lines clipboard://
f1/text: form clipin
show out
] return
button "Biggest" [
f1/font: biggest-font-f1
show out
]
button "Bigger" [
f1/font: bigger-font-f1
show out
]
button "Normal" [
f1/font: normal-font-f1
show out
]
button 96 "Exit" [ quit ] return
; fill the field with contents of clipin
f1: area wrap clipin return
]
;; font-size procedures for biggest/bigger/normal buttons
normal-font-f1: make f1/font []
bigger-font-f1: make f1/font [size: 16]
biggest-font-f1: make f1/font [size: 20]
;; show the GUI
view out
; end code for fix-quotes.r
;; here's code for a mini smart-quotes changer that does the same thing
;; mini-fix-quotes.r
ReBOL [
Title: "Mini Changer"
Date: 4-May-2006
Author: "Doug Edmunds"
Version: 1.1.0
;replaces smart right quotes with apostrophe
;replaces smart left quotes with backquote (under tilde)
; '(a b c) is equal to '(a b c).
; becomes
; `(a b c) is equal to '(a b c).
;
; copy text to clipboard, click button, paste from clipboard
]
clipin: read/lines clipboard://
findval_1: "'"
replaceval_1: "'"
findval_2:"'"
replaceval_2: "`"
;findval_2:"e"
;replaceval_2: "-just-testing-"
field_last: copy clipin
replace/all field_last findval_1 replaceval_1
replace/all field_last findval_2 replaceval_2
out: layout [
across
;; text "S-Q Changer" return
button "Change Clip" [
clipin: read/lines clipboard://
field_last: copy clipin
replace/all field_last findval_1 replaceval_1
replace/all field_last findval_2 replaceval_2
write clipboard:// field_last
show out
] return
button 96 "Exit" [ quit ]
return
; text "Change clipboard contents: " return
; text "smart right quotes to apostrophes," return
; text "smart left quotes to backquotes." return
]
view out
"NicknameOptional" <··········@yahoo.com> writes:
> [...]
> To help me with this little problem,
> I wrote a little desktop GUI program
> that changes smart right quotes ( ' ) to apostrophes ( ' )
> and smart left quotes ( ' ) to backquotes ( ` )
> using just the clipboard.
>
> [85 LOC of REBOL]
This is crazy!
The lisp way is:
[331]> (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK
(get-macro-character #\GRAVE_ACCENT))
T
[332]> (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK
(get-macro-character #\APOSTROPHE))
T
[333]> (coerce #(#\LEFT_SINGLE_QUOTATION_MARK #\RIGHT_SINGLE_QUOTATION_MARK) 'string)
"‘’"
[334]> ‘( ,(+ 1 2) ’(+ 1 2))
(3 '(+ 1 2))
[335]>
Then you can just copy-and-paste the code with the special characters...
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
Lispworks:
Error while reading: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
Error while reading: Wrong character name: GRAVE_ACCENT.
CL-USER 1 > (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK
(get-macro-character #\GRAVE_ACCENT))
Error: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
1 (abort) Return to level 0.
2 Return to top loop level 0.
Type :b for backtrace, :c <option number> to proceed, or :? for other
options
Error while reading: Wrong character name: RIGHT_SINGLE_QUOTATION_MARK.
CL-USER 1 : 1 > (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK
(get-macro-character #\APOSTROPHE))
Error: Wrong character name: RIGHT_SINGLE_QUOTATION_MARK.
1 (abort) Return to level 1.
2 Return to debug level 1.
3 Return to level 0.
4 Return to top loop level 0.
Type :b for backtrace, :c <option number> to proceed, or :? for other
options
Error while reading: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
Error while reading: Wrong character name: RIGHT_SINGLE_QUOTATION_MARK.
CL-USER 1 : 2 > (coerce #(#\LEFT_SINGLE_QUOTATION_MARK
#\RIGHT_SINGLE_QUOTATION_MARK) 'string)
Error: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
1 (abort) Return to level 2.
2 Return to debug level 2.
3 Return to level 1.
4 Return to debug level 1.
5 Return to level 0.
6 Return to top loop level 0.
Type :b for backtrace, :c <option number> to proceed, or :? for other
options
CL-USER 1 : 3 >
"NicknameOptional" wrote:
> Lispworks:
>
> Error while reading: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
> Error while reading: Wrong character name: GRAVE_ACCENT.
[etc]
Well, if it comes to that, your REBOL code doesn't work too well
in LispWorks either.
--
Gareth McCaughan
.sig under construc
"NicknameOptional" <··········@yahoo.com> writes:
> Lispworks:
>
> Error while reading: Wrong character name: LEFT_SINGLE_QUOTATION_MARK.
> Error while reading: Wrong character name: GRAVE_ACCENT.
I'm sorry to read that. (Let's add standarizing character names to
the agenda of the next round of Common Lisp standardization. After
all, Unicode is becoming stable).
If your implementation still supports Unicode, you can use directly
the characters themselve. I used the name only to avoid any confusion
in the news readers.
(set-macro-character #\‘ (get-macro-character #\`_ACCENT))
(set-macro-character #\’ (get-macro-character #\'))
‘( ,(+ 1 2) ’(+ 1 2))
If your implementation doesn't support Unicode, I'd advise to change
the implementation, but YMMV.
--
__Pascal Bourguignon__ http://www.informatimago.com/
"Do not adjust your mind, there is a fault in reality"
-- on a wall many years ago in Oxford.
Allegro CL 7.0 :
International Allegro CL Trial Edition
7.0 [Windows] (Apr 9, 2006 10:40)
Copyright (C) 1985-2004, Franz Inc., Oakland, CA, USA. All Rights
Reserved.
This development copy of Allegro CL is licensed to:
Trial User
CG version 1.54.2.17.2.12 / IDE version 1.48.2.20.2.16
;; Optimization settings: safety 1, space 1, speed 1, debug 2.
;; For a complete description of all compiler switches given the
current optimization settings evaluate
;; (EXPLAIN-COMPILER-SETTINGS).
[changing package from "COMMON-LISP-USER" to "COMMON-GRAPHICS-USER"]
CG-USER(1): (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK
(get-macro-character #\GRAVE_ACCENT))
T
CG-USER(2): (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK
(get-macro-character #\APOSTROPHE))
T
CG-USER(3): (coerce #(#\LEFT_SINGLE_QUOTATION_MARK
#\RIGHT_SINGLE_QUOTATION_MARK) 'string)
"''"
CG-USER(4):
CG-USER(4): (mapcar #'+
'(1 2 3)
'(10 100 1000))
Error: No dispatch function defined for #\right_single_quotation_mark.
[condition type: SIMPLE-ERROR]
CG-USER(5):
From: Bill Atkins
Subject: Re: Removing Smart Quotes in PDF files
Date:
Message-ID: <87ac9wc43g.fsf@rpi.edu>
"NicknameOptional" <··········@yahoo.com> writes:
> Allegro CL 7.0 :
>
> International Allegro CL Trial Edition
> 7.0 [Windows] (Apr 9, 2006 10:40)
> Copyright (C) 1985-2004, Franz Inc., Oakland, CA, USA. All Rights
> Reserved.
>
> This development copy of Allegro CL is licensed to:
> Trial User
>
> CG version 1.54.2.17.2.12 / IDE version 1.48.2.20.2.16
> ;; Optimization settings: safety 1, space 1, speed 1, debug 2.
> ;; For a complete description of all compiler switches given the
> current optimization settings evaluate
> ;; (EXPLAIN-COMPILER-SETTINGS).
>
> [changing package from "COMMON-LISP-USER" to "COMMON-GRAPHICS-USER"]
> CG-USER(1): (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK
> (get-macro-character #\GRAVE_ACCENT))
> T
> CG-USER(2): (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK
> (get-macro-character #\APOSTROPHE))
> T
> CG-USER(3): (coerce #(#\LEFT_SINGLE_QUOTATION_MARK
> #\RIGHT_SINGLE_QUOTATION_MARK) 'string)
>
> "''"
> CG-USER(4):
> CG-USER(4): (mapcar #'+
> '(1 2 3)
> '(10 100 1000))
> Error: No dispatch function defined for #\right_single_quotation_mark.
> [condition type: SIMPLE-ERROR]
> CG-USER(5):
>
Are you going to go through every implementation available? Why not
just say "that didn't work"?
Pascal's example was CLISP-specific, but the overall idea is not.
--
This is a song that took me ten years to live and two years to write.
- Bob Dylan
> Why not just say "that didn't work"?
I started out saying "I took an interest in learning lisp".
Pascal provided code. I tried it on the two versions of
common isp which I have on this machine and it did not work
for either. I notice that Pascal's listener was up to line
[331], leaving open the possibility that there was a
line missing that he would spot and add, or that
someone else could provide it. The concept of
"No dispatch function defined" is not exactly lisp 101.
The code as written in the PDF files is *wrong*, so my original
idea was to make it *right* before it goes into lisp.
One problem with doing this transformation inside lisp, is that
effectively it allows for more wrong coding to be arbitrarily added
to source code: (in *all* cases, replace x1 with y1, replace x2
with y2). Wrong source code can be propagated by further
copy-pasting the examples into other source code.
"NicknameOptional" <··········@yahoo.com> writes:
>> Why not just say "that didn't work"?
>
> I started out saying "I took an interest in learning lisp".
> Pascal provided code. I tried it on the two versions of
> common isp which I have on this machine and it did not work
> for either. I notice that Pascal's listener was up to line
> [331], leaving open the possibility that there was a
> line missing that he would spot and add, or that
> someone else could provide it.
Not, it's just that I keep running the same REPL as long as I can :-)
[344]> (uptime)
uptime: 1 day, 1 hour, 55 minutes, 53 seconds.
93353
[345]>
I put this in my startup file:
(defparameter *start-time* (get-universal-time))
(defun uptime ()
(let ((uptime (- (get-universal-time) *start-time*)))
(multiple-value-bind (se mi ho da mo ye) (decode-universal-time uptime 0)
(decf ye 1900)
(decf mo 1)
(decf da 1)
(format t "~&uptime: ~:[~D year~:*~P, ~;~*~]~
~:[~D month~:*~P, ~;~*~]~
~:[~D day~:*~P, ~;~*~]~
~:[~D hour~:*~P, ~;~*~]~
~:[~D minute~:*~P, ~;~*~]~
~:[~D second~:*~P.~;~*~]~%"
(zerop ye) ye (zerop mo) mo (zerop da) da
(zerop ho) ho nil mi nil se))
uptime))
> The concept of "No dispatch function defined" is not exactly lisp 101.
Yes, I don't usually use #', so I overlooked it, but Allegro doesn't
seem to be able to cope use setting the dispatch macro character either:
I would have expected the three set-dispatch-macro-characters forms
below to work...
$ telnet prompt.franz.com
Trying 206.169.106.7...
Connected to prompt.franz.com.
Escape character is '^]'.
WARNING: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
WARNING: Allegro CL will expire in 0 days.
WARNING: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
International Allegro CL Trial Edition
6.2 [Linux (x86)] (May 21, 2003 12:21)
Copyright (C) 1985-2002, Franz Inc., Berkeley, CA, USA. All Rights Reserved.
This development copy of Allegro CL is licensed to:
Trial license for prompt.franz.com, DO NOT DELETE THIS LICENSE!
;; Optimization settings: safety 1, space 1, speed 1, debug 2.
;; For a complete description of all compiler switches given the
;; current optimization settings evaluate (EXPLAIN-COMPILER-SETTINGS).
CL-USER(1): (setf (stream-external-format *standard-output*) :utf-8)
(setf (stream-external-format *standard-output*) :utf-8)
:UTF-8
CL-USER(2): (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK (get-macro-character #\GRAVE_ACCENT))
(set-macro-character #\LEFT_SINGLE_QUOTATION_MARK (get-macro-character #\GRAVE_ACCENT))
T
CL-USER(3): (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK (get-macro-character #\APOSTROPHE))
(set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK (get-macro-character #\APOSTROPHE))
T
CL-USER(4): *standard-input*
*standard-input*
#<TERMINAL-SIMPLE-STREAM [initial terminal io] fd 0/1 @ #x710acc8a>
CL-USER(5): *standard-output*
*standard-output*
#<TERMINAL-SIMPLE-STREAM [initial terminal io] fd 0/1 @ #x710acc8a>
CL-USER(6): (stream-external-format *standard-input*)
(stream-external-format *standard-input*)
:UTF-8
CL-USER(7): (stream-external-format *standard-output*)
(stream-external-format *standard-output*)
:UTF-8
CL-USER(8): (set-dispatch-macro-character #\# #\right_single_quotation_mark (get-dispatch-macro-character #\# #\'))
(set-dispatch-macro-character #\# #\right_single_quotation_mark (get-dispatch-macro-character #\# #\'))
T
CL-USER(9): (mapcar #’+ ’(1 2 3) ’(10 100 1000))
(mapcar #’+ ’(1 2 3) ’(10 100 1000))
Error: Illegal function object: (8217 . #<Function SHARP-QUOTE>).
[condition type: PROGRAM-ERROR]
Restart actions (select using :continue):
0: Abort entirely from this process.
[1] CL-USER(10): :res
:res
CL-USER(11): (set-dispatch-macro-character #\# #\right_single_quotation_mark (lambda (stream dieze quote) `(function ,(read stream t nil t))))
(set-dispatch-macro-character #\# #\right_single_quotation_mark (lambda (stream dieze quote) `(function ,(read stream t nil t))))
T
CL-USER(12): (mapcar #’+ ’(1 2 3) ’(10 100 1000))
(mapcar #’+ ’(1 2 3) ’(10 100 1000))
Error: Illegal function object:
(8217 . #<Interpreted Function (unnamed) @ #x7162806a>).
[condition type: PROGRAM-ERROR]
Restart actions (select using :continue):
0: Abort entirely from this process.
[1] CL-USER(13): :res
:res
CL-USER(14): (apropos :sharp-quote)
(apropos :sharp-quote)
EXCL::SHARP-QUOTE [function] (STREAM CHR ARG)
:SHARP-QUOTE value: :SHARP-QUOTE
CL-USER(15): (set-dispatch-macro-character #\# #\right_single_quotation_mark (function EXCL::SHARP-QUOTE))
(set-dispatch-macro-character #\# #\right_single_quotation_mark (function EXCL::SHARP-QUOTE))
T
CL-USER(16): (mapcar #’+ ’(1 2 3) ’(10 100 1000))
(mapcar #’+ ’(1 2 3) ’(10 100 1000))
Error: Illegal function object: (8217 . #<Function SHARP-QUOTE>).
[condition type: PROGRAM-ERROR]
Restart actions (select using :continue):
0: Abort entirely from this process.
[1] CL-USER(17): :res
:res
CL-USER(18): (exit)
(exit)
; Exiting Lisp
Connection closed by foreign host.
> The code as written in the PDF files is *wrong*, so my original
> idea was to make it *right* before it goes into lisp.
> One problem with doing this transformation inside lisp, is that
> effectively it allows for more wrong coding to be arbitrarily added
> to source code: (in *all* cases, replace x1 with y1, replace x2
> with y2). Wrong source code can be propagated by further
> copy-pasting the examples into other source code.
Yes.
On the other hand, sometimes (with some keyboard configurations, or
with some programs), you cannot even type a quote or backquote. The
keyboard may map only the apostrophe and curly quotation marks, or the
word processor may automatically convert to them. That's what we get
for making our computers "user-friendly".
--
__Pascal Bourguignon__ http://www.informatimago.com/
IMPORTANT NOTICE TO PURCHASERS: The entire physical universe,
including this product, may one day collapse back into an
infinitesimally small space. Should another universe subsequently
re-emerge, the existence of this product in that universe cannot be
guaranteed.
Keyboard mapping works if these questions can be answered:
1) Can this version of lisp access the clipboard contents?
Allegro: yes, it has (clipboard-object :text)
Lispworks: ?
Others: ?
With allegro, this is possible:
; gsc get-set-clipboard
(defun gsc ()
(setf (clipboard-object :text )
(FIX-QUOTES (clipboard-object :text))))
(defun FIX-QUOTES (from-clipboard) "search/replace routine goes here")
where FIXQUOTES is defined to do two search/replace loops
on the string in the clipboard, which is sent back to the clipboard
via setf.
2) Does the IDE allow user-defined accelerator keys (like CTRL-V)
that can be tied into a function?
-- This question requires more knowledge of the IDEs involved than I
have.
Maybe the support crews can answer it.
If 1 and 2 hold, then we set up an unused accelerator key combination
(i.e. CTRL-SHIFT-V) to run the gsc function.
Once the clipboard content is fixed, then the standard CTRL-V paste
can be used. Or alternatively, incorporate the 'paste' functionality
into the gsc function, so that CTRL-SHIFT-V not only fixes the
clipboard
but also puts the result on the screen.
"NicknameOptional" <··········@yahoo.com> writes:
> Allegro CL 7.0 :
>
> International Allegro CL Trial Edition
> 7.0 [Windows] (Apr 9, 2006 10:40)
> Copyright (C) 1985-2004, Franz Inc., Oakland, CA, USA. All Rights
> Reserved.
>
> This development copy of Allegro CL is licensed to:
> Trial User
>
> CG version 1.54.2.17.2.12 / IDE version 1.48.2.20.2.16
> ;; Optimization settings: safety 1, space 1, speed 1, debug 2.
> ;; For a complete description of all compiler switches given the
> current optimization settings evaluate
> ;; (EXPLAIN-COMPILER-SETTINGS).
>
> [changing package from "COMMON-LISP-USER" to "COMMON-GRAPHICS-USER"]
> CG-USER(1): (set-macro-character #\LEFT_SINGLE_QUOTATION_MARK
> (get-macro-character #\GRAVE_ACCENT))
> T
> CG-USER(2): (set-macro-character #\RIGHT_SINGLE_QUOTATION_MARK
> (get-macro-character #\APOSTROPHE))
> T
> CG-USER(3): (coerce #(#\LEFT_SINGLE_QUOTATION_MARK
> #\RIGHT_SINGLE_QUOTATION_MARK) 'string)
>
> "''"
> CG-USER(4):
> CG-USER(4): (mapcar #'+
> '(1 2 3)
> '(10 100 1000))
> Error: No dispatch function defined for #\right_single_quotation_mark.
> [condition type: SIMPLE-ERROR]
> CG-USER(5):
Now is a good time to figure out how dispatch macro characters work.
Try throwing in a:
(set-dispatch-macro-character #\# #\right_single_quotation_mark
(get-dispatch-macro-character #\# #\'))
Here is code that will convert smart quotes inside
a particular IDE environment (ACL 7.0 for windows).
It has the same functionality as the rebol code, i.e,
modifying the clipboard before it is pasted into lisp.
It does not otherwise modify how lisp interprets
smart quotes.
; Convert PDF smart quotes
; This code is specific to ACL 7.0 for Windows
; in cg-user mode (not console mode)
; and is not portable to other IDEs.
; To use: Copy from PDF to clipboard as usual (CTRL-C)
; Instead of CTRL-V, Press CTRL-1 (one, not L )
(defun fix-quotes (clipin)
(let (clipout )
(setf clipout (substitute #\` #\' clipin ))
(setf clipout (substitute #\' #\' clipout ))
clipout ))
; ACL only -- non-portable
(defun get-set-clipboard (window)
(setf window cg.base:*window-under-cursor*)
(setf (clipboard-object :text )
(fix-quotes (clipboard-object :text)))
(paste-command window)
t)
; ACL only -- non-portable
; assign modified text to CTRL-1
(add-global-keyboard-accelerator
'(control-key #\1) 'get-set-clipboard)