From: Edi Weitz
Subject: CLtL2 - generating index from LaTeX sources
Date:
Message-ID: <ulk3dc0oy.fsf@agharta.de>
Has someone with enough TeX fu figured out how to generate the indices
from the CLtL2 LaTeX sources? The DVI files floating around come
without indices, and if I just run latex209 on clm.tex I also end up
with something without any index.
Yes, the .idx file is generated and I can use mkindex to generate an
.ind file, but what next? Using the "makeidx" extension results in
error messages, and it seems CLTL.sty defines its own index commands
anyway.
I'm lost at this point. Yes, the last time I used TeX in earnest must
have been more than ten years ago.
Thanks,
Edi.
PS: A PDF with indices would also be fine - that's what I want to
create anyway.
--
European Common Lisp Meeting, Amsterdam, April 19/20, 2008
http://weitz.de/eclm2008/
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
On Apr 16, 7:51 am, Edi Weitz <········@agharta.de> wrote:
> Has someone with enough TeX fu figured out how to generate the indices
> from the CLtL2 LaTeX sources? The DVI files floating around come
> without indices, and if I just run latex209 on clm.tex I also end up
> with something without any index.
>
> Yes, the .idx file is generated and I can use mkindex to generate an
> .ind file, but what next? Using the "makeidx" extension results in
> error messages, and it seems CLTL.sty defines its own index commands
> anyway.
>
> I'm lost at this point. Yes, the last time I used TeX in earnest must
> have been more than ten years ago.
I have some experience making indexed LaTeX. For instance, I did this
when documenting the Kazlib C library. I haven't gone anywhere near
this in years, but the steps are captured in the Makefile.
This uses the new \documentclass{article} so that's another thing to
consider; though I'm not familiar with the internals of the cltl
document, it probably uses outdated, legacy LaTeX. E.g. \documentstyle
instead of \documentclass. It might be worth converting to modern
macros.
The working copy of my LaTeX source is dated:
%
% $Id: docs.ltx,v 1.79 2001/07/08 08:17:25 kaz Exp $
% $Name: $
%
The index information is collected by means of manually inserted
\index commands. I define some trivial macros to take some gruntwork
out of some of these:
··························@{\tt #1} type}}
···························@{\tt #1} macro}}
····························@{\tt #1} object}}
··························@{\tt #1} function}}
··························@{\tt #1} enum constant}}
Thus
\indextype{foo} -> ··········@{\tt foo} type}
ISTR that the @ separates the indexed word, which determines its
position in the lexicographical order, from the way it is to be
displayed.
The document ends like this:
\index{external names|see {functions}}
\index{reference implementation|see {implementation}}
\index{names|see {symbols}}
\index{identifiers|see {symbols}}
\index{structure names|see{tags}}
\index{preprocessor symbols|see{macros}}
\index{defines|see{macros}}
\index{reserved symbols|see{symbols}}
\index{symbols!preprocessor|see{macros}}
\index{symbols!type names|see{typedefs}}
\index{symbols!function names|see{functions}}
\printindex
\end{document}
In other words, here I define some ``see also'' type index entries
which aren't associated with any page number of the document, and then
a \printindex command indicates where the index goes. The ! in the
entries indicates hierarchical subordination. So symbols!preprocessor
means there will be a ``preprocessor symbols'' entry in the index
under P, but also a ``symbols'' category under S, with an indented
``preprocessor'' entry. Both will tell the reader to go see ``macros''
on page 56 or whatever.
Next, I have a file called docs.ist. This contains:
preamble
"\\begin{theindex}\n\\addcontentsline{toc}{section}{Index}\n"
postamble
"\n\\end{theindex}\n"
The preamble will be added to the front of docs.ind, and the postamble
to the bottom. I think these are important; without the
\begin{theindex} and corresponding end, it probably doesn't work; the
commands in the index probably work only in the theindex environment.
The addcontentsline is probably something I added, so that the index
appears in the table of contents.
Running latex on the docs.ltx file produces a file called docs.idx.
This contains the raw database of index entries, formatted like this:
\indexentry{successor!of a list element}{6}
\indexentry{List!successor of an element}{6}
\indexentry{predecessor!of a list element}{6}
\indexentry{List!predecessor of an element}{6}
As you can see, the page numbers are pinned down at this point
already, but otherwise, the entries contain similar content to the
original \index commands in the document.
The makeindex command is run like this:
makeindex -o docs.ind -s docs.ist docs.idx.stable
The reason my Makefile command uses docs.idx.stable rather than
docs.idx is to eliminate some iteration, because of the circular
dependencies. The document DVI depends on the index, but the index is
also the result of running latex. So I have a step in the Makefile
which checks whether docs.idx.stable is different from docs.idx and
copies the former to the former.
# Makefile rule
docs.idx.stable: docs.idx
@if ! diff -q docs.idx docs.idx.stable ; then \
cp docs.idx docs.idx.stable ; \
fi
If they are the same, then no copy is made and hence the timestamp of
the docs.idx.stable is not disturbed. The makeindex step then doesn't
have to be run, and since it is not run, it doesn't disturb the index,
which would then cause the .dvi to be out of date again!
In any case, the output of makeindex is the docs.ind file, using the
index style file specified by -s, and the raw index database specified
by the docs.idx.stable argument.
The docs.ind file is just the LaTeX source which is brought into the
translation by the \printindex command:
begin{theindex}
\addcontentsline{toc}{section}{Index}
\item append node to list, 8
\indexspace
\item comparison function, 3
\item constraint, 3
\item create
\subitem dictionary object, 34
\subitem hash object, 24
\subitem list object, 8
[ ... etc ]
\item {\tt XCEPT_GROUP_ANY} macro, 45
\item {\tt XCEPT_H} macro, 45
\end{theindex}
The .ind is nicely indented so you can see the structure, even though
the whitespace is not significant. These \item commands and such are
almost certainly defined by the ``theindex'' environment which
surrounds all this.
The rule chain in the Makefile has to be invoked at least twice. The
first make job will spit out a document that has no index and no table
of contents, because at the time it was generated, there was no index
database. The second make job will have the table of contents and
index, so it can generate the final document. But because of the table
of contents, the page numbers of the index may have shifted. So the
index is not yet stable at that point: the docs.idx is different from
docs.idx.stable, and therefore is copied over and the makeindex will
run. Then if you run make for a third time, the .dvi is regenerated
with the correct line numbers in the index. The docs.idx.stable should
be identical to docs.idx at this point, and so the dependency loop is
closed off.
So that's it. If you want to have look at all this, Google for "kazlib
1.20". I don't know, maybe the clues there can serve as a guide to
getting indexing working over the cltl, maybe not.
It could be worth doing some maintenance on the cltl document, if its
LaTeX dialect is out of date, since it has value to the community.
On Apr 16, 7:51 am, Edi Weitz <········@agharta.de> wrote:
> Has someone with enough TeX fu figured out how to generate the indices
> from the CLtL2 LaTeX sources? The DVI files floating around come
> without indices, and if I just run latex209 on clm.tex I also end up
> with something without any index.
>
> Yes, the .idx file is generated and I can use mkindex to generate an
> .ind file, but what next?
What LaTeX distro are you using? The modern LaTeX2e chokes on this
document, in spite of attempting to provide backward compatibility. I
don't have a ``latex209'' command installed.
What seems to be the case that the clm.tex document doesn't have any
command to spit out the index. In its place, there is a
% index
line. Try replacing this with \printindex.
From: Edi Weitz
Subject: Re: CLtL2 - generating index from LaTeX sources
Date:
Message-ID: <ulk3d9zdw.fsf@agharta.de>
On Wed, 16 Apr 2008 14:48:21 -0700 (PDT), Kaz Kylheku <········@gmail.com> wrote:
> What LaTeX distro are you using?
The one that comes with Debian/Ubuntu.
> The modern LaTeX2e chokes on this document, in spite of attempting
> to provide backward compatibility. I don't have a ``latex209''
> command installed.
http://packages.debian.org/etch/latex209-bin
> What seems to be the case that the clm.tex document doesn't have any
> command to spit out the index. In its place, there is a
>
> % index
>
> line. Try replacing this with \printindex.
That was the first thing I tried.
! Undefined control sequence.
l.88 \printindex
And, as I said in my original message, if I add "makeidx" to the
documentstyle options it chokes as well.
Document style option `makeidx' - released 21 Oct 91
! You can't use `\spacefactor' in vertical mode.
Edi.
--
European Common Lisp Meeting, Amsterdam, April 19/20, 2008
http://weitz.de/eclm2008/
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Edi Weitz <········@agharta.de> wrote:
> Yes, the .idx file is generated and I can use mkindex to generate an
> .ind file, but what next? Using the "makeidx" extension results in
> error messages, and it seems CLTL.sty defines its own index commands
> anyway.
Indeed, it looks like it's doing something fairly complicated.
The index which is produced by LaTeX only seems to list the symbols
rather than the other random things which the index of the proper book
contains. For example, there's nothing that I can see in the source
which suggests that `aardvarks' ought to be indexed, apart from an entry
in the Index-stuff-sorted file.
It certainly seems that constructing the index is more of a manual
process than we'd like, While it'd be possible to use the generated
clm.ind to populate the various symbol indices, constructing a full
index by attempting to merge that lot into the Index-stuff-sorted file
somehow is doomed to failure due to page renumbering. The only way I
can see to do this in an automated way is to trundle through the
Index-stuff-sorted list and corresponding LaTeX, putting in commands to
write stuff to the .idx file.
Main index aside, the job of building the various symbol indices
probably isn't too difficult, fortunately, but I don't have time tonight
to play with it.
Sorry.
-- [mdw]
On Apr 16, 11:28 am, Mark Wooding <····@distorted.org.uk> wrote:
> The index which is produced by LaTeX only seems to list the symbols
> rather than the other random things which the index of the proper book
> contains. For example, there's nothing that I can see in the source
> which suggests that `aardvarks' ought to be indexed, apart from an entry
> in the Index-stuff-sorted file.
That is interesting, because you would expect explicit \index commands
to appear. Given who wrote that document, you can expect some clever
surprises.
Ah, I see. There are two curious files in the distribution:
Function-data-by-kind
Function-data-by-name
I don't see any software to keep this up to date. It seems to
incorporate line numbers from the .idx, but the information which
assigns symbols to categories like Function or Macro is nowhere else.
Somehow the .idx file ends up with these categories.
But the categories do not come from these files. In fact, the files
are out of date. The line numbers in Function-data-by-kind and
Function-data-by-name do not match the line numbers in the index. So I
suspect these files are not sources, but derived objects; they appear
to be red herrings. I don't see any software to generate these. Steele
probably had some script to do this, but it's not included in the
distribution.
Somehow the \makeindex macro in clm.tex generates the .idx file with
all of the symbols assigned categories like ``Special form''.
What's going on is that there is some funky macrology in defun.tex.
For instance, in this file you find the following macro:
··············@defmacbegin{Special form}}
\let\enddefspec\enddefun
Now, for example, inside progs.tex you find this:
\begin{defspec}
eval-when ({situation}*) {\,form}*
...
\end{defspec}
Somehow this defspec environment picks up the symbol eval-when
(possibly because it's the immediately following token?). And I
suspect this is how eval-when ends up being indexed, and categorized
as a ``Special form'' in the .idx file.
To see what is going on, you have to chase through the \defspec macro.
I see, the expansion ends up calling @defmacname which takes two
arguments #1 #2. This is probably what steals the secodn token, such
as eval-when. Then there is a call:
·@defunindex#2:\par{\lowercase{#1}}
I think that this is what introduces the identifier #2 as an index
entry of category #1. In fact the \lowercase form appears verbatim in
the .idx.
Indeed @defunindex is the macro which expands to the \index call by
way of @idefunindex.
So that is how the index is generated; from the \begin{defspec},
\begin{defun} and others.
On Apr 16, 10:51 am, Edi Weitz <········@agharta.de> wrote:
> Has someone with enough TeX fu figured out how to generate the indices
> from the CLtL2 LaTeX sources? The DVI files floating around come
> without indices, and if I just run latex209 on clm.tex I also end up
> with something without any index.
>
> Yes, the .idx file is generated and I can use mkindex to generate an
> .ind file, but what next? Using the "makeidx" extension results in
> error messages, and it seems CLTL.sty defines its own index commands
> anyway.
>
> I'm lost at this point. Yes, the last time I used TeX in earnest must
> have been more than ten years ago.
>
> Thanks,
> Edi.
>
> PS: A PDF with indices would also be fine - that's what I want to
> create anyway.
>
> --
>
> European Common Lisp Meeting, Amsterdam, April 19/20, 2008
>
> http://weitz.de/eclm2008/
>
> Real email: (replace (subseq ·········@agharta.de" 5) "edi")
The latex source for the index of functions, variables, and special
forms appears to be in the file clm.ind. But this file is damaged.
Every \subitem command appears to have the 5 following characters
deleted. Perhaps someone put it through some kind of script and
inadvertently munged it. If someone can find an undamaged version of
the file and post a pointer to it, I will probably be able to get it
printed, at least as a standalone document.