From: Marc Battyani
Subject: cl-typesetting
Date: 
Message-ID: <bkkvpe$4ar@library2.airnews.net>
I'm currently working on cl-typesetting, a CL typesetting engine based on
cl-pdf.
I want a Lispy text representation similar to the one used by the zillions of
HTML generation macros already there (including mine). So that it can very
easily be manipulated and extended in Lisp while being user friendly enough.
Thought as I'm not a typography expert I would like to know if it is a good
enough representation or if I will hit some limitation in the future.

Here is an example of the current syntax:

(with-text
 (paragraph (:h-align :centered)
    (text-style (:font "Helvetica" :font-size 20)
 "cl-typsetting") :eol
    (text-style (:font "Helvetica Bold Oblique" :font-size 10)
 "The cool Common Lisp typesetting system"))
  (paragraph (:h-align :justified :top-margin 10)
      "This typesetting system's goal is to be an alternative to the TeX
typesetting system. It has the following advantages: " :eol
      (loop for i from 1
     for advantage in '("extensible" "powerful" "fast" "based on lisp"
"etc.")
     do (add-to-text "#" i " :" advantage :eol))))

Comments welcomed.

Marc

From: John Williams
Subject: Re: cl-typesetting
Date: 
Message-ID: <87wuc15onq.fsf@heisenberg.aston.ac.uk>
One solution to this issue that I use is to provide a s-exp to latex
conversion as well as an s-exp to html. I then run latex and ancillary
programs to produce a pdf file which, typically, I serve off the web
server as the print alternative for content. It copes with included
figures as well as the more usual markup. This is I think a much
simpler task than trying to write a completely new typesetter. Figures
etc are also dealt with. The amount of effort to recreate TeX/LaTeX is
I think very substantial - and is there any need since LaTeX does its
job very well and is ubiqituous.

>>>>> "Marc" == Marc Battyani <·············@fractalconcept.com> writes:

    Marc> I'm currently working on cl-typesetting, a CL typesetting
    Marc> engine based on cl-pdf.  I want a Lispy text representation
    Marc> similar to the one used by the zillions of HTML generation
    Marc> macros already there (including mine). So that it can very
    Marc> easily be manipulated and extended in Lisp while being user
    Marc> friendly enough.  Thought as I'm not a typography expert I
    Marc> would like to know if it is a good enough representation or
    Marc> if I will hit some limitation in the future.

    Marc> Here is an example of the current syntax:

    Marc> (with-text (paragraph (:h-align :centered) (text-style
    Marc> (:font "Helvetica" :font-size 20) "cl-typsetting") :eol
    Marc> (text-style (:font "Helvetica Bold Oblique" :font-size 10)
    Marc> "The cool Common Lisp typesetting system")) (paragraph
    Marc> (:h-align :justified :top-margin 10) "This typesetting
    Marc> system's goal is to be an alternative to the TeX typesetting
    Marc> system. It has the following advantages: " :eol (loop for i
    Marc> from 1 for advantage in '("extensible" "powerful" "fast"
    Marc> "based on lisp" "etc.")  do (add-to-text "#" i " :"
    Marc> advantage :eol))))

    Marc> Comments welcomed.

    Marc> Marc



-- 
Dr. John A.R. Williams    | http://www.aston.ac.uk/~willijar
Photonics Research Group  | http://www.ee.aston.ac.uk/research/photonics
Electronic Engineering    | http://www.ee.aston.ac.uk/
Aston University          | http://www.aston.ac.uk

	
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <87r8288yna.fsf@g.mccaughan.ntlworld.com>
Marc Battyani wrote:

> I'm currently working on cl-typesetting, a CL typesetting engine based on
> cl-pdf.

Cool!

> I want a Lispy text representation similar to the one used by the zillions of
> HTML generation macros already there (including mine). So that it can very
> easily be manipulated and extended in Lisp while being user friendly enough.
> Thought as I'm not a typography expert I would like to know if it is a good
> enough representation or if I will hit some limitation in the future.

You'll want to define a light-weight representation that lets
users do simple things simply while still providing an escape
into something more powerful. If the syntax that follows, or
anything like it, is the *only* representation then you'll
probably find that only existing CL users (and maybe Schemers)
will use CL-TYPESETTING...

> (with-text
>  (paragraph (:h-align :centered)
>     (text-style (:font "Helvetica" :font-size 20)
>  "cl-typesetting") :eol
>     (text-style (:font "Helvetica Bold Oblique" :font-size 10)
>  "The cool Common Lisp typesetting system"))
>   (paragraph (:h-align :justified :top-margin 10)
>       "This typesetting system's goal is to be an alternative to the TeX
> typesetting system. It has the following advantages: " :eol
>       (loop for i from 1
>      for advantage in '("extensible" "powerful" "fast" "based on lisp"
> "etc.")
>      do (add-to-text "#" i " :" advantage :eol))))
> 
> Comments welcomed.

I suspect you'll want a higher level of abstraction for
fonts and text styles. Maybe that will be taken care of
using macros or something. For instance, surely you'll
want notions like
  - font family ("Helvetica", "Times", "Minion")
  - font size scale (embodying a set of size decisions:
    10 point for normal text, 14 for sub-headings, 18
    for headings, 8 for small print, etc)
  - text style (specifying some combination of font,
    size, size scale, various spacing things, etc)

Before you start thinking too hard about syntax, you
should probably pay careful attention to the underlying
typesetting model of the system; it may have implications.
Are you, for instance, going to go with TeX-like boxes
and glue? Or with a constraint-satisfaction system,
where you say "this thing must be at the same position
as that thing, vertically" and so on? Or what?

Suppose you wanted to make that list be typeset in some
more obviously list-like way -- with, say, the numbers
right-aligned to some fixed point, and the list items
given hanging indentation (or, thinking about it differently,
typeset in a box whose left margin is further right than
that of the enclosing material), like this:

  1 extensible
  2 powerful
  3 fast
  4 based on lisp
  5 this is a much longer advantage than any of
    the ones you listed, just to show what should
    be done when the text is too long for a single
    line
 17 and this is an item with a longer number
 18 etc.

How would that look? And suppose your system comes with
a standard way of indicating "this stuff is in an
itemized list; please format it in whatever the usual
way of formatting itemized lists is at the moment".
How, then, would you write that loop?

How will you encode

  - drop capitals?
  - embedded images?
  - complicated tables?
  - cross-references?
  - tables of contents?
  - mathematical formulae?
  - footnotes?

What will the system do as a result of whatever you write?

The point of all these questions isn't that I think you
can't answer them. It's that answering them will tell you
much more about whether your syntax has the flexibility
it needs than anything someone else can say. (But if you
can't answer them then you probably need to do some more
work on the semantics before freezing the syntax...)

-- 
Gareth McCaughan
.sig under construc
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bknnfl$crh@library2.airnews.net>
"Gareth McCaughan" <················@pobox.com> wrote
> Marc Battyani wrote:
>
> > I'm currently working on cl-typesetting, a CL typesetting engine based on
> > cl-pdf.
>
> Cool!

Good, at least one person interested. ;-)

> > I want a Lispy text representation similar to the one used by the
zillions of
> > HTML generation macros already there (including mine). So that it can
very
> > easily be manipulated and extended in Lisp while being user friendly
enough.
> > Thought as I'm not a typography expert I would like to know if it is a
good
> > enough representation or if I will hit some limitation in the future.
>
> You'll want to define a light-weight representation that lets
> users do simple things simply while still providing an escape
> into something more powerful. If the syntax that follows, or
> anything like it, is the *only* representation then you'll
> probably find that only existing CL users (and maybe Schemers)
> will use CL-TYPESETTING...

Sure. (There have been some debate on the lispweb list about this.)
This Lisp representation is the medium layer one. Below this is the low level
box+glue one. Above this will be a more user friendly one, though I find it
already lispnik friendly.
I want the medium level to be extremely powerful so that the system can be
easily extended.

> > (with-text
> >  (paragraph (:h-align :centered)
> >     (text-style (:font "Helvetica" :font-size 20)
> >  "cl-typesetting") :eol
> >     (text-style (:font "Helvetica Bold Oblique" :font-size 10)
> >  "The cool Common Lisp typesetting system"))
> >   (paragraph (:h-align :justified :top-margin 10)
> >       "This typesetting system's goal is to be an alternative to the TeX
> > typesetting system. It has the following advantages: " :eol
> >       (loop for i from 1
> >      for advantage in '("extensible" "powerful" "fast" "based on lisp"
> > "etc.")
> >      do (add-to-text "#" i " :" advantage :eol))))
> >
> > Comments welcomed.
>
> I suspect you'll want a higher level of abstraction for
> fonts and text styles. Maybe that will be taken care of
> using macros or something. For instance, surely you'll
> want notions like
>   - font family ("Helvetica", "Times", "Minion")
>   - font size scale (embodying a set of size decisions:
>     10 point for normal text, 14 for sub-headings, 18
>     for headings, 8 for small print, etc)
>   - text style (specifying some combination of font,
>     size, size scale, various spacing things, etc)

This will be in the upper layers though I already have named styles.
(paragraph (:style abstract)...)

> Before you start thinking too hard about syntax, you
> should probably pay careful attention to the underlying
> typesetting model of the system; it may have implications.
> Are you, for instance, going to go with TeX-like boxes
> and glue? Or with a constraint-satisfaction system,
> where you say "this thing must be at the same position
> as that thing, vertically" and so on? Or what?

I use the classical boxes+glue model. I toyed with the idea of using
something like Screamer but not for this release.

> Suppose you wanted to make that list be typeset in some
> more obviously list-like way -- with, say, the numbers
> right-aligned to some fixed point, and the list items
> given hanging indentation (or, thinking about it differently,
> typeset in a box whose left margin is further right than
> that of the enclosing material), like this:
>
>   1 extensible
>   2 powerful
>   3 fast
>   4 based on lisp
>   5 this is a much longer advantage than any of
>     the ones you listed, just to show what should
>     be done when the text is too long for a single
>     line
>  17 and this is an item with a longer number
>  18 etc.
>
> How would that look? And suppose your system comes with
> a standard way of indicating "this stuff is in an
> itemized list; please format it in whatever the usual
> way of formatting itemized lists is at the moment".
> How, then, would you write that loop?

Well I'm at version 0.01 so I'm open to suggestions...
In that case I would set a left margin to x with a first line indentation
of -y and a goto x after the number. (or something like that) But this would
be wrapped in a cool with-numbered-list macro.

> How will you encode
>
>   - drop capitals?
>   - embedded images?
>   - complicated tables?
>   - cross-references?
>   - tables of contents?
>   - mathematical formulae?
>   - footnotes?

Complicated tables are a complicated subject that I will treat in priority.
Math mode will be left away for now as I don't need it.
Embedded images will be jpeg only for now as I already support them in
cl-pdf.
The other points will follow later.

Obviously I can't write a complete replacement of TeX + LaTeX + ConTeX + ...
alone in a few days.
The first objective is to be able to write small documents like reports,
memos, letters, etc.

> What will the system do as a result of whatever you write?

I don't understand the question. The system will generate a pdf file.

> The point of all these questions isn't that I think you
> can't answer them. It's that answering them will tell you
> much more about whether your syntax has the flexibility
> it needs than anything someone else can say. (But if you
> can't answer them then you probably need to do some more
> work on the semantics before freezing the syntax...)

Nothing is frozen yet.

Marc
From: Gisle Sælensminde
Subject: Re: cl-typesetting
Date: 
Message-ID: <slrnbmvr2n.p5s.gisle@ginkgo.ii.uib.no>
In article <··········@library2.airnews.net>, Marc Battyani wrote:
> 
>> > (with-text
>> >  (paragraph (:h-align :centered)
>> >     (text-style (:font "Helvetica" :font-size 20)
>> >  "cl-typesetting") :eol
>> >     (text-style (:font "Helvetica Bold Oblique" :font-size 10)
>> >  "The cool Common Lisp typesetting system"))
>> >   (paragraph (:h-align :justified :top-margin 10)
>> >       "This typesetting system's goal is to be an alternative to the TeX
>> > typesetting system. It has the following advantages: " :eol
>> >       (loop for i from 1
>> >      for advantage in '("extensible" "powerful" "fast" "based on lisp"
>> > "etc.")
>> >      do (add-to-text "#" i " :" advantage :eol))))
>> >
>> > Comments welcomed.

In a formated text, the text is still the most important part of it. The 
s-expr syntax as stated above reduce the text to a less important element
syntax-wise (it must be in quotes, unlike the formating), and also, in text
(like this for example), I often use "quotes". This can make it easy to 
intoduce errors, and you may need quoting for quotes and parantheses.
The syntax like this will be ideal for making figures and other elements
where the formating dominates, but less ideal for documents with mostly 
text.

(La)TeX, that you state you are an alternative to, does it the other way around.
It is readable as long as you don't add too much formating. When adding much
formating, care must be taken to keep it readable.

--
Gisle S�lensminde
Computational biology unit, University of Bergen, Norway
Email: ·····@cbu.uib.no
From: Ray Blaak
Subject: Re: cl-typesetting
Date: 
Message-ID: <uvfrjjvjl.fsf@STRIPCAPStelus.net>
Gisle S�lensminde <·····@ginkgo.ii.uib.no> writes:
> In article <··········@library2.airnews.net>, Marc Battyani wrote:
> > 
> >> > (with-text
> >> >  (paragraph (:h-align :centered)
> >> >     (text-style (:font "Helvetica" :font-size 20)
> >> >  "cl-typesetting") :eol
> >> >     (text-style (:font "Helvetica Bold Oblique" :font-size 10)
> >> >  "The cool Common Lisp typesetting system"))
> >> >   (paragraph (:h-align :justified :top-margin 10)
> >> >       "This typesetting system's goal is to be an alternative to the TeX
> >> > typesetting system. It has the following advantages: " :eol
> >> >       (loop for i from 1
> >> >      for advantage in '("extensible" "powerful" "fast" "based on lisp"
> >> > "etc.")
> >> >      do (add-to-text "#" i " :" advantage :eol))))
> >> >
> >> > Comments welcomed.

I would argue it is better to try and make the typing of normal text easy
*and* make the typing of code easy, *and* try and have everything readable.

Take TeX's idea of a special "escape" char, but in this case make it "lispy":
consider \( to start code.

Make common controls be shorted, e.g. "par" instead of "paragraph".

Paragraph controls don't have to nest. Instead make them separators. This can
be generalized: separate grouping and make text styles and such apply to the
current group.

Make the default mode be text, not code, but code is easily a \( step away.

Make common no-arg controls also be available directly, e.g. \newline as well
as \(newline).

Separate paragraph and text style attributes into different controls,
e.g. setting the base font, setting italics, bold, centering, etc. The reason
is that then you can have text "flows" with minimal tweaks to presentation
that can be succinctly describe, as opposed to verbose setups.

With these thoughts here is the example redone:

 \(font "Helvetica")
 \(font-size 10)

 \(par :center)
 \(group
   \(font-size 20)
   cl-typesetting \newline)

 \(group
   \bold\italic
   The cool Common Lisp typesetting system)

 \(par :justified :top-margin 10)
 This typesetting system's goal is to be an alternative to the TeX
 typesetting system. It has the following advantages: \newline

 \(loop for i from 1
    for advantage in
        '("extensible" "powerful" "fast" "based on lisp" "etc.")
    do (emit-text "#" i " :" advantage :eol))

 \par
 Here is some followup text that is presented in the current style.

-- 
Cheers,                                        The Rhythm is around me,
                                               The Rhythm has control.
Ray Blaak                                      The Rhythm is inside me,
········@STRIPCAPStelus.net                    The Rhythm has my soul.
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkq00b$s07@library2.airnews.net>
"Ray Blaak" <········@STRIPCAPStelus.net> wrote

> I would argue it is better to try and make the typing of normal text easy
> *and* make the typing of code easy, *and* try and have everything readable.
>
> Take TeX's idea of a special "escape" char, but in this case make it
"lispy":
> consider \( to start code.

Not a bad idea. But only if there is an Emacs mode that can properly cope
with this.

> Make common controls be shorted, e.g. "par" instead of "paragraph".
>
> Paragraph controls don't have to nest. Instead make them separators. This
can
> be generalized: separate grouping and make text styles and such apply to
the
> current group.
>
> Make the default mode be text, not code, but code is easily a \( step away.
>
> Make common no-arg controls also be available directly, e.g. \newline as
well
> as \(newline).
>
> Separate paragraph and text style attributes into different controls,
> e.g. setting the base font, setting italics, bold, centering, etc. The
reason
> is that then you can have text "flows" with minimal tweaks to presentation
> that can be succinctly describe, as opposed to verbose setups.

> With these thoughts here is the example redone:
>
>  \(font "Helvetica")
>  \(font-size 10)
>
>  \(par :center)
>  \(group
>    \(font-size 20)
>    cl-typesetting \newline)
>
>  \(group
>    \bold\italic
>    The cool Common Lisp typesetting system)
>
>  \(par :justified :top-margin 10)
>  This typesetting system's goal is to be an alternative to the TeX
>  typesetting system. It has the following advantages: \newline
>
>  \(loop for i from 1
>     for advantage in
>         '("extensible" "powerful" "fast" "based on lisp" "etc.")
>     do (emit-text "#" i " :" advantage :eol))
>
>  \par
>  Here is some followup text that is presented in the current style.

I rather like it.
I will continue my sexpr syntax and then I will ask for volunteers to
implement several human friendly syntax layers above it.

Marc
From: Thomas F. Burdick
Subject: Re: cl-typesetting
Date: 
Message-ID: <xcvvfri27hc.fsf@famine.OCF.Berkeley.EDU>
"Marc Battyani" <·············@fractalconcept.com> writes:

> "Ray Blaak" <········@STRIPCAPStelus.net> wrote
> 
> > I would argue it is better to try and make the typing of normal text easy
> > *and* make the typing of code easy, *and* try and have everything readable.
> >
> > Take TeX's idea of a special "escape" char, but in this case make
> > it "lispy": consider \( to start code.
> 
> Not a bad idea. But only if there is an Emacs mode that can properly cope
> with this.

If that's your greatest worry, don't.  I'll gladly make you an Emacs
mode.  I've been doing it longer than I've been writing CL.  I mean,
if the underlying system makes it worth using over AuxTeX (or if it
looks like it's headed that way)...

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkumjb$o0g@library1.airnews.net>
"Thomas F. Burdick" <···@famine.OCF.Berkeley.EDU> wrote in message
····················@famine.OCF.Berkeley.EDU...
> "Marc Battyani" <·············@fractalconcept.com> writes:
>
> > "Ray Blaak" <········@STRIPCAPStelus.net> wrote
> >
> > > I would argue it is better to try and make the typing of normal text
easy
> > > *and* make the typing of code easy, *and* try and have everything
readable.
> > >
> > > Take TeX's idea of a special "escape" char, but in this case make
> > > it "lispy": consider \( to start code.
> >
> > Not a bad idea. But only if there is an Emacs mode that can properly cope
> > with this.
>
> If that's your greatest worry, don't.  I'll gladly make you an Emacs
> mode.  I've been doing it longer than I've been writing CL.  I mean,
> if the underlying system makes it worth using over AuxTeX (or if it
> looks like it's headed that way)...

It's good as I have never written more than a few lines of el.
As for being as good as TeX, we will see this when it's finished. It's
already much better in the language for writing macros. But like TeX it will
be the packages written by users that will make it really useful. And I hope
that writing packages in Common Lisp will be fun enough for lots of people to
contribute.

Marc
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkpn4i$chq@library2.airnews.net>
"Gisle S�lensminde" <·····@ginkgo.ii.uib.no> wrote
> Marc Battyani wrote:
> >
> >> > (with-text
> >> >  (paragraph (:h-align :centered)
> >> >     (text-style (:font "Helvetica" :font-size 20)
> >> >  "cl-typesetting") :eol
> >> >     (text-style (:font "Helvetica Bold Oblique" :font-size 10)
> >> >  "The cool Common Lisp typesetting system"))
> >> >   (paragraph (:h-align :justified :top-margin 10)
> >> >       "This typesetting system's goal is to be an alternative to the
TeX
> >> > typesetting system. It has the following advantages: " :eol
> >> >       (loop for i from 1
> >> >      for advantage in '("extensible" "powerful" "fast" "based on lisp"
> >> > "etc.")
> >> >      do (add-to-text "#" i " :" advantage :eol))))
>
> In a formated text, the text is still the most important part of it. The
> s-expr syntax as stated above reduce the text to a less important element
> syntax-wise (it must be in quotes, unlike the formating), and also, in text
> (like this for example), I often use "quotes". This can make it easy to
> intoduce errors, and you may need quoting for quotes and parantheses.
> The syntax like this will be ideal for making figures and other elements
> where the formating dominates, but less ideal for documents with mostly
> text.

I agree with this. Text is not the most important part. I want this syntax
layer to be the most powerful for manipulation by Lisp programs.

> (La)TeX, that you state you are an alternative to, does it the other way
around.
> It is readable as long as you don't add too much formating. When adding
much
> formating, care must be taken to keep it readable.

IMHO, the main problem of TeX/LaTeX/ConTeX is the underlying language which
is several orders of magnitude less usable tahn Common Lisp. I also want to
be able to insert some native pdf stuff.

Marc
From: Thomas F. Burdick
Subject: Re: cl-typesetting
Date: 
Message-ID: <xcvzngu27sa.fsf@famine.OCF.Berkeley.EDU>
"Marc Battyani" <·············@fractalconcept.com> writes:

> "Gisle S�lensminde" <·····@ginkgo.ii.uib.no> wrote
>
> > In a formated text, the text is still the most important part of it. The
> > s-expr syntax as stated above reduce the text to a less important element
> > syntax-wise (it must be in quotes, unlike the formating), and also, in text
> > (like this for example), I often use "quotes". This can make it easy to
> > intoduce errors, and you may need quoting for quotes and parantheses.
> > The syntax like this will be ideal for making figures and other elements
> > where the formating dominates, but less ideal for documents with mostly
> > text.
> 
> I agree with this. Text is not the most important part. I want this syntax
> layer to be the most powerful for manipulation by Lisp programs.

From what I've seen here and on LispWeb, the sexp syntax looks like a
pretty damn good intermediate syntax.  I'd rush to start exploring the
high-level syntax.  You said you've got kerning working -- cool, so
you've kind of determined that this approach might work.  As a Lisper
and typesetter's grandson, I *love* TeX, compared to everything else
I've seen.  I'd also like to be able to manipulate s-expressions in my
text without losing the benefits of easy TeX typing.

Sure, it's wanting my cake and to eat it, but mostly I want to eat.
I'd easily give up the cake/Lisp, if it meant that I could eat/type
well...

> IMHO, the main problem of TeX/LaTeX/ConTeX is the underlying language which
> is several orders of magnitude less usable tahn Common Lisp. I also want to
> be able to insert some native pdf stuff.

Yeah, but it's mostly good.  When you want to program, it sucks, but
programming is only a little bit of what you do -- you mainly markup
text, and that's so damn nice in *TeX.

On the other hand...

  \TeX is essentially {\Tex} is essen (TeX)

  {\foo ...} is essentially (foo ...)

  \def\foo#1#2#3 ... is disturbingly like
  (def foo (x y z) ...)

Just getting rid of the icky lambda-calculus notation, in preference
for a sexp one would be a win.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: Ray Blaak
Subject: Re: cl-typesetting
Date: 
Message-ID: <uoexarbv7.fsf@STRIPCAPStelus.net>
···@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
>   \TeX is essentially {\Tex} is essen (TeX)
> 
>   {\foo ...} is essentially (foo ...)
> 
>   \def\foo#1#2#3 ... is disturbingly like
>   (def foo (x y z) ...)
> 

Careful. ( and ) are very commonly used in normal text. (I'm sure Knuth went
through these exercises like crazy -- we are only repeating them again.)

Have a reasonably uncommon, yet readable, and easy to type escape code:

   \TeX is essentially {\Tex} is essen \(TeX)
 
   {\foo ...} is essentially \(foo ...)
 
   \def\foo#1#2#3 ... is disturbingly like
   \(def foo (x y z) ...)

> Just getting rid of the icky lambda-calculus notation, in preference
> for a sexp one would be a win.

Yes indeed.

-- 
Cheers,                                        The Rhythm is around me,
                                               The Rhythm has control.
Ray Blaak                                      The Rhythm is inside me,
········@STRIPCAPStelus.net                    The Rhythm has my soul.
From: Joerg Hoehle
Subject: Re: cl-typesetting
Date: 
Message-ID: <un0crhqfg.fsf@users.sourceforge.net>
Ray Blaak <········@STRIPCAPStelus.net> writes:
> Careful. ( and ) are very commonly used in normal text. (I'm sure Knuth went
> through these exercises like crazy -- we are only repeating them again.)

My own thoughts about using a home-grown markup language went similarly:

I want text to be typical almost verbatim. This includes using ", ', (
and even [.

So as escaping brace I opted for {, which I "only" know from
mathematics. I think I could write Lisp code using {} as braces, but
that's not even required, since once you've escaped, you can use
normal Lisp () braces.

That's what TeX and RTF use. Maybe there's a reason. Maybe that's a
good reason in itself.

I don't like all those sexp-based languages that have have the text in
"quotes", e.g. (par "Here I am"). I believe that:
1. I cannot share such a markup with other, non-Lisp people.
2. it's error prone without Emacs, which few people use, see argument 1.

Of course, the internal notation is not what I talk about
here. Internally, and especially where code and text are expected to
mix a lot, I'd use normal s-exp.

Actually, for compatibility and ease of migration, I thought about
using a restricted LaTeX (also similar to RTF):
My writings {\emph focus} on text, with {\strong markup} as an option.
BTW, \em,\emph or \emphasis?

{\par Similarly to TeX, and unlike Lisp or XML docbook, I believe I
shouldn't force users to put a brace around a complete paragraph, or around a
whole (sub)section.}
This may be easy to achieve in Emacs, have advantages for
reliability, but is not easy to ensure using a stupid notepad-style
editor.
I believe that this also means that the format is not self-containing,
but needs processing rules (e.g. \subsection ends with next
\subsection or \section), like TeX. Hmm.

Regards,
	Jorg Hohle
Telekom/T-Systems Technology Center
From: Joe Marshall
Subject: Re: cl-typesetting
Date: 
Message-ID: <smmjpuy4.fsf@ccs.neu.edu>
Joerg Hoehle <······@users.sourceforge.net> writes:

> My writings {\emph focus} on text, with {\strong markup} as an option.
> BTW, \em,\emph or \emphasis?

\emphasis (\emphasize maybe?) by default, but users can obviously
define \em or \emph if they need to talk like William Shatner.
From: Alexander Schmolck
Subject: Re: cl-typesetting
Date: 
Message-ID: <yfszngrsjdz.fsf@black132.ex.ac.uk>
Joerg Hoehle <······@users.sourceforge.net> writes:

> Ray Blaak <········@STRIPCAPStelus.net> writes:
> > Careful. ( and ) are very commonly used in normal text. (I'm sure Knuth went
> > through these exercises like crazy -- we are only repeating them again.)
> 
> My own thoughts about using a home-grown markup language went similarly:
> 
> I want text to be typical almost verbatim. This includes using ", ', (
> and even [.

Have you had a look at http://docutils.sourceforge.net/rst.html? Despite some
flaws, I think the overall approach is really quite promising: 'normal' text,
including sectioning, emphasis, footnotes, various lists, etc. is both pretty
easy to write *and* quite readable (unlike latex or the numerous sexp-to-html
schemes I'v seen).

I should think this approach could easily be married with sexps to deliver a
combination of simplicity and readability for common cases with full
expressiveness (using curly ('{','}') braces for sexps and maybe something
like {(foo bar)} to embed lisp code; curlies seem really quite attractive
because they neither tend to occur much in normal text nor in lisp code, which
I guess should allow some nesting of both).

'as
From: Adam Warner
Subject: Re: cl-typesetting
Date: 
Message-ID: <pan.2003.09.26.10.20.14.133906@consulting.net.nz>
Hi Joerg Hoehle,

> Ray Blaak <········@STRIPCAPStelus.net> writes:
>> Careful. ( and ) are very commonly used in normal text. (I'm sure Knuth
>> went through these exercises like crazy -- we are only repeating them
>> again.)
> 
> My own thoughts about using a home-grown markup language went similarly:
> 
> I want text to be typical almost verbatim. This includes using ", ', (
> and even [.
> 
> So as escaping brace I opted for {, which I "only" know from
> mathematics. I think I could write Lisp code using {} as braces, but
> that's not even required, since once you've escaped, you can use normal
> Lisp () braces.

For non-programmers I created a document format with no escaping. Typical
text is completely verbatim. Given how easily organisations adapted to it
has been successful.

To avoid escaping the parser has to be context-sensitive and this is a
complication. The parser turns the document into my Lisp/LaTeX document
format for evaluation (this is the format I am comfortable with because it
provides the full power of Common Lisp).

The Syntactic Text "specification" is discussed here:
<https://macrology.co.nz/syntext/>

The best example of a Syntactic Text document is the source to the page
itself: <https://macrology.co.nz/syntext/?source>

This is the machine-generated file that gets evaluated:
<https://macrology.co.nz/syntext/?ldf-source>

Marc is on the right track in creating an underlying document format that
is based upon s-expressions. It is relatively easy to build another level
of abstraction upon them by creating a parser to machine generate the
s-expressions (in effect writing a Lisp program for evaluation).

...

> I don't like all those sexp-based languages that have have the text in
> "quotes", e.g. (par "Here I am"). I believe that: 1. I cannot share such
> a markup with other, non-Lisp people. 2. it's error prone without Emacs,
> which few people use, see argument 1.
> 
> Of course, the internal notation is not what I talk about here.
> Internally, and especially where code and text are expected to mix a
> lot, I'd use normal s-exp.

I increasing like to mix code and text so I am happy to write text as
strings so that the programming syntax is not inhibited. But in writing a
document format for non-programmers I agree with the comment.

> Actually, for compatibility and ease of migration, I thought about using
> a restricted LaTeX (also similar to RTF): My writings {\emph focus} on
> text, with {\strong markup} as an option. BTW, \em,\emph or \emphasis?
>
> {\par Similarly to TeX, and unlike Lisp or XML docbook, I believe I
> shouldn't force users to put a brace around a complete paragraph, or
> around a whole (sub)section.}

Here's the previous two paragraphs in Syntactic Text:

Actually, for compatibility and ease of migration, I thought about using a
restricted LaTeX (also similar to RTF): My writings (:i focus) on text,
with (:b markup) as an option. BTW, \em,\emph or \emphasis?

Similarly to TeX, and unlike Lisp or XML docbook, I believe I shouldn't
force users to put a brace around a complete paragraph, or around a whole
(sub)section.

> This may be easy to achieve in Emacs, have advantages for reliability,
> but is not easy to ensure using a stupid notepad-style editor. I believe
> that this also means that the format is not self-containing, but needs
> processing rules (e.g. \subsection ends with next \subsection or
> \section), like TeX. Hmm.

I generate TeX that is processed by TeX4ht so these processing rules
naturally apply. The approach isn't ideal. I'd rather have Common Lisp PDF
and HTML typesetters. But it is one solution to a problem that Marc may
be yet to tackle. That problem is the simultaneous support of Web and
print formats. I don't want to write a document in a format that only
generates PDF. I want to write it in a format that can generate HTML for
the WWW and PDF for print. And that's what I can currently do via
syntext/ldf and Eitan Gurari's TeX4ht.

Regards,
Adam

PS: Marc, your typesetting work is very impressive.
From: Adam Warner
Subject: Re: cl-typesetting
Date: 
Message-ID: <pan.2003.09.26.10.29.52.624022@consulting.net.nz>
[Message superseded with updated final paragraph]

Hi Joerg Hoehle,

> Ray Blaak <········@STRIPCAPStelus.net> writes:
>> Careful. ( and ) are very commonly used in normal text. (I'm sure Knuth
>> went through these exercises like crazy -- we are only repeating them
>> again.)
> 
> My own thoughts about using a home-grown markup language went similarly:
> 
> I want text to be typical almost verbatim. This includes using ", ', (
> and even [.
> 
> So as escaping brace I opted for {, which I "only" know from
> mathematics. I think I could write Lisp code using {} as braces, but
> that's not even required, since once you've escaped, you can use normal
> Lisp () braces.

For non-programmers I created a document format with no escaping. Typical
text is completely verbatim. Given how easily organisations adapted to it
has been successful.

To avoid escaping the parser has to be context-sensitive and this is a
complication. The parser turns the document into my Lisp/LaTeX document
format for evaluation (this is the format I am comfortable with because it
provides the full power of Common Lisp).

The Syntactic Text "specification" is discussed here:
<https://macrology.co.nz/syntext/>

The best example of a Syntactic Text document is the source to the page
itself: <https://macrology.co.nz/syntext/?source>

This is the machine-generated file that gets evaluated:
<https://macrology.co.nz/syntext/?ldf-source>

Marc is on the right track in creating an underlying document format that
is based upon s-expressions. It is relatively easy to build another level
of abstraction upon them by creating a parser to machine generate the
s-expressions (in effect writing a Lisp program for evaluation).

...

> I don't like all those sexp-based languages that have have the text in
> "quotes", e.g. (par "Here I am"). I believe that: 1. I cannot share such
> a markup with other, non-Lisp people. 2. it's error prone without Emacs,
> which few people use, see argument 1.
> 
> Of course, the internal notation is not what I talk about here.
> Internally, and especially where code and text are expected to mix a
> lot, I'd use normal s-exp.

I increasing like to mix code and text so I am happy to write text as
strings so that the programming syntax is not inhibited. But in writing a
document format for non-programmers I agree with the comment.

> Actually, for compatibility and ease of migration, I thought about using
> a restricted LaTeX (also similar to RTF): My writings {\emph focus} on
> text, with {\strong markup} as an option. BTW, \em,\emph or \emphasis?
>
> {\par Similarly to TeX, and unlike Lisp or XML docbook, I believe I
> shouldn't force users to put a brace around a complete paragraph, or
> around a whole (sub)section.}

Here's the previous two paragraphs in Syntactic Text:

Actually, for compatibility and ease of migration, I thought about using a
restricted LaTeX (also similar to RTF): My writings (:i focus) on text,
with (:b markup) as an option. BTW, \em,\emph or \emphasis?

Similarly to TeX, and unlike Lisp or XML docbook, I believe I shouldn't
force users to put a brace around a complete paragraph, or around a whole
(sub)section.

> This may be easy to achieve in Emacs, have advantages for reliability,
> but is not easy to ensure using a stupid notepad-style editor. I believe
> that this also means that the format is not self-containing, but needs
> processing rules (e.g. \subsection ends with next \subsection or
> \section), like TeX. Hmm.

I generate TeX that is processed by TeX4ht and PDFLaTeX so these
processing rules naturally apply. The approach isn't ideal. I'd rather
have common-format, Common Lisp HTML and PDF typesetters. But it provides
one solution to a problem that Marc may be yet to tackle. That problem is
the simultaneous support of Web and print formats. I don't want to write a
document in a format that only generates PDF. I want to write it in a
format that can generate HTML for the WWW and PDF for print. And that's
what I can currently do via syntext/ldf, TeX typesetters and Eitan
Gurari's TeX4ht.

Regards,
Adam

PS: Marc, your typesetting work is very impressive.
From: Pascal Costanza
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkrnve$uf0$1@f1node01.rhrz.uni-bonn.de>
Gisle S�lensminde wrote:

> In a formated text, the text is still the most important part of it. The 
> s-expr syntax as stated above reduce the text to a less important element
> syntax-wise (it must be in quotes, unlike the formating), and also, in text
> (like this for example), I often use "quotes". This can make it easy to 
> intoduce errors, and you may need quoting for quotes and parantheses.
> The syntax like this will be ideal for making figures and other elements
> where the formating dominates, but less ideal for documents with mostly 
> text.

Maybe it's a good idea to take a look at Curl, which is basically a Lisp 
dialect that uses curly braces. ;)

They have also made a switch from code being the default element to text.

See http://www.curl.com/


Pascal

-- 
Pascal Costanza               University of Bonn
···············@web.de        Institute of Computer Science III
http://www.pascalcostanza.de  R�merstr. 164, D-53117 Bonn (Germany)
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkumja$o0g@library1.airnews.net>
"Pascal Costanza" <········@web.de> wrote

> Maybe it's a good idea to take a look at Curl, which is basically a Lisp
> dialect that uses curly braces. ;)

Hmm, The Lisp dialect I prefer is the Common one...

> They have also made a switch from code being the default element to text.
>
> See http://www.curl.com/

I will look at this.

Marc
From: Roland Kaufmann
Subject: Re: cl-typesetting
Date: 
Message-ID: <tl2n0cvzsio.fsf@space.at>
                                Marc,
have you looked at Scheme scribe?

  Presentation 

  Scheme Scribe (Scribe hereafter) is a text processor. Even if it is
  a general purpose tool, it best suits the writing of technical
  documents such as web pages or technical reports, API
  documentations, etc. At first glance, Scribe looks like a mark-up
  language � la HTML. So, there is no need to be provided with
  computer programming skills in order to use Scribe.

  A second look reveals that Scribe is actually a true programming
  language, provided with high level features (such as objects, higher
  order functions, regular and syntactic parsing, etc.). Scribe is
  based on the Scheme programming language.

  From Scribe source files it is possible to produce various targets: 

        HTML pages that can be used to implement a web site. 
        LaTeX files that can be used to produce high quality
          Postscript or PDF files.
        Info files that suit the Emacs Info mode. 
        Man pages for Unix documentation. 
        Plain texts. 
        MagicPoint slides.        

http://www-sop.inria.fr/mimosa/fp/Scribe/

Even if you use a different approach, maybe you can use the same
syntax?

There is also the sad story of NTS (New Typesetting System), a
reimplementation of TeX in a more modular, extensible fashion.
After long discussions, where CLOS was considered as a possible
implementation language, it was finally implemented in Java.

                                   best regards
                                      Roland
From: Rene de Visser
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkpa77$msa$1@news1.wdf.sap-ag.de>
>                                 Marc,
> have you looked at Scheme scribe?

There's also the Scheme Scribe parser on CLIKI written in Common Lisp.

It performs the translation of the Scribe syntax into s-expressions.

i.e. it is attended as a front end for common lisp text processing systems
that
operate on s-expressions.

Rene.
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkpn50$am4@library2.airnews.net>
"Roland Kaufmann" <···············@space.at> wrote
>
>                                 Marc,
> have you looked at Scheme scribe?
...
> http://www-sop.inria.fr/mimosa/fp/Scribe/
>
> Even if you use a different approach, maybe you can use the same
> syntax?

Yes. Scribe is not a typesetting system, it outputs HTML or LaTeX. There is a
Common Lisp parser for this syntax. This could be used for the more user
friendly syntax.

> There is also the sad story of NTS (New Typesetting System), a
> reimplementation of TeX in a more modular, extensible fashion.
> After long discussions, where CLOS was considered as a possible
> implementation language, it was finally implemented in Java.

For what I heard, it is very very slow (may this has changed?) but pin
compatible with TeX.

Marc
From: Bill Clementson
Subject: Re: cl-typesetting
Date: 
Message-ID: <1b3ac8a3.0309230650.c108606@posting.google.com>
"Marc Battyani" <·············@fractalconcept.com> wrote in message news:<··········@library2.airnews.net>...
> I'm currently working on cl-typesetting, a CL typesetting engine based on
> cl-pdf.

Hi Marc,

Sounds neat - I was really impressed by cl-pdf and look forward to
seeing what you come up with for cl-typesetting.

Have you considered basing your work on XSL-FO (Note: the XSL standard
is made up of 2 sub-standards: XSLT for transformations and XSL-FO for
formatting. I'm referring specifically to XSL-FO here)? It is a W3C
recommendation (their terminology for a standard) and was designed as
an XML vocabulary for describing documents that could be rendered
using different engines (PDF was the first engine that most
implementations targeted). In addition to supporting text typesetting
and raster graphics output, it supports scalable vector graphics (SVG)
as well. (SVG output is trivial to do in Lisp but there is no
high-level SVG definition language - this could potentially be another
component in your offering). One of the benefits of basing it on
XSL-FO would be that your input definition would be in a
standards-based format and that might encourage a broader acceptance
for your product outside of the Lisp community. Another benefit would
be that there are validation test suites available for it which would
make your development job a lot easier.

Good luck with the project.
--
Bill Clementson
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkpt41$54f@library2.airnews.net>
"Bill Clementson" <···············@yahoo.com> wrote
> "Marc Battyani" <·············@fractalconcept.com> wrote
> > I'm currently working on cl-typesetting, a CL typesetting engine based on
> > cl-pdf.
>
> Hi Marc,
Hi Bill,

> Sounds neat - I was really impressed by cl-pdf and look forward to
> seeing what you come up with for cl-typesetting.

Thanks.

> Have you considered basing your work on XSL-FO (Note: the XSL standard
> is made up of 2 sub-standards: XSLT for transformations and XSL-FO for
> formatting. I'm referring specifically to XSL-FO here)? It is a W3C
> recommendation (their terminology for a standard) and was designed as
> an XML vocabulary for describing documents that could be rendered
> using different engines (PDF was the first engine that most
> implementations targeted). In addition to supporting text typesetting
> and raster graphics output, it supports scalable vector graphics (SVG)
> as well. (SVG output is trivial to do in Lisp but there is no
> high-level SVG definition language - this could potentially be another
> component in your offering). One of the benefits of basing it on
> XSL-FO would be that your input definition would be in a
> standards-based format and that might encourage a broader acceptance
> for your product outside of the Lisp community. Another benefit would
> be that there are validation test suites available for it which would
> make your development job a lot easier.

Well, this can be another high level syntax. But for now I'm working on the
intermediate representation and this one will be sexpr based.
As for users outside the Lisp comunity, I didn't tought about it but I have
been told it could be of interest. In that case it will be important to have
front-ends for several representations.

> Good luck with the project.

:)

Marc
From: Arthur Lemmens
Subject: Re: cl-typesetting
Date: 
Message-ID: <3F70278C.F6A666FB@xs4all.nl>
Marc Battyani wrote:
> 
> I'm currently working on cl-typesetting, a CL typesetting engine based on
> cl-pdf.

Great idea!

> I want a Lispy text representation similar to the one used by the zillions of
> HTML generation macros already there (including mine). So that it can very
> easily be manipulated and extended in Lisp while being user friendly enough.
> Thought as I'm not a typography expert I would like to know if it is a good
> enough representation or if I will hit some limitation in the future.

Have you looked at the DSSSL standard (Document Style Semantics and 
Specification Language)? I've never used it myself and I'm not a typography
expert either, but if I were to design a Lisp-based document typesetting 
system, I would take a very serious look at DSSSL first. In a way, DSSSL is 
like the Common Lisp of document typesetting languages: it includes so much
stuff that most people seem to have decided it's too complicated for them
and have settled for something simpler.

Here's a quote from the introductory chapter:

>  DSSSL defines the semantics, syntax, and processing model of two languages 
>  for the specification of document processing:
>
>  a) The transformation language for transforming SGML documents [...]
>
>  b) The style language, where the result is achieved by applying a set of 
>     formatting characteristics to portions of the data, and the specification 
>     is, therefore, as precise as the application requires, leaving some 
>     formatting decisions, such as line-end and column-end decisions, to the 
>     composition and layout process.
>
>  The DSSSL style language is intended to be used in a wide variety of 
>  environments with typographic requirements ranging from simple single-
>  column layouts to complex multiplecolumn layouts.

The "transformation language" is a slightly modified version of Scheme. Since
you already have CL, you can skip that part. But the "style language" (defined
in chapter 12 of the standard) includes lots of constructs for document 
formatting and layouting. Here's another quote from the standard to give you 
an idea of the kind of things they've thought pretty hard about:

>  The following flow object classes are provided in this International 
>   Standard:
>     Sequence flow object class
>     Display-group flow object class
>     Simple-page-sequence flow object class
>     Page-sequence flow object class
>     Column-set-sequence flow object class
>     Paragraph flow object class
>     Paragraph-break flow object class
>     Line-field flow object class
>     Sideline flow object class
>     Anchor flow object class
>     Character flow object class
>     Leader flow object class
>     Embedded-text flow object class
>     Rule flow object class
>     External-graphic flow object class
>     Included-container-area flow object class
>     Score flow object class
>     Box flow object class
>     Side-by-side flow object class
>     Glyph-annotation flow object class
>     Alignment-point flow object class
>     Aligned-column flow object class
>     Multi-line-inline-note flow object class
>     Emphasizing-mark flow object class
>     Flow object classes for mathematical formulae
>     Flow object classes for tables
>     Flow object classes for online display

You use exactly the same kind of syntax as I've been using in my own 
HTML-in-Lisp library for the last couple of years, by the way. 

Regards,

Arthur Lemmens
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkpn4h$chq@library2.airnews.net>
"Arthur Lemmens" <········@xs4all.nl> wrote
>
> Have you looked at the DSSSL standard (Document Style Semantics and
> Specification Language)? I've never used it myself and I'm not a typography
> expert either, but if I were to design a Lisp-based document typesetting
> system, I would take a very serious look at DSSSL first. In a way, DSSSL is
> like the Common Lisp of document typesetting languages: it includes so much
> stuff that most people seem to have decided it's too complicated for them
> and have settled for something simpler.

I will download the standard and look at it.
I don't want the XML representation but there could be some interesting ideas
in it.

...
> You use exactly the same kind of syntax as I've been using in my own
> HTML-in-Lisp library for the last couple of years, by the way.

This is what I stated in the first post: The HTML in lisp representation has
proved to be extremely handy and powerful.

Marc
From: Gisle Sælensminde
Subject: Re: cl-typesetting
Date: 
Message-ID: <slrnbn0r02.6g7.gisle@kaktus.ii.uib.no>
In article <··········@library2.airnews.net>, Marc Battyani wrote:
> 
> "Arthur Lemmens" <········@xs4all.nl> wrote
>>
>> Have you looked at the DSSSL standard (Document Style Semantics and
>> Specification Language)? I've never used it myself and I'm not a typography
<snip>
> I will download the standard and look at it.
> I don't want the XML representation but there could be some interesting ideas
> in it.
> 

I think a previous version of this standard was based on s-expr syntax, but
I have not looked too deep into it.

--
Gisle S�lensminde
Computational biology unit, University of Bergen, Norway
Email: ·····@cbu.uib.no
From: Peter Seibel
Subject: Re: cl-typesetting
Date: 
Message-ID: <m3fzinij58.fsf@javamonkey.com>
Gisle S�lensminde <·····@kaktus.ii.uib.no> writes:

> In article <··········@library2.airnews.net>, Marc Battyani wrote:
> > 
> > "Arthur Lemmens" <········@xs4all.nl> wrote
> >>
> >> Have you looked at the DSSSL standard (Document Style Semantics and
> >> Specification Language)? I've never used it myself and I'm not a typography
> <snip>
> > I will download the standard and look at it.
> > I don't want the XML representation but there could be some interesting ideas
> > in it.
> > 
> 
> I think a previous version of this standard was based on s-expr syntax, but
> I have not looked too deep into it.

DSSSL, as I understand it, is based on Scheme. The SGML folks realized
that sometimes you really just want a programming language. It's only
in the XML world that the "style" language (XSL) is also represented
in the markup languge (XML, in this case.)

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Paul Stanley
Subject: Re: cl-typesetting
Date: 
Message-ID: <4978ee1b.0309240516.1de49515@posting.google.com>
"Marc Battyani" <·············@fractalconcept.com> wrote in message news:<··········@library2.airnews.net>...

> I'm currently working on cl-typesetting, a CL typesetting engine based on
> cl-pdf.

I think this is an excellent idea, especially with the refinements
which appear later in this thread. As I understand it you are
specifically looking to produce a typesetting *programming* language,
probably not to be used (at least by most people) directly, but to be
used as a base for other things. That seems admirable; I've often
thought about it myself (but would be completely incompetent to
attempt it).

If that is the purpose, I would pay especially close attention to some
of the following issues:

1. Quality of the output. This is crucial. The amazing thing about
TeX, for instance, is that despite its antiquity is that it still
produces output that is typographically better even than most
publishing software, let alone word-processors and other software.
Things like automatic insertion of ligatures, optical scaling of
fonts, a hyphenation algorithm that works paragraph-by-paragraph
rather than line by line and so on are only now beginning to appear in
products such as InDesign.

2. Automated operation. One of the things people need is software that
will be guaranteed to produce reasonable results without human
intervention. On this, TeX rather fails: for any long document it will
almost always require human intervention to fix up some over-long
lines, adjust page-breaking and so on. One may compare, for instance,
its holistic line-breaking algorithm with its 'page-by-page'
page-breaking algorithm. (It also follows that this is an area where
anything which uses TeX as its typesetter will probably fail.)

3. Ease of maintenance and extension. Consider, for example, the
enormous efforts one has to go to to install new fonts in TeX. Yet
some of the functions you need (such as pulling ff, ffi and ffl
ligatures out of an expert set if one is available, using real
small-caps if possible and faking them if not and so forth) are far
from trivial. Something akin to virtual fonts is needed, but
automating them is tricky. Font management is one of the key issues to
be addressed.

4. Built-in (or easily configurable) support for some common but
tricky functions such as cross-referencing to pages, marginal notes,
letter-spacing, dropped capitals and so on. On the other hand, it is
probably worth separating things which require knowledge of the
typeset document (such as page references or indexes, which cannot be
constructed without knowledge of where each page breaks) with things
which do not (such as references to a bibliography, which can be
constructed without reference to how the document is typeset, and
should probably not be handled by a type-setting program, or at least
not one of the level you are working on.

5. The area where these points all come together most strikingly is in
relation to typesetting maths. I think you will need to give serious
thought to whether you are going to attempt to handle that, and if so
how.

-- 
Paul Stanley
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkumj9$o0g@library1.airnews.net>
"Paul Stanley" <········@essexcourt-chambers.co.uk> wrote
> "Marc Battyani" <·············@fractalconcept.com> wrote in message

> > I'm currently working on cl-typesetting, a CL typesetting engine based on
> > cl-pdf.
>
> I think this is an excellent idea, especially with the refinements
> which appear later in this thread. As I understand it you are
> specifically looking to produce a typesetting *programming* language,
> probably not to be used (at least by most people) directly, but to be
> used as a base for other things. That seems admirable; I've often
> thought about it myself (but would be completely incompetent to
> attempt it).
>
> If that is the purpose, I would pay especially close attention to some
> of the following issues:
>
> 1. Quality of the output. This is crucial. The amazing thing about
> TeX, for instance, is that despite its antiquity is that it still
> produces output that is typographically better even than most
> publishing software, let alone word-processors and other software.
> Things like automatic insertion of ligatures, optical scaling of
> fonts, a hyphenation algorithm that works paragraph-by-paragraph
> rather than line by line and so on are only now beginning to appear in
> products such as InDesign.

I agree with you. Output quality is very important. I've already started to
add micro-typography features like using a small amount of variability
between the letters. Next I will also add a small amount of glyph stretching
as well.

> 2. Automated operation. One of the things people need is software that
> will be guaranteed to produce reasonable results without human
> intervention. On this, TeX rather fails: for any long document it will
> almost always require human intervention to fix up some over-long
> lines, adjust page-breaking and so on. One may compare, for instance,
> its holistic line-breaking algorithm with its 'page-by-page'
> page-breaking algorithm. (It also follows that this is an area where
> anything which uses TeX as its typesetter will probably fail.)

This is the main reason for which I'm doing this. I want to be able to
automatically generate, manipulate and process texts and documents (like I
already do in my web framework) before typesetting it. And for this Common
Lisp is great.

> 3. Ease of maintenance and extension. Consider, for example, the
> enormous efforts one has to go to to install new fonts in TeX. Yet
> some of the functions you need (such as pulling ff, ffi and ffl
> ligatures out of an expert set if one is available, using real
> small-caps if possible and faking them if not and so forth) are far
> from trivial. Something akin to virtual fonts is needed, but
> automating them is tricky. Font management is one of the key issues to
> be addressed.

Ease of extension is built-in. Ligatures are easy if they are specified in
the font data. Fonts can be a problem though I think every font can be made
pdf compatible.

> 4. Built-in (or easily configurable) support for some common but
> tricky functions such as cross-referencing to pages, marginal notes,
> letter-spacing, dropped capitals and so on. On the other hand, it is
> probably worth separating things which require knowledge of the
> typeset document (such as page references or indexes, which cannot be
> constructed without knowledge of where each page breaks) with things
> which do not (such as references to a bibliography, which can be
> constructed without reference to how the document is typeset, and
> should probably not be handled by a type-setting program, or at least
> not one of the level you are working on.

Only the page refs are a problem as when you write them back in the document
you could change the pagination. The other ones are not a problem

> 5. The area where these points all come together most strikingly is in
> relation to typesetting maths. I think you will need to give serious
> thought to whether you are going to attempt to handle that, and if so
> how.

For now math mode is not planned....

Marc
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <o3Dcb.13782$lZ6.4054471@twister.nyc.rr.com>
Marc Battyani wrote:
>>5. The area where these points all come together most strikingly is in
>>relation to typesetting maths. I think you will need to give serious
>>thought to whether you are going to attempt to handle that, and if so
>>how.
> 
> 
> For now math mode is not planned....
> 

Hmmm.... Cells were invented to solve math layout. Hmmmm...

:)

kenny
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkv2nv$m7o@library1.airnews.net>
"Kenny Tilton" <·······@nyc.rr.com> wrote
>
> Marc Battyani wrote:
> >>5. The area where these points all come together most strikingly is in
> >>relation to typesetting maths. I think you will need to give serious
> >>thought to whether you are going to attempt to handle that, and if so
> >>how.
> >
> > For now math mode is not planned....
>
> Hmmm.... Cells were invented to solve math layout. Hmmmm...

Thanks for offering yourself as a volunteer! ;-)

Marc
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <JqEcb.14378$lZ6.4071851@twister.nyc.rr.com>
Marc Battyani wrote:

> "Kenny Tilton" <·······@nyc.rr.com> wrote
> 
>>Marc Battyani wrote:
>>
>>>>5. The area where these points all come together most strikingly is in
>>>>relation to typesetting maths. I think you will need to give serious
>>>>thought to whether you are going to attempt to handle that, and if so
>>>>how.
>>>
>>>For now math mode is not planned....
>>
>>Hmmm.... Cells were invented to solve math layout. Hmmmm...
> 
> 
> Thanks for offering yourself as a volunteer! ;-)

Well thank /you/ for returning the favor and pitching in on RoboCells! 
:) Tell you what, after ILC2003 when Kenny University opens its virtual 
doors for distance learning, I'll look at basing lessons on a cl-pdf 
"math mode" project as well as RoboCells. Maybe diff students will have 
diff interests.

You know, this is Most Excellent. I never just learn a language, I 
always do it in the context of a project. When I tackled Prolog, I 
picked one out of the air, getting it to bid at bridge according to a 
bidding cheat sheet someone gave me. I had not thought of it, but of 
course I should teach the way I learn.

The other good thing is that this puts the burden on the student to be 
active, not the classic passive receptacle model of formal education. If 
students are not active, there is nothing I can do that will somehow 
leave them knowing Lisp.

kenny
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bkvcod$jit@library1.airnews.net>
"Kenny Tilton" <·······@nyc.rr.com> wrote in
>
> Marc Battyani wrote:
> > "Kenny Tilton" <·······@nyc.rr.com> wrote
> >>Marc Battyani wrote:
...
> >>>For now math mode is not planned....
> >>Hmmm.... Cells were invented to solve math layout. Hmmmm...
> > Thanks for offering yourself as a volunteer! ;-)

> Well thank /you/ for returning the favor and pitching in on RoboCells!
> :) Tell you what, after ILC2003 when Kenny University opens its virtual
> doors for distance learning, I'll look at basing lessons on a cl-pdf
> "math mode" project as well as RoboCells. Maybe diff students will have
> diff interests.

May be I will look at RoboCells but I'm already making a real one (a robot).
With FPGA, DC motors, PWM controllers, etc.. to play with my kids. This will
be programmable in Lisp of course ;-)

> You know, this is Most Excellent. I never just learn a language, I
> always do it in the context of a project. When I tackled Prolog, I
> picked one out of the air, getting it to bid at bridge according to a
> bidding cheat sheet someone gave me. I had not thought of it, but of
> course I should teach the way I learn.

Seems a good idea.

> The other good thing is that this puts the burden on the student to be
> active, not the classic passive receptacle model of formal education. If
> students are not active, there is nothing I can do that will somehow
> leave them knowing Lisp.

The key here is motivation.

Marc
From: Paul Stanley
Subject: Re: cl-typesetting
Date: 
Message-ID: <4978ee1b.0309260322.328431cf@posting.google.com>
"Marc Battyani" <·············@fractalconcept.com> wrote in message news:<··········@library1.airnews.net>...
> "Paul Stanley" <········@essexcourt-chambers.co.uk> wrote
> > "Marc Battyani" <·············@fractalconcept.com> wrote in message
>  
> Ease of extension is built-in. Ligatures are easy if they are specified in
> the font data. Fonts can be a problem though I think every font can be made
> pdf compatible.

It depends what you mean by "the font data". The trouble with
ligatures is that some of them (especially ff, ffl and ffi ligatures)
are often specified in the font data for an expert set; the system
needs to be intelligent enough to pull data from more than one font
file, i.e. to appreciate that "font" is not the same as "font file".
(The same goes for oldstyle figures.) At least until everything is
open type (and assuming you can support that). I think ligatures cause
hyphenation difficulties, too, which are quite tricky.
 
> Only the page refs are a problem as when you write them back in the document
> you could change the pagination. The other ones are not a problem.

I think page refs are a problem that really has to be solved (else how
to do things like tables of contents?). One idea would be to "dummy"
set any paragraph containing a reference using a "guess" about the
length of the reference. Since references are normally in figures,
which have pretty standard width in most fonts (i.e. 877 will be much
the same width as 234, so it's really only the *number* of figures
that matters), some fairly simple heuristics would enable a moderately
intelligent guess to be made. For instance, if one knew (a) the
average words to the page from processing, say 10 pages (or all the
pages to this point) and (b) how many "words" into the document the
reference was pointing to, one would normally be able to guess whether
it should be a 1, 2 or 3 figure reference. One would then have to go
back to reprocess paragraphs which had references in once pages had
been broken, but using a line-breaking and justification algorithm
that *guaranteed* the same number of lines would be produced, even if
that meant stretching or shrinking glue badly. Those lines could then
simply be replaced without mucking everything else up. The worst one
would then end up with is a single poor paragraph, and I think that
would be pretty rare, especially if one's "guessing" heuristic was as
good as it could be made.

This would have the advantage (a) that one would not have to reprocess
the entire document and (b) that one would be guaranteed that
inserting page references did not recursively change them.

(There would be similar problems if one wanted footnote numbers to
restart on every page ... but this seems to me to be a less serious
issue.)
From: Ray Blaak
Subject: Re: cl-typesetting
Date: 
Message-ID: <uu16z317w.fsf@STRIPCAPStelus.net>
········@essexcourt-chambers.co.uk (Paul Stanley) writes:
> I think page refs are a problem that really has to be solved (else how
> to do things like tables of contents?). One idea would be to "dummy"
> set any paragraph containing a reference using a "guess" about the
> length of the reference. Since references are normally in figures,
> which have pretty standard width in most fonts (i.e. 877 will be much
> the same width as 234, so it's really only the *number* of figures
> that matters), some fairly simple heuristics would enable a moderately
> intelligent guess to be made.

How does TeX/LaTex do it, and why couldn't cl-typesetting to the same?

Or, use the heuristical approach, and only if that fails do full reprocessing
until things settle.

-- 
Cheers,                                        The Rhythm is around me,
                                               The Rhythm has control.
Ray Blaak                                      The Rhythm is inside me,
········@STRIPCAPStelus.net                    The Rhythm has my soul.
From: Brian Downing
Subject: Re: cl-typesetting
Date: 
Message-ID: <jm%cb.595063$uu5.97055@sccrnsc04>
In article <·············@STRIPCAPStelus.net>,
Ray Blaak  <········@STRIPCAPStelus.net> wrote:
> How does TeX/LaTex do it, and why couldn't cl-typesetting to the same?

It indexes the page numbers for each reference when you run the document, and
saves them away in a separate file.  So you basically have to keep running the
document through until the page numbers stabilize.

-bcd
-- 
*** Brian Downing <bdowning at lavos dot net> 
From: ·········@cern.ch
Subject: Re: cl-typesetting
Date: 
Message-ID: <yzopthnh11h.fsf@localhost.localdomain>
Ray Blaak <········@STRIPCAPStelus.net> writes:

(on typesetting forward-references)

> How does TeX/LaTex do it, and why couldn't cl-typesetting to the same?
> 
> Or, use the heuristical approach, and only if that fails do full reprocessing
> until things settle.

...which is more-or-less what LaTeX does - that's what the aux-files
are all about.

Ole
From: Marc Battyani
Subject: Re: cl-typesetting
Date: 
Message-ID: <bl7au0$8j2@library2.airnews.net>
"Paul Stanley" <········@essexcourt-chambers.co.uk> wrote
> I think page refs are a problem that really has to be solved (else how
> to do things like tables of contents?). One idea would be to "dummy"
> set any paragraph containing a reference using a "guess" about the
> length of the reference. Since references are normally in figures,
> which have pretty standard width in most fonts (i.e. 877 will be much
> the same width as 234, so it's really only the *number* of figures
> that matters), some fairly simple heuristics would enable a moderately
> intelligent guess to be made. For instance, if one knew (a) the
> average words to the page from processing, say 10 pages (or all the
> pages to this point) and (b) how many "words" into the document the
> reference was pointing to, one would normally be able to guess whether
> it should be a 1, 2 or 3 figure reference. One would then have to go
> back to reprocess paragraphs which had references in once pages had
> been broken, but using a line-breaking and justification algorithm
> that *guaranteed* the same number of lines would be produced, even if
> that meant stretching or shrinking glue badly. Those lines could then
> simply be replaced without mucking everything else up. The worst one
> would then end up with is a single poor paragraph, and I think that
> would be pretty rare, especially if one's "guessing" heuristic was as
> good as it could be made.
>
> This would have the advantage (a) that one would not have to reprocess
> the entire document and (b) that one would be guaranteed that
> inserting page references did not recursively change them.

It's much more easier to deal with excess space than with not enough space.
Too much excess spacing would only give an ugly layout while not enough
spacing could mean an unreadable result.

I will think about it.

Marc
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <87n0csylfn.fsf@g.mccaughan.ntlworld.com>
Marc Battyani wrote:

> For now math mode is not planned....

That's a pity, for two reasons.

The selfish reason: I use TeX when I have to write mathematics,
because everything else is too ugly or too hard to use or otherwise
unbearable. It would be nice to have something that can replace TeX.

The less selfish reason: Mathematical typesetting is TeX's
killer feature, the one really compelling reason for using it.
If we can use Common Lisp to make a genuinely better TeX --
better at everything, including mathematics -- then that
could be an effective way of getting some good publicity
for Lisp.

Of course, the trouble is that TeX is good enough that you'll
have trouble making something so much better as to compel
people to switch...

-- 
Gareth McCaughan
.sig under construc
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <2BYcb.46002$u67.7739@twister.nyc.rr.com>
Gareth McCaughan wrote:

> Marc Battyani wrote:
> 
> 
>>For now math mode is not planned....
> 
> 
> That's a pity, for two reasons.
> 
> The selfish reason: I use TeX when I have to write mathematics,
> because everything else is too ugly or too hard to use or otherwise
> unbearable. It would be nice to have something that can replace TeX.
> 
> The less selfish reason: Mathematical typesetting is TeX's
> killer feature, the one really compelling reason for using it.
> If we can use Common Lisp to make a genuinely better TeX --
> better at everything, including mathematics -- then that
> could be an effective way of getting some good publicity
> for Lisp.

Yep. We should all pitch in to help Marc with cl-pdf. That or stop 
whining about CL being unpopular.

I am going to do math mode, if anyone wants to help with that. It has 
occurred to me that the best way to market Cells is simply to penetrate 
interesting projects and insert its mitochondrial DNA. First RoboCup, 
now cl-TeX.

Are we doing Unicode, so we can get to all those Greek symbols etc? Do 
we have the Bodoni font? I think that is the font used in math. What 
/are/ the layout rules for math, ie, where should things go to look pretty?

I plan to do a WYSIWYG math editor (my tenth, I think). I have win32 
(<spit>) under control. Who wants to cover CLX? Or we could use 
FreeGlut/OpenGL/Mesa, tho I do not yet know if OpenGL can do decent 
text. Tell you, what. I'll use FreeGlut/OpenGL on win32, maybe someone 
can see if that dawg will hunt on Linux or OS X.

btw, FreeGlut is being picked up by RedHat after they belatedly realized 
the original Glut was encumbered by a non-open license. So that crew is 
all fired up and working on a new release, and should be getting a burst 
of energy from the Redhat adoption.


> 
> Of course, the trouble is that TeX is good enough that you'll
> have trouble making something so much better as to compel
> people to switch...
> 

Point of information, because I really do not know: I thought TeX could 
be quite frustrating to get right. I know the output roolz, but isn't it 
a pain to use? I mean, I thought I heard that somewhere. If so, would 
that give us a hook to get people to switch?

kenny
From: Eric Daniel
Subject: Re: cl-typesetting
Date: 
Message-ID: <3f746d51$1_3@corp.newsgroups.com>
In article <····················@twister.nyc.rr.com>, Kenny Tilton wrote:
>  
>  Yep. We should all pitch in to help Marc with cl-pdf. That or stop 
>  whining about CL being unpopular.
>  
>  I am going to do math mode, if anyone wants to help with that. It has 
>  occurred to me that the best way to market Cells is simply to penetrate 
>  interesting projects and insert its mitochondrial DNA. First RoboCup, 
>  now cl-TeX.
>  
>  Are we doing Unicode, so we can get to all those Greek symbols etc? Do 
>  we have the Bodoni font? I think that is the font used in math. What 
>  /are/ the layout rules for math, ie, where should things go to look pretty?

Two sources you may want to consider:
  1) TeX, after all, that's why it became so famous, so there must be
something to learn from it. However I don't think the TeXbook goes
into much details about the inner workings of math mode

  2) the user manual for Lout (google for it) has a good explanation
of its math typesetting algorithms.

[...]

>  Point of information, because I really do not know: I thought TeX could 
>  be quite frustrating to get right. I know the output roolz, but isn't it 
>  a pain to use? I mean, I thought I heard that somewhere. If so, would 
>  that give us a hook to get people to switch?
>  
>  kenny
>  

As soon as you want to do things Knuth hadn't thought of, you hit two
obstacles. First, TeX (the language) is terrible to program with, I've
never seen a language like this before (maybe I'm too young?). Second,
there are things TeX (the program) doesn't let TeX (the language) access:
for example, once a box is built, you can no longer see the characters
inside (short of dumping it to a file and re-parsing the file).

Where cl-typesetting really, really rules, is that its implementation
language is the same as its control language, so the document author
has access to everything the program knows. Will that be enough for people
to switch? I don't know. But I tell you, I'll be the first in line
to try it when Marc releases it :-)

Eric Daniel


-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 100,000 Newsgroups - 19 Different Servers! =-----
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <8765jfz1gk.fsf@g.mccaughan.ntlworld.com>
Eric Daniel wrote:

> Two sources you may want to consider:
>   1) TeX, after all, that's why it became so famous, so there must be
> something to learn from it. However I don't think the TeXbook goes
> into much details about the inner workings of math mode
> 
>   2) the user manual for Lout (google for it) has a good explanation
> of its math typesetting algorithms.

Lout's mathematics typesetting is -- or, at least, was when
I last looked at it -- way inferior to TeX's.

> Where cl-typesetting really, really rules, is that its implementation
> language is the same as its control language, so the document author
> has access to everything the program knows. Will that be enough for people
> to switch? I don't know. But I tell you, I'll be the first in line
> to try it when Marc releases it :-)

I would advise you not to count on being first.

-- 
Gareth McCaughan
.sig under construc
From: Daniel Barlow
Subject: Re: cl-typesetting
Date: 
Message-ID: <87u16ztwb3.fsf@noetbook.telent.net>
Kenny Tilton <·······@nyc.rr.com> writes:

> Point of information, because I really do not know: I thought TeX
> could be quite frustrating to get right. I know the output roolz, but
> isn't it a pain to use? I mean, I thought I heard that somewhere. If
> so, would that give us a hook to get people to switch?

I wouldn't put TeX in the hands of the average secretary (unless he
expressed a particular wish), but at least in its LaTeX guise it's
really pretty inoffensive for anyone who can drive a text editor.
Until you want to do something complicated, but the chances are it was
either something that yer average word processor can't do at all, or a
bad idea anyway.  Provided you accept Knuth as your arbiter of good
ideas (personal saviour is optional), this is OK.  If you rebel and
insist on asking for double-spaced output, why are you using a
typesetting program anyway?


-dan

-- 

   http://www.cliki.net/ - Link farm for free CL-on-Unix resources 
From: Edi Weitz
Subject: Re: cl-typesetting
Date: 
Message-ID: <874qyz5z35.fsf@bird.agharta.de>
On Fri, 26 Sep 2003 16:16:00 +0100, Daniel Barlow <···@telent.net> wrote:

> I wouldn't put TeX in the hands of the average secretary

At the university where I was working the secretaries used TeX (LaTeX,
that is) - at least in the mathematical department. These were normal
secretaries, not mathematicians.

Well, this was a couple of years ago. I guess they've "upgraded" to
Word now...

Edi.
From: Matthias
Subject: Re: cl-typesetting
Date: 
Message-ID: <36wllsbzblf.fsf@chagall.ti.uni-mannheim.de>
Edi Weitz <···@agharta.de> writes:

> On Fri, 26 Sep 2003 16:16:00 +0100, Daniel Barlow <···@telent.net> wrote:
> 
> > I wouldn't put TeX in the hands of the average secretary
> 
> At the university where I was working the secretaries used TeX (LaTeX,
> that is) - at least in the mathematical department. These were normal
> secretaries, not mathematicians.
> 
> Well, this was a couple of years ago. I guess they've "upgraded" to
> Word now...

I don't think so.  At least not in math and CS.

BTW: double-spaced output in LaTeX (a popular macro package to TeX) is
as simple as

 \renewcommand{\baselinestretch}{2.0}

or

 \usepackage{doublespace}

on some systems.  Students typically take 1-3 days until they write
decent documents in LaTeX.  Admitted, it's hard to do something
strange and TeX programming probably is also hard.  (All the packages
I ever needed already existed.)  But using LaTeX is not.
From: Fred Gilham
Subject: Re: cl-typesetting
Date: 
Message-ID: <u77k3v9c97.fsf@snapdragon.csl.sri.com>
> I wouldn't put TeX in the hands of the average secretary (unless he
> expressed a particular wish), but at least in its LaTeX guise it's
> really pretty inoffensive for anyone who can drive a text editor.

We had some secretaries who wrote TeX macros back in the late '80s and
early '90s.

They didn't know they were programming....they probably would have
wanted more money.... :-)

Actually they were really sharp.  None of them work here any more.

-- 
Fred Gilham                                        ······@csl.sri.com
Propaganda makes up our mind for us, but in such a way that it leaves
us the sense of pride and satisfaction of men who have made up their
own minds. And in the last analysis, propaganda achieves this effect
because we want it to. This is one of the few real pleasures left to
modern man: this illusion that he is thinking for himself when, in
fact, someone else is doing his thinking for him.  - Thomas Merton
From: Peter Seibel
Subject: Re: cl-typesetting
Date: 
Message-ID: <m3oex7cyo0.fsf@javamonkey.com>
Kenny Tilton <·······@nyc.rr.com> writes:

> Gareth McCaughan wrote:
> 
> > Marc Battyani wrote:
> >
> >>For now math mode is not planned....
> > That's a pity, for two reasons.
> > The selfish reason: I use TeX when I have to write mathematics,
> > because everything else is too ugly or too hard to use or otherwise
> > unbearable. It would be nice to have something that can replace TeX.
> > The less selfish reason: Mathematical typesetting is TeX's
> > killer feature, the one really compelling reason for using it.
> > If we can use Common Lisp to make a genuinely better TeX --
> > better at everything, including mathematics -- then that
> > could be an effective way of getting some good publicity
> > for Lisp.
> 
> Yep. We should all pitch in to help Marc with cl-pdf. That or stop
> whining about CL being unpopular.
> 
> I am going to do math mode, if anyone wants to help with that. It has
> occurred to me that the best way to market Cells is simply to
> penetrate interesting projects and insert its mitochondrial DNA. First
> RoboCup, now cl-TeX.
> 
> Are we doing Unicode, so we can get to all those Greek symbols etc? Do
> we have the Bodoni font? I think that is the font used in math. What
> /are/ the layout rules for math, ie, where should things go to look
> pretty?
> 
> I plan to do a WYSIWYG math editor (my tenth, I think). I have win32
> (<spit>) under control. Who wants to cover CLX? Or we could use
> FreeGlut/OpenGL/Mesa, tho I do not yet know if OpenGL can do decent
> text. Tell you, what. I'll use FreeGlut/OpenGL on win32, maybe someone
> can see if that dawg will hunt on Linux or OS X.
> 
> btw, FreeGlut is being picked up by RedHat after they belatedly
> realized the original Glut was encumbered by a non-open license. So
> that crew is all fired up and working on a new release, and should be
> getting a burst of energy from the Redhat adoption.
> 
> 
> > Of course, the trouble is that TeX is good enough that you'll
> > have trouble making something so much better as to compel
> > people to switch...
> >
> 
> Point of information, because I really do not know: I thought TeX
> could be quite frustrating to get right. I know the output roolz, but
> isn't it a pain to use? I mean, I thought I heard that somewhere. If
> so, would that give us a hook to get people to switch?

I suspect that in a lot of the places where TeX/LaTex is currently
being used, it is "the way". And the people who use it have probably
already gone through the pain of making it work for them.

But it seems that we/you/one could do an Innovator's Dillema[1] on TeX
by providing a system that initially serves a much smaller purpose but
using superior technology and eventually grows to be capable of doing
everything TeX can can do plus more.

Plus this strategy seems more fun than setting out with the immediate
goal of trying to convert existing TeX users. You're immediately
solving a real problem for real people (folks who want to produce nice
looking documents but don't want to wrestle with TeX) and don't have
to spend any time wailing and gnashing one's teeth about why all those
TeX users don't recognize the technical superiority of cl-typesetting
and switch. (Sound familiar. ;-))

I think the big hook for a new typesetting system would be the one
another poster mentioned (and which I outlined in my post with the
ASCII art): if I can write a single document in any old text editor
that produces beautiful looking printed output, good looking HTML, and
something that I can send to someone who wants a Word file, then I'm
wining big time. (And if the intermediate format is rightous, then
folks who like WYSIWYG tools can write fancier editors that ultimately
generate the same format. Wheee!)

-Peter

[1] _Innovator's Dillema_ by Clayton Christensen, one of those
business books that was quite trendy for a while among the PHB crowd
but worth reading despite that. The short version is big companies
have to keep giving their customers what they want in order to keep up
their revenue. And they have no incentive to invest in "inferior"
newer technologies that don't immediately solve their existing
customer's problems because even if they can sell a few units of the
new technology, the revenue is in the noise compared to their existing
revenue stream. So new companies start up and build products based on
the new technology and eat away from the bottom--the customers whose
needs *can* be met with the new technology and who weren't really
being served by the existing companies. As the new companies start
making money, they invest in the development of the new technology,
improving it so it begins to compete with the low-end products of the
old-technology companies. The old-technology companies then focus
their efforts on the customers that still need the capabilities that
only their products can provide. But the new companies keep eating
away and pretty soon the old-technology either dies or is relegated to
a niche.

An example of this not from the computer industry is what happened in
the earth-moving equipment industry. Back in the day, excavators all
used cables, not pneumatics. The name of the game was to make
excavators that could scoop up a *lot* of earth and pneumatics were
too weak. But the guys working on pneumatic excavators discovered that
they could make backhoes that were sufficient for digging ditches and
cheap enough that farmers could buy them. Before backhoes the farmers
just dug them with shovels, so the pneumatic exacavator was a step up
for them, even if a contractor digging out the foundation of a
building would think it's a toy. But pneumatics had other
advantages--they were cheaper and safer (no danger of a cable
snapping). So the pnematics guys made money selling to farmers and
eventually improved the capacity of their excavators to the point that
they could handle most of the jobs previously only cable-based
excavators could. These days there are only a few cable-based
excavators companies still around and they have been forced to the far
right edge of the "scoop capacity" curve, making massive cable-based
excavators for strip mining, while pneumatics are used for pretty much
everything else.

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Daniel Barlow
Subject: Re: cl-typesetting
Date: 
Message-ID: <87k77vtrdn.fsf@noetbook.telent.net>
Peter Seibel <·····@javamonkey.com> writes:

> [1] _Innovator's Dillema_ [...] big companies
> have to keep giving their customers what they want in order to keep up
> their revenue. And they have no incentive to invest in "inferior"
> newer technologies that don't immediately solve their existing
> customer's problems because even if they can sell a few units of the
> new technology, the revenue is in the noise compared to their existing
> revenue stream. So new companies start up and build products based on
> the new technology and eat away from the bottom--the customers whose
> needs *can* be met with the new technology and who weren't really
> being served by the existing companies. As the new companies start

I'm pretty sure this was described in Crossing the Chasm (target a
niche that's underserved by existing solutions, dominate it, spread)
and it's also the basis of Worse is Better (a sun 3 did half the job
that the lispm did, at a tenth the cost[*]).  Wonder who described it
first - presumably the Lockheed Martin buys would take it for their
own as part of the 'Skunkworks' concept as well.


-dan

[*] I made these numbers up.  No, i'm not considering TCO

-- 

   http://www.cliki.net/ - Link farm for free CL-on-Unix resources 
From: Peter Seibel
Subject: Re: cl-typesetting
Date: 
Message-ID: <m3fzijcvbx.fsf@javamonkey.com>
Daniel Barlow <···@telent.net> writes:

> Peter Seibel <·····@javamonkey.com> writes:
> 
> > [1] _Innovator's Dillema_ [...] big companies
> > have to keep giving their customers what they want in order to keep up
> > their revenue. And they have no incentive to invest in "inferior"
> > newer technologies that don't immediately solve their existing
> > customer's problems because even if they can sell a few units of the
> > new technology, the revenue is in the noise compared to their existing
> > revenue stream. So new companies start up and build products based on
> > the new technology and eat away from the bottom--the customers whose
> > needs *can* be met with the new technology and who weren't really
> > being served by the existing companies. As the new companies start
> 
> I'm pretty sure this was described in Crossing the Chasm (target a
> niche that's underserved by existing solutions, dominate it, spread)
> and it's also the basis of Worse is Better (a sun 3 did half the job
> that the lispm did, at a tenth the cost[*]).  Wonder who described it
> first - presumably the Lockheed Martin buys would take it for their
> own as part of the 'Skunkworks' concept as well.

Yeah. I think the insight that Christensen may have added is an
examination of why it's *so hard* for the existing companies to be the
ones to take advantage of the new "disruptive" technologies. (A
properly run skunkworks project can sometimes pull it off but it
requires--according to Christensen--a lot of careful attention to keep
it from getting swallowed up by the needs of the dominant business.)

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Paolo Amoroso
Subject: Re: cl-typesetting
Date: 
Message-ID: <87he2ywpge.fsf@plato.moon.paoloamoroso.it>
Daniel Barlow writes:

> that the lispm did, at a tenth the cost[*]).  Wonder who described it
> first - presumably the Lockheed Martin buys would take it for their
> own as part of the 'Skunkworks' concept as well.

Pay attention to how you use that word. Some time ago DDJ used it, and
they received a kind letter from the Lockheed guys.


Paolo
-- 
Paolo Amoroso <·······@mclink.it>
From: Paolo Amoroso
Subject: Re: cl-typesetting
Date: 
Message-ID: <87eky34c1r.fsf@plato.moon.paoloamoroso.it>
Kenny Tilton writes:

> we have the Bodoni font? I think that is the font used in math. What
> /are/ the layout rules for math, ie, where should things go to look
> pretty?

Knuth wrote a book series telling about his computers and typesetting
"sabbatical", but I can't remember the title right now. It should also
cover the layout rules for math.


Paolo
-- 
Paolo Amoroso <·······@mclink.it>
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <87k77uwk9d.fsf@g.mccaughan.ntlworld.com>
Paolo Amoroso wrote:

> Kenny Tilton writes:
> 
> > we have the Bodoni font? I think that is the font used in math. What
> > /are/ the layout rules for math, ie, where should things go to look
> > pretty?
> 
> Knuth wrote a book series telling about his computers and typesetting
> "sabbatical", but I can't remember the title right now. It should also
> cover the layout rules for math.

The series was collectively titled "Computers and typesetting",
oddly enough :-). The books were, in some order that I now forget
but might possibly be the following,
  - The TeXbook
  - The METAFONTbook
  - TeX: the program
  - METAFONT: the program
  - Computer Modern typefaces.

There's also a single volume called "Digital typography"
in CSLI's series of Knuth things, There's basically nothing
about typesetting mathematics in there, but there's a nice
paper about the line-breaking algorithm and some other
potentially interesting things.

-- 
Gareth McCaughan
.sig under construc
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <871xu3z0v6.fsf@g.mccaughan.ntlworld.com>
Kenny Tilton wrote:

> I am going to do math mode, if anyone wants to help with that. It has
> occurred to me that the best way to market Cells is simply to
> penetrate interesting projects and insert its mitochondrial DNA. First
> RoboCup, now cl-TeX.

I have this scary vision of 20 people getting involved
in CL-TYPESETTING and each adding their own pet project
as a dependency, with the result that the whole thing is
impossible to build anywhere and becomes a licensing
nightmare later on... How portable is Cells?

> Are we doing Unicode, so we can get to all those Greek symbols etc? Do
> we have the Bodoni font? I think that is the font used in math. What
> /are/ the layout rules for math, ie, where should things go to look
> pretty?

Um. If you have to ask that question, perhaps you shouldn't "do
math mode". :-)

You can do mathematics with whatever typefaces you want,
provided they provide the necessary characters. Lots of
mathematics gets typeset in Knuth's "Computer Modern"
faces because Knuth gave them a good supply of bits for
mathematical typesetting (extending brackets, different
sizes of summation signs, useful spacing information
that isn't present in normal fonts) and taught TeX how
to use them effectively. To do good mathematical typesetting
in some random font (e.g., *spit* Times Roman) you need
to supply some extra glyphs and all that other useful
information about spacings and sizes. This will need to
be done differently for every typeface family you use
for mathematical typesetting.

Some people have already done a lot of the work for
certain popular typefaces: google for "MathTime", IIRC,
to find out more.

> Point of information, because I really do not know: I thought TeX
> could be quite frustrating to get right. I know the output roolz, but
> isn't it a pain to use? I mean, I thought I heard that somewhere. If
> so, would that give us a hook to get people to switch?

Possibly, yes. Easy stuff is quite easy. Tweaking it to
make the output beautiful rather than merely good is
fiddly but not difficult. What's hard is getting any
precisely-defined typographical effect that Knuth (for TeX)
or <whoever> (for LaTeX) didn't already plan out for you.
The primitive operations you have available are a bit
funky, and if you want to say "please line up this thing
nested in these boxes with this other things nested in
those other boxes" then you have a world of pain ahead
of you. And, of course, the "programming" language is a
hideous macro-based thing; telling it what to do is a bit
like writing nested backquote expressions and a bit like
trying to get a Unix shell to expand the right stuff at
the right time. You do loops with tail recursion. And
so on. It hurts.

If Marc does a good job of exposing the right bits of
functionality, then I think CL-TYPESETTING has the prospect
of being much, much easier to program than TeX.

But most TeX users never do any programming beyond the
easy stuff (which will almost certainly not be easier,
or at least not much easier, in CL-TYPESETTING). And the
TeX users who really care have invested all the pain
and effort required to get good at it, and have the
mental scars to prove it. It is not always easy to
persuade someone to switch away from something they've
invested a lot of trouble learning...

-- 
Gareth McCaughan
.sig under construc
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <_I3db.48112$u67.4326@twister.nyc.rr.com>
Gareth McCaughan wrote:
> Kenny Tilton wrote:
> 
> 
>>I am going to do math mode, if anyone wants to help with that. It has
>>occurred to me that the best way to market Cells is simply to
>>penetrate interesting projects and insert its mitochondrial DNA. First
>>RoboCup, now cl-TeX.
> 
> 
> I have this scary vision of 20 people getting involved
> in CL-TYPESETTING and each adding their own pet project
> as a dependency, with the result that the whole thing is
> impossible to build anywhere and becomes a licensing
> nightmare later on... How portable is Cells?

What? You haven't installed Cells yet? :) Latest version (well, is on my 
hard drive) but the penultimate version is embedded in the DNA of RoboCells:

    http://sourceforge.net/projects/robocells/

Cells work on ACL, LW, MCL, CLisp, CormanCL, and CMUCL. Others just have 
not been tried. I'll gladly patch as necessary if they break somewhere.

btw, the upside of your scary scenario is that we'd have twenty people 
working on it. So far only Burdick has offered to help. Where is Tolton? 
I thought he was looking for a project. :)

> 
> 
>>Are we doing Unicode, so we can get to all those Greek symbols etc? Do
>>we have the Bodoni font? I think that is the font used in math. What
>>/are/ the layout rules for math, ie, where should things go to look
>>pretty?
> 
> 
> Um. If you have to ask that question, perhaps you shouldn't "do
> math mode". :-)

I heard the best way to learn something was to teach a course in it. 
This is my variation on that. I ignore prior art and reinvent from 
scratch. Burdick made me order "TeX the Program", but don't worry, I 
won't read it.

> 
> You can do mathematics with whatever typefaces you want,
> provided they provide the necessary characters. Lots of
> mathematics gets typeset in Knuth's "Computer Modern"
> faces because Knuth gave them a good supply of bits for
> mathematical typesetting (extending brackets,

Oh, geez, I forgot about extending brackets. My WYSIWYG math editor for 
my Algebra software drew it's own (really ugly) parens, brackets, and 
braces sized to fit. Oh, and you want ugly? The radical was horriffic.

OK, of course we'll use Computer Modern.


> mental scars to prove it. It is not always easy to
> persuade someone to switch away from something they've
> invested a lot of trouble learning...

I figured out a while ago that persuading others to use something is 
almost counterproductive, because they actively resist. Better to wait 
until they say, Cool. What did  you use?

kenny
From: Gareth McCaughan
Subject: Re: cl-typesetting
Date: 
Message-ID: <87oex6wksf.fsf@g.mccaughan.ntlworld.com>
Kenny Tilton wrote:

[I said:]
>> I have this scary vision of 20 people getting involved
>> in CL-TYPESETTING and each adding their own pet project
>> as a dependency, with the result that the whole thing is
>> impossible to build anywhere and becomes a licensing
>> nightmare later on... How portable is Cells?
> 
> What? You haven't installed Cells yet? :) Latest version (well, is on
> my hard drive) but the penultimate version is embedded in the DNA of
> RoboCells:
> 
>     http://sourceforge.net/projects/robocells/
> 
> Cells work on ACL, LW, MCL, CLisp, CormanCL, and CMUCL. Others just
> have not been tried. I'll gladly patch as necessary if they break
> somewhere.

Sounds reasonably portable. Cool.

> btw, the upside of your scary scenario is that we'd have twenty people
> working on it. So far only Burdick has offered to help. Where is
> Tolton? I thought he was looking for a project. :)

Yes, that thought occurred to me too. :-)

>> Um. If you have to ask that question, perhaps you shouldn't "do
>> math mode". :-)
> 
> I heard the best way to learn something was to teach a course in
> it. This is my variation on that. I ignore prior art and reinvent from
> scratch. Burdick made me order "TeX the Program", but don't worry, I
> won't read it.

You should at least read the relevant bits of the TeXbook
(the chapters with "Math" in the titles, and appendix G),
if only to give you some idea of what the issues are.

>> You can do mathematics with whatever typefaces you want,
>> provided they provide the necessary characters. Lots of
>> mathematics gets typeset in Knuth's "Computer Modern"
>> faces because Knuth gave them a good supply of bits for
>> mathematical typesetting (extending brackets,
> 
> Oh, geez, I forgot about extending brackets. My WYSIWYG math editor
> for my Algebra software drew it's own (really ugly) parens, brackets,
> and braces sized to fit. Oh, and you want ugly? The radical was
> horrific.

I bet. :-)

> OK, of course we'll use Computer Modern.

Initially. The Right Thing (so it seems to me) is to make sure
that there's an abstraction layer that, later on, will allow
using (say) Type 1 fonts with an extra metrics file and an
extra font containing only the special weirdo bits.

>> mental scars to prove it. It is not always easy to
>> persuade someone to switch away from something they've
>> invested a lot of trouble learning...
> 
> I figured out a while ago that persuading others to use something is
> almost counterproductive, because they actively resist. Better to wait
> until they say, Cool. What did  you use?

In which case, making CL-TYPESETTING less painful to program
probably won't help all that much. Alas.

-- 
Gareth McCaughan
.sig under construc
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <Zehdb.18394$lZ6.4780553@twister.nyc.rr.com>
Gareth McCaughan wrote:

> Kenny Tilton wrote:

>>I heard the best way to learn something was to teach a course in
>>it. This is my variation on that. I ignore prior art and reinvent from
>>scratch. Burdick made me order "TeX the Program", but don't worry, I
>>won't read it.
> 
> 
> You should at least read the relevant bits of the TeXbook
> (the chapters with "Math" in the titles, and appendix G),
> if only to give you some idea of what the issues are.

I ordered the wrong book?! :)

> Initially. The Right Thing (so it seems to me) is to make sure
> that there's an abstraction layer that, later on, will allow
> using (say) Type 1 fonts with an extra metrics file and an
> extra font containing only the special weirdo bits.

Oh, sure. I have actually done quite a bit with fontmetrics since all my 
GUI work involves knowing the various dimensions, and the CliniSys app 
has a wysiwyg requirement. I may have oversold my lack of experience.

> In which case, making CL-TYPESETTING less painful to program
> probably won't help all that much. Alas.

Well I am a lazy SOB so I make things painless anyway. My wysiwyg math 
editor does things like let you type "x12-x/3-7" and get x^12 - <x over 
3> - 7, ie, it guesses at DWIM to make the most common things easy, and 
lets you get the other things with just another keystroke to override 
the DWIM.

I gather you'll be signing on to this project if it gets moving? :)

kenny
From: Kenny Tilton
Subject: Re: cl-typesetting
Date: 
Message-ID: <1Rjdb.18460$lZ6.4845040@twister.nyc.rr.com>
Gareth McCaughan wrote:

> You should at least read the relevant bits of the TeXbook
> (the chapters with "Math" in the titles, and appendix G),
> if only to give you some idea of what the issues are.

OK, I decide to cut to the chase and order Knuth's Digital Typography:

    http://www.powells.com/cgi-bin/biblio?inkey=62-1575860104-2

Sounded good. But you know, I can at least do math mode just putting 
things roughly in the right place and generate some momentum. Then folks 
can yell at me and say "don't you know that goes three ems to the 
right?!" and I can fix it. But getting this off the ground is the key.

I have noticed that it is hard to get the busy, over-stressed people of 
modern life to sit down and contribute to projects, but everyone has 
time for a quick bitch-and-moan email or post.

Maybe I can find a way to  harness the energy of newsgroup nitpickers. 
Cold fusion, move over!

:)

kenny