From: Jürgen Böhm
Subject: Intelligent editor framework in Lisp?
Date: 
Message-ID: <fi2ltk$4ps$00$1@news.t-online.com>
Hi,

  I just finished writing a little microassembler in Lisp, using a
self-built LL(1) parser-generator and a self-written lexer. So far so
good, now I can compile microprogram files that are built conforming to
the syntax I defined and get warnings if syntax errors occur.
   Unfortunately it is much more difficult to keep the semantic side of
the input file in view. The first problem is, there can be syntactically
right but semantically nonsensically statements, which could be detected
by a special soundness check.
  But secondly, it would be much more helpful, to have an intelligent
editor at hand for entering the microassembler file. To give an example,
what this editor should do, consider an assignment:

...., datbus = val, .....

Here the "intelligent" editor could present the proposal "val" among a
list of fitting values after "..., datbus =  " has been entered. The
point here is that the proposal "val" can be made only after parsing the
just entered text.

   I have seen such things done perfectly to a fascinating degree in the
Eclipse environment, but of course I would prefer to stay with Lisp,
especially as it would take me probably a lot of time learning how to
customize Eclipse in this way.

Is emacs here the solution, or are there "editor frameworks" in Lisp
tailored for such tasks (building intelligent editors for self-defined
languages)?

Thanks

J�rgen


-- 
J�rgen B�hm                                            www.aviduratas.de
"At a time when so many scholars in the world are calculating, is it not
desirable that some, who can, dream ?"  R. Thom
From: Alan Crowe
Subject: Re: Intelligent editor framework in Lisp?
Date: 
Message-ID: <86lk8qe0yk.fsf@cawtech.freeserve.co.uk>
J�rgen B�hm <······@gmx.net> writes:

> Hi,
> 
>   I just finished writing a little microassembler in Lisp, using a
> self-built LL(1) parser-generator and a self-written lexer. So far so
> good, now I can compile microprogram files that are built conforming to
> the syntax I defined and get warnings if syntax errors occur.
...
>   But secondly, it would be much more helpful, to have an intelligent
> editor at hand for entering the microassembler file. 
...
> Is emacs here the solution, or are there "editor frameworks" in Lisp
> tailored for such tasks (building intelligent editors for self-defined
> languages)?

I think that emacs is the solution.

There doesn't seem to be much interest in intelligent
editors for self-defined languages and I don't think this is
just a matter of saying "Oh, just use emacs.".  There are
also two "behind the scenes" issues that bear on the
question.

1)Think about extensible programming languages. In Common
  Lisp a source file may start with a macro definition and
  then continue on to use it. (Though you will need to load
  the file if you want to be sure that you can use the macro
  in other files, see 3.2.3.1.1 Processing of Defining
  Macros)

  So an intelligent editor for a similar extensible language
  needs to grok the definition when it reads in the file at
  the start of the editing session and do a bit of dynamic
  reconfiguration so that it recognises the new
  constructs. Perhaps the parser needs to be updated with
  some new keyword and their grammar.  How does it cope when
  you start to edit the macro definition? The mere thought
  of this is frightening.

  The Lisp Way is to duck this challenge. Parsing is
  directed by the parentheses. The parser doesn't attempt to
  understand defmacro and doesn't communicate with it.

  The underlying issue is the conflict between extensibility
  and custom syntax. You cannot practically extend the
  custom syntax. The Lisp response to this is to not bother
  with a custom syntax. 

2)One custom syntax is a triumph, a dozen are a
  tragedy. Writing a parser generator and a lexer are
  valuable skills. There are applications in which many
  users will be using a single, fixed language and tuning
  the language, which is the user interface, tightly to the
  application is worth a lot of money to the organisation
  employing all those users. I intend to acquire those
  skills myself one day.

  And yet...  There is a benefit from using a new custom
  syntax in an application and a cost from now having to
  cope with n+1 custom syntaxes. All too often, once the
  immediate excitement of the project has cooled, and the
  syntax is no longer fresh in the mind but just an extra
  one to remember, the cost exceeds the benefit.

  The Lisp Way is to standardise on the standard io syntax
  used for Lisp source code for all human readable data
  files. Write them out with (write stuff :escape t
  :readably t) and read them back in with (read).

  You can customise the Lisp Reader with read macros. The
  interface is a little bit clunky, but the disincentive for
  doing this is very different. The cost is
  organisational. Every-one has to know the new syntax. It
  has to be documented. One has to talk to the training
  department about updating the course for new hires...

I'm tempted to ramble on about Maudslay and the screw
cutting lathe and how, before 1800, nuts and bolts where
hand made in individual pairs. Muddle up your nuts and bolts
and "screwed" is exactly the wrong word, screwing things
together is what you become unable to do. The parallel I
intend to draw is the plethora of file formats that one
encounters, each file type (bolt) has its own custom format
(nut). Computing is somehow a pre-1800 craft occupation.

On the other hand I'm hors de combat. You must be careful to
test my opinions against your own reason and experience. 

Alan Crowe
Edinburgh
Scotland