From: Timofei Shatrov
Subject: Implementing REPL...
Date: 
Message-ID: <45e1e756.46285865@news.readfreenews.net>
Now this may look a bit stupid, but when the need arised to implement a little
REPL inside of my application I met some pretty annoying problems.

I made Gray streams for input and output. read-char for input stream waits for
user's keypress and returns the corresponding character (and also echoes it back
to the terminal). The REPL is implemented as follows:

(loop (prompt)
      (handler-case  (pr (eval (rd)))
           ...))

pr and rd are basically PRINT and READ with some additional function calls that
deal with the scrolling of the screen. There are two problems with READ that I
noticed:

1. Backspace key. When the backspace key is pressed, read-char returns
#\Backspace, and READ actually thinks this character is part of token name.
Which results in horrible errors.

2. READ finishes reading as soon as I press the final closing paren, which is in
stark contrast with actual Lisp's REPL, where I can enter several expressions on
the same line and they all get evaluated. Now that's some Lisp magic going on
here: when READ is called on a file stream, it always reads a single expression,
and this is also how it behaves in my app; however when called from within
Lisp's REPL, READ waits until I press Enter key (though still reading only the
first expression). That's probably controlled by some obscure special variable
for all I know.

I actually searched a bit in CLISP's source but it appears that most of its REPL
is implemented in C (SYSTEM::READ-EVAL-PRINT). So, at this point I've run out of
ideas and have to invoke the group's wisdom...

-- 
|Don't believe this - you're not worthless              ,gr---------.ru
|It's us against millions and we can't take them all... |  ue     il   |
|But we can take them on!                               |     @ma      |
|                       (A Wilhelm Scream - The Rip)    |______________|

From: Vassil Nikolov
Subject: Re: Implementing REPL... [especially the "read" part]
Date: 
Message-ID: <yy8vtzx9oeyv.fsf@eskimo.com>
On Sun, 25 Feb 2007 20:13:47 GMT, ····@mail.ru (Timofei Shatrov) said:

| Now this may look a bit stupid, but when the need arised to implement a little
| REPL inside of my application I met some pretty annoying problems.

  A "meta-question": how much time/effort would you like to invest in addressing
  this, and how portable would you like the solution to be?

  There are many ways to skin this proverbial cat, and they vary a lot in the
  above two aspects.  One extreme, for example, _might_ be to call READ-LINE and
  then READ-FROM-STRING instead of READ, which has the advantage of requiring
  very little effort and being (at least relatively) portable and the disadvantage
  of being a "worse is better" solution and thus possibly not a good fit.  (It
  might address the specific annoyances you described, but not solve your
  actual problem.  It may be worth describing in a more comprehensive way what
  behavior you need.  And then there may well be some code out there that
  already does what you want.)

  You might also want to try out running in an emacs shell buffer as
  another possible workaround, at least a temporary one.

  By the way, CLisp uses GNU readline (q.v.) in its REPL.

  ---Vassil.

-- 
Our programs do not have bugs; it is just that the users' expectations
differ from the way they are implemented.
From: Rob Warnock
Subject: Re: Implementing REPL... [especially the "read" part]
Date: 
Message-ID: <-cOdnfH28czS0n_YnZ2dnUVZ_qarnZ2d@speakeasy.net>
Vassil Nikolov  <···············@pobox.com> wrote:
+---------------
| ····@mail.ru (Timofei Shatrov) said:
| | ...a little REPL ... some pretty annoying problems.
...
| There are many ways to skin this proverbial cat, and they vary a lot...
| One extreme, for example, _might_ be to call READ-LINE and then
| READ-FROM-STRING instead of READ...
+---------------

Timofei, what Vassil is obliquely trying to tell you is that
your "problem" has more to do with the terminal driver on your
system and less to do with Common Lisp or CLISP per se [except
that CLISP uses GNU "readline()", which is a separate can of worms].

If you ran some other CL's normal REPL in an environment with
terminal line buffering turned off (e.g., "stty -icanon" on most
Unix systems) you'll get exactly the same "annoying problems"
you're seeing with your application: type in "(+ 1 2) " [that is,
#\( #\+ #\1 #\2 #\) #\Space] and the REPL *immediately* prints "3
" [that is, #\3 #\newline], without waiting for you to type a
newline yourself.

This may also prove helpful:

    http://www.nhplace.com/kent/PS/Ambitious.html
    Ambitious Evaluation: A New Reading of an Old Issue


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: ···············@gmail.com
Subject: Re: Implementing REPL... [especially the "read" part]
Date: 
Message-ID: <1172482474.835568.153490@8g2000cwh.googlegroups.com>
On 26 Feb, 02:00, Vassil Nikolov <···············@pobox.com> wrote:
> On Sun, 25 Feb 2007 20:13:47 GMT, ····@mail.ru (Timofei Shatrov) said:
>
> | Now this may look a bit stupid, but when the need arised to implement a little
> | REPL inside of my application I met some pretty annoying problems.
>
>   A "meta-question": how much time/effort would you like to invest in addressing
>   this, and how portable would you like the solution to be?

This is probably of no use to Timofei at all as it sounds like he
needs a proper REPL but in my code I cheated in a number of ways:

1. I'm using a GUI library so my 'evaluate' doesn't happen until the
user presses Enter which fires off a GUI event. The event handler just
reads the chunk of text as a string.

2. As a result, I don't allow expressions in the 'REPL' to span
multiple lines of input (easy to change later).

3. If the user happened to execute code which required input, like a
(read-line), then I hook in a Gray stream but again, it's connected to
a GUI widget which probably simplifies things.

FWIW, this is the code that I use (I'm sure it's full of holes):

(defun my-reader (code-string)
  (defun inner-reader (code-string start)
    (handler-case
       (when (< start (length code-string))
             (multiple-value-setq (read-result pos)
                (read-from-string code-string t nil :start start))
             (handler-case
                (pprint (eval read-result))
                (error (ex) (print-to-widget (format NIL "~%~a" ex)
"error"))
                (warning (ex) (print-to-widget (format NIL "~%~a" ex)
"warning")))
             (inner-reader code-string pos))
       (error (ex) (print-to-widget (format NIL "~%~a" ex) "error"))))
  (inner-reader code-string 0))

But to bring it back to the changed topic, it did seem to me that the
"read" part was the hard part and once I removed that it became much
easier.

Phil
From: Dmitry V. Gorbatovsky
Subject: Re: Implementing REPL...
Date: 
Message-ID: <es0rdu$mj1$1@aioe.org>
Timofei Shatrov wrote:

> 1. Backspace key. When the backspace key is pressed, read-char returns
> #\Backspace, and READ actually thinks this character is part of token
> #name.
>...
> 2. READ finishes reading as soon as I press the final closing paren, which
> is in stark contrast with actual Lisp's REPL, where I can enter several
> expressions on the same line and they all get evaluated.  

Hello,

As far as I understand this is about terminal emulator 
and windows/xwindow settings for keyboard.

Regards, 
Dmitry
From: Timofei Shatrov
Subject: Re: Implementing REPL...
Date: 
Message-ID: <45e41416.15209470@news.readfreenews.net>
On Tue, 27 Feb 2007 10:51:10 +0200, "Dmitry V. Gorbatovsky"
<·········@midasitech.com> tried to confuse everyone with this message:

>Timofei Shatrov wrote:
>
>> 1. Backspace key. When the backspace key is pressed, read-char returns
>> #\Backspace, and READ actually thinks this character is part of token
>> #name.
>>...
>> 2. READ finishes reading as soon as I press the final closing paren, which
>> is in stark contrast with actual Lisp's REPL, where I can enter several
>> expressions on the same line and they all get evaluated.  
>
>Hello,
>
>As far as I understand this is about terminal emulator 
>and windows/xwindow settings for keyboard.

Well, since terminal is implemented by me, I would like to see how to implement
line-buffering terminal with Gray streams. I guess READ-CHAR should wait until
the whole line is entered, and then feed characters from this line one by one.
This may even work, now that I think about it. I probably got sidetracked by the
name READ-CHAR, thinking that it should behave like getch() in C programs...

-- 
|Don't believe this - you're not worthless              ,gr---------.ru
|It's us against millions and we can't take them all... |  ue     il   |
|But we can take them on!                               |     @ma      |
|                       (A Wilhelm Scream - The Rip)    |______________|
From: Thomas A. Russ
Subject: Re: Implementing REPL...
Date: 
Message-ID: <ymislcr75oi.fsf@sevak.isi.edu>
····@mail.ru (Timofei Shatrov) writes:

> Well, since terminal is implemented by me, I would like to see how to implement
> line-buffering terminal with Gray streams. I guess READ-CHAR should wait until
> the whole line is entered, and then feed characters from this line one by one.
> This may even work, now that I think about it. I probably got sidetracked by the
> name READ-CHAR, thinking that it should behave like getch() in C programs...

In what way do you think READ-CHAR differs from getch() in C programs?

And is that really a difference in the way getch() works, or is it an
artifact of the way standard-input buffering works from a terminal under
your normal operating environment?  If you tried getch() on a file
stream under C, do you get any editting functions?  In other words, if
you had delete (backspace?) characters in a file, would you still get
editting?


-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Dmitry V. Gorbatovsky
Subject: Re: Implementing REPL...
Date: 
Message-ID: <es1ua6$615$1@aioe.org>
Timofei Shatrov wrote:

> Well, since terminal is implemented by me, I would like to see how to
> implement line-buffering terminal with Gray streams. 
If its still relevant, Climacs or Hemlock may well include terminal emulator
implemented in CL.

DG