From: Larry Clapp
Subject: Lisp with Vim
Date: 
Message-ID: <li5i4a.oj1.ln@rabbit.ddts.net>
Hi, all,

With his voluminous interview on /., Kent Pitman inspired me to check out
Lisp.  I've seen it before (in Hofstadter's Metamagical Themas and other
places), and even wrote a small Lisp interpreter in Turbo Pascal (v4) while in
high school (which helped immensely with my understanding of recursion &
linked lists, as you might imagine :).

However, Kent didn't quite inspire me to learn Emacs.  I fiddled with it, and
purchased _Learning GNU Emacs_, so I can figure it out if I absolutely must,
but I've been a vi/Vim user since I discovered Unix, and I figured I could
skip Emacs if I could figure out how to get Vim to talk to Lisp.  I've
implemented the absolute basics -- sending an expression from the editor to
the interpreter -- and hope to do more, as time goes by.  (For you Emacs fans:
I know that Emacs/Ilisp does a lot more than this; I'm sure it's the height of
hubris for me to consider duplicating it.  :)

I've enclosed VIlisp.vim, which defines functions and key mappings to send
Lisp code to a CMU CL process running in a seperate terminal window, via
funnel.pl.  I've also enclosed funnel.pl (oddly enough :), a Perl script which
accepts input both from Vim (via a fifo, which it creates) and from its own
terminal window (via GNU ReadLine), and sends it to CMU CL, and prints the
output (via "print" :).

Put this at an appropriate place in your ~/.vimrc:

    autocmd BufRead *.lsp,*.lisp so ~/path/to/VIlisp.vim

In an Xterm, run

    export PERL_RL=gnu			# use GNU readline in funnel
    funnel.pl $HOME/.lisp-pipe lisp

In a different xterm, run e.g.

    vim test.lisp

Put the cursor in a Lisp expression.  Type ,es to _e_valuate the
_s_-expression.  Type ,ec to _e_valuate the _c_urrent expression.  Example:

    (defun fib (n)
      (cond ((or (= n 0)
		 (= n 1)) 1)
	    (t (+ (fib (- n 1))
	    ;      ^ cursor here
		  (fib (- n 2))))))

With the cursor at the indicated spot, ",es" will eval the entire defun, and
",ec" will evaluate "(fib (- n 1))".

See VIlisp.vim for other mappings, or to change the current ones.  (I could
see using ,et (Evaluate Toplevel) instead of ,es, and ,ee (Evaluate
Expression) instead of ,ec.  To each his own.)

I expect I'll keep working on this -- functionality is still pretty minimal
(adding folding would be nice), and the code isn't all that clean, either.
Feel free to e-mail me if you have any questions, comments, suggestions, or
(be still my heart :) patches.

I use: Vim v6.0.93, CMU Common Lisp x86-linux 3.0.8 18c+, and Perl 5.6.1.
funnel.pl requires Perl modules IO::Pty, Term::ReadLine, and probably
Term::ReadLine::Gnu.  Developed on Debian "woody" GNU/Linux, kernel 2.4.17.
YMMV.

funnel.pl would probably work with any Lisp that will run in an xterm; just
change how it exits Lisp; CMU CL uses "(quit)".

-- Larry Clapp


p.s.  Attachments follow.  Sorry they're not "real" attachments; I can't
figure out how to do MIME in slrn.  I Read (some of) The Fine Manual, but only
saw how to read MIME messages, not post them.  Bummer.


VIlisp.vim:

BEGIN -- end of file has "END"
--- 8< Cut here ---

" Last updated: Thu Feb 14 23:45:13 EST 2002 

" only load the first time
if exists( "g:VIlisp_loaded" )
  finish
else
  let g:VIlisp_loaded = 1
endif

" ###################################################################
" ###################################################################
" functions


function! VIlisp_goto_buffer_or_window( buff )
  if -1 == bufwinnr( a:buff )
    exe "bu" a:buff
  else
    exe bufwinnr( a:buff ) . "wincmd w"
  endif
endfunction


function! VIlisp_load_syntax()
  if !exists( "b:current_syntax" ) || b:current_syntax != "lisp"
    set syntax=lisp
  endif
endfunction

function! VIlisp_get_pos()
  " what buffer are we in?
  let bufname = bufname( "%" )

  " get current position
  let c_cur = wincol()
  let l_cur = line( "." )
  normal H
  let l_top = line( "." )

  let pos = bufname . "|" . l_top . "," . l_cur . "," . c_cur

  " go back
  exe "normal " l_cur . "G" . c_cur . "|"

  return( pos )
endfunction


function! VIlisp_goto_pos( pos )
  let mx = '\(\f\+\)|\(\d\+\),\(\d\+\),\(\d\+\)'
  let bufname = substitute( a:pos, mx, '\1', '' )
  let l_top = substitute( a:pos, mx, '\2', '' )
  let l_cur = substitute( a:pos, mx, '\3', '' )
  let c_cur = substitute( a:pos, mx, '\4', '' )

  exe "bu" bufname
  exe "normal " . l_top . "Gzt" . l_cur . "G" . c_cur . "|"
endfunction


function! VIlisp_yank( motion )
  let p = VIlisp_get_pos()
  let old_l = @l
  exec 'normal "ly' . a:motion
  let value = @l
  let @l = old_l
  call VIlisp_goto_pos( p )
  return( value )
endfunction


" copy an expression to a buffer
function! VIlisp_send_sexp_to_buffer( sexp, buffer )
  let p = VIlisp_get_pos()

  " go to the given buffer, go to the bottom
  exe "bu" a:buffer
  silent normal G

  " tried append() -- doesn't work the way I need it to
  let old_l = @l
  let @l = a:sexp
  silent exe "put l"
  " normal "lp
  let @l = old_l

  call VIlisp_goto_pos( p )
endfunction
  

" destroys contents of VIlisp_scratch buffer
function! VIlisp_send_to_lisp( sexp )
  let p = VIlisp_get_pos()

  " goto VIlisp_scratch, delete it, put sexp, write it to lisp
  exe "bu" g:VIlisp_scratch
  exe "%d"
  normal 1G

  " tried append() -- doesn't work the way I need it to
  let old_l = @l
  let @l = a:sexp
  normal "lP
  let @l = old_l

  exe 'w >>' s:pipe_name

  call VIlisp_goto_pos( p )
endfunction


" Actually evals current top level form
function! VIlisp_eval_defun_lisp()
  " save position
  let p = VIlisp_get_pos()

  " find defun
"   if search( "^(defun", "bW" ) > 0
"     call VIlisp_send_to_lisp( VIlisp_yank( "%" ) )
"   endif
  normal 99[(
  call VIlisp_send_to_lisp( VIlisp_yank( "%" ) )

  " fix cursor position, in case of error below
  call VIlisp_goto_pos( p )
endfunction


function! VIlisp_eval_next_sexp_lisp()
  " save position
  let pos = VIlisp_get_pos()

  " find & yank current sexp
  normal [(
  let sexp = VIlisp_yank( "%" )
  call VIlisp_send_to_lisp( sexp )
  call VIlisp_goto_pos( pos )
endfunction


function! VIlisp_eval_block() range
  " save position
  let pos = VIlisp_get_pos()

  " yank current visual block
  let old_l = @l
  '<,'> yank l
  let sexp = @l
  let @l = old_l

  call VIlisp_send_to_lisp( sexp )
  call VIlisp_goto_pos( pos )
endfunction


function! VIlisp_copy_sexp_to_test()
  " save position
  let pos = VIlisp_get_pos()

  " find & yank current sexp
  normal [(
  call VIlisp_send_sexp_to_buffer( VIlisp_yank( "%" ), g:VIlisp_test )

  call VIlisp_goto_pos( pos )
endfunction



" ###################################################################
" ###################################################################
" startup stuff
let g:VIlisp_scratch = $HOME. "/.VIlisp_scratch"
let g:VIlisp_test = $HOME . '/.VIlisp_test'
let s:pipe_name = $HOME . '/.lisp-pipe'

exe "new" g:VIlisp_scratch
doauto lisp BufRead x.lsp
set syntax=lisp
set buftype=nowrite
set bufhidden=hide
set nobuflisted
set noswapfile
hide

exe "new" g:VIlisp_test
doauto lisp BufRead x.lsp
set syntax=lisp
" set buftype=nofile
set bufhidden=hide
set nobuflisted
" set noswapfile
hide

augroup VIlisp
    au!

    " autocmd BufEnter VIlisp* call VIlisp_load_syntax()
    autocmd BufLeave .VIlisp_* set nobuflisted
    autocmd BufLeave *.lsp,*.lisp let VIlisp_last_lisp = bufname( "%" )
augroup END

" hide from the user that we created and deleted (hid, really) a couple of
" buffers
exe 'normal '

" ###################################################################
" ###################################################################
" mappings

" ###################################################################
" Interact with Lisp interpreter

" send top-level sexp to lisp: eval s-exp
map ,es :call VIlisp_eval_defun_lisp()<cr>

" send current s-exp to lisp: eval current
map ,ec :call VIlisp_eval_next_sexp_lisp()<cr>

" eval block
map ,eb :call VIlisp_eval_block()<cr>

" reset interpreter
map ,ri :call VIlisp_send_to_lisp( "q\n" )<cr>

" ###################################################################
" Dunno?

" copy current s-exp to test buffer: Copy to Test
map ,ct :call VIlisp_copy_sexp_to_test()<cr>


" ###################################################################
" load/compile files

" load file: Load File; Load Any File, Load Compiled File
map ,lf :call VIlisp_send_to_lisp( "(load \"" . expand( "%:p" ) . "\")\n")<cr>
map ,laf :call VIlisp_send_to_lisp( "(load \"" . expand( "%:p:r" ) . "\")\n")<cr>
map ,lcf ,laf

" compile file: Compile File; Compile & Load File
map ,cf :call VIlisp_send_to_lisp( "(compile-file \"" . expand( "%:p" ) . "\")\n")<cr>
map ,clf ,cf,laf

" ###################################################################
" Move around among buffers

" goto test or scratch buffer
map ,tb :call VIlisp_goto_buffer_or_window( g:VIlisp_test )<cr>
map ,wtb :sb <bar> call VIlisp_goto_buffer_or_window( g:VIlisp_test )<cr>
map ,sb :exe "bu" g:VIlisp_scratch<cr>

" return to VIlisp_last_lisp -- "Test Return"
map ,tr :call VIlisp_goto_buffer_or_window( VIlisp_last_lisp )<cr>

" return to "lisp buffer", or "last buffer", even
map ,lb ,tr

" ###################################################################
" Mark & format code

" mark the current top-level sexpr: Mark Sexp
map ,ms 99[(V%

" format current, format sexp
map ,fc [(=%`'
map ,fs 99,fc

" ###################################################################
" Add & delete code

" remove my ,ilu mapping, which makes the ,il mapping slow
if maparg( ",ilu" ) != ""
    unmap ,ilu
endif

" Put a list around the current form: Insert List
map ,il [(%a)<esc>h%i(

" comment current -- doesn't work correctly on some forms
map ,cc m`[(i<cr><esc>v%<esc>a<cr><esc>:'<,'>s/^/; /<cr>``%/(<cr>


" ###################################################################
" Do stuff with VIlisp

" reload this file -- can't do this in a function
map ,rvil :exe "unlet! g:VIlisp_loaded <bar> so ~/lisp/VIlisp.vim"<cr>

--- 8< Cut Here ---
END

funnel.pl:

BEGIN
--- 8< Cut Here ---
#!/usr/bin/perl

# By Larry Clapp, ·····@theclapp.org
#
# Last updated: Thu Feb 14 22:42:54 EST 2002 
#
# spawn() adapted from the function of the same name in Expect.pm.  Docs for
# IO::Pty leave a lot to be desired, esp. if you haven't fiddled with ttys
# before.

use IO::Pty;                    # This appears to require 5.004
use Term::ReadLine;

use POSIX;                      # For setsid. 
use Fcntl;                      # For checking file handle settings.

sub spawn
{
  my( $tty, $name_of_tty );

  # Create the pty which we will use to pass process info.
  my($self) = new IO::Pty;

  # spawn is passed command line args.
  my(@cmd) = @_;

  $name_of_tty = $self->IO::Pty::ttyname();
  die "Could not assign a pty" 
      unless $name_of_tty;
  $self->autoflush();

  $pid = fork();
  unless (defined( $pid )) {
    warn "Cannot fork: $!";
    return undef;
  }
  unless ($pid) {
    # Child
    # Create a new 'session', lose controlling terminal.
    POSIX::setsid() 
	or warn "Couldn't perform setsid. Strange behavior may result.\r\n  Problem: $!\r\n";
    $tty = $self->IO::Pty::slave(); # Create slave handle.

    # We have to close everything and then reopen ttyname after to get a
    # controlling terminal.
    close($self);
    close STDIN; close STDOUT;
    open(STDIN,"<&". $tty->fileno()) || die "Couldn't reopen ". $name_of_tty ." for reading, $!\r\n";
    open(STDOUT,">&". $tty->fileno()) || die "Couldn't reopen ". $name_of_tty ." for writing, $!\r\n";

    # put this here or we would never see those die's above...
    close STDERR;
    open(STDERR,">&". $tty->fileno()) || die "Couldn't redirect STDERR, $!\r\n";

    exec (@cmd);
#    open(STDERR,">&2");
    die "Cannot exec ·@cmd': $!\n";
    # End Child.
  }

  # sleep 1/4 second; allow child to start
  # select( undef, undef, undef, 0.25 );

  return $self;
}



sub process_readline_data2
{
    print "got a line\n"
	if $debug;
    $lineread2 = $_[ 0 ];
    $got_a_line2 = 1; 				# global

    &uninstall_handler();
}


sub check_tty_data
{
    my( $tty_read, $rout, @bits, $n, $l );

    # poll tty
    $n = select( $rout = $tty_rin, undef, undef, 0 );

    if ($n)
    {
	# print ( "\nreading from spawned tty ...\n" );
	$tty_read = sysread( $master, $l, 10240 );
	if ($tty_read)
	{
	    print "$tty_read bytes read from tty\n"
		if $debug;

	    if ($stdin_data)
	    {
		print "stdin_data is >$stdin_data<\n",
		    "tty_data is >$l<\n"
			if $debug;

		$stdin_data =~ s/[\r\n]+$//;
		if ($l =~ /(\Q$stdin_data\E[\r\n]*)/)
		{
		    $match = $1;
		    print "Deleting >$match< from tty_data\n"
			if $debug;
		    $l =~ s/\Q$match\E//;

		    $stdin_data = undef;
		}
		else
		{
		    print "tty data doesn't match stdin_data\n"
			if $debug;
		}
	    }
	    else
	    {
		print "no stdin data\n"
		    if $debug;
	    }

	    print $l;
	    $tty_data = $l;

	    if ($tty_data !~ /[\r\n]$/)
	    {
		$tty_data =~ /\n?([^\r\n]*)$/;
		$last_partial_line = $1;
		if ($last_partial_line eq '')
		{
		    print "resetting last_partial_line\n"
			if $debug3;
		    $last_partial_line = undef;
		}
		else
		{
		    print "last partial line is >$last_partial_line<\n"
			if $debug3;
		}
	    }
	}
	else
	{
	    $tty_open = 0;
	    if (defined( $tty_read ))
	    {
		print "sysread of tty returned 0\n";
	    }
	    else
	    {
		print "sysread of tty returned undef\n";
		print "error code is $!\n";
	    }
	}
    }
    else
    {
	print "nothing read from tty\n"
	    if $debug;
    }

    return( $tty_read );
}

sub check_pipe_data
{
    my( $pipe_read, $rout, @bits, $n, $l );

    # poll fifo
    $n = select( $rout = $pipe_rin, undef, undef, 0 );

    if ($n)
    {
	print ( "\nreading from pipe ...\n" )
	    if $debug;
	$pipe_read = sysread( PIPE, $l, 10240 );
	if ($pipe_read)
	{
	    print "$pipe_read bytes read from pipe\n"
		if $debug;

	    print $master $l;
	    $pipe_data = $l;
	}
	else
	{
	    print "sysread on pipe returned 0\n";
	    print $master "(quit)\n";
	    $pipe_open = 0;
	}
    }
    else
    {
	print "nothing read from pipe\n"
	    if $debug;
    }

    return( $pipe_read );
}

sub install_handler
{
    if (defined( $last_partial_line ))
    {
	print "Installing handler with prompt >$last_partial_line<\n",
	    "resetting last_partial_line\n"
	    if $debug3;
	$rl->{already_prompted} = 1;
	$rl->callback_handler_install( $last_partial_line, \&process_readline_data2 );
	$handler_installed = 1;
	$told_readline = 1;
	$last_partial_line = undef;
    }
    else
    {
	print "Installing handler with no prompt\n"
	    if $debug3;
	$rl->{already_prompted} = 0;
	$rl->callback_handler_install( '', \&process_readline_data2 );
	$handler_installed = 1;
	$told_readline = 0;
    }
}

sub uninstall_handler
{
    print "Uninstalling handler\n"
	if $debug3;
    $rl->callback_handler_install( '', undef );
    $handler_installed = 0;
}


sub check_stdin_data
{
    my( $stdin_read, $rout, @bits, $n );

    if ($handler_installed)
    {
	if ($last_partial_line
	    && !$told_readline)
	{
	    print "Telling readline on new line; resetting last_partial_line\n"
		if $debug3;
	    $told_readline = 1;
	    $rl->set_prompt( $last_partial_line );
	    $rl->on_new_line_with_prompt;
	    $last_partial_line = undef;
	}

	# poll stdin
	$n = select( $rout = $stdin_rin, undef, undef, 0 );

	if ($n)
	{
	    # process the character
	    $rl->callback_read_char();
	}
    }
    else
    {
	&install_handler();
    }

    # process the line *after* you process the character, if any
    if ($got_a_line2)
    {
	$got_a_line2 = 0;
	if (defined( $lineread2 ))
	{
	    # $lineread .= "\n";
	    print $master $lineread2, "\n";
	    $stdin_data = $lineread2;
	    $stdin_read = length( $stdin_data );

	    $lineread2 =~ s/[\r\n]+//;
	    if ($lineread2 ne '')
	    {
		print "Adding history >$lineread2<\n"
		    if $debug2;
		$rl->AddHistory( $lineread2 );
		if ($debug2)
		{
		    printf "where_history is %d\n", $rl->where_history;
		    foreach $f ($rl->GetHistory)
		    {
			printf( "History is %s ", $f );
		    }
		}
	    }
	}
	else
	{
	    print "readline on stdin returned empty line\n";
	    $stdin_open = 0;
	}
    }

    return( $stdin_read );
}


# timeout in seconds
$timeout = 0.05;

$debug = 0;
$debug2 = 0;
$debug3 = 0;

$pipe_name = shift @ARGV;

if (! -e $pipe_name)
{
    system( "mkfifo -m go-rwx $pipe_name" );
}
elsif (! -p $pipe_name)
{
    print STDERR "$pipe_name exists, and is not a fifo\n";
    exit( 1 );
}

$writer_pid = fork();
if (0 == $writer_pid)
{
    # child

    # Open it for output, so the open-for-input call doesn't block.  This allows
    # any other process to just open it, write to it, and close it, and we don't
    # have to worry about it closing due to lack of writers.
    open( PIPE_OUT, ">$pipe_name" ) or die "couldn't open pipe for output: $!";

    # sleep for a year
    sleep( 3600 * 24 * 365 );
    exit;
}

# don't need to sleep to wait for above child -- the OS will block us until
# the child opens the pipe
open( PIPE, "<$pipe_name" ) or die "couldn't open fifo '$pipe_name': $!";

select( PIPE ); $| = 1;
select( STDOUT ); $| = 1;

$last_partial_line = undef;
$tty_open = 1;
$pipe_open = 1;
$stdin_open = 1;

$master = &spawn( @ARGV );
print "spawned @ARGV\n"
    if $debug;

$tty_rin = '';   vec( $tty_rin, $master->fileno(), 1 ) = 1;
$pipe_rin = '';  vec( $pipe_rin, fileno( PIPE ), 1 ) = 1;
$stdin_rin = ''; vec( $stdin_rin, fileno( STDIN ), 1 ) = 1;

$rl = new Term::ReadLine 'funnel';
$rl->prep_terminal( 0 );
$attribs = $rl->Attribs;
$rl->using_history();

if ($debug)
{
    @features = keys %{ $rl->Features };
    print "ReadLine features: @features\n";
}

&install_handler();

$rin_all = '';
vec( $rin_all, $master->fileno(), 1 ) = 1;
vec( $rin_all, fileno( PIPE ), 1 ) = 1;
vec( $rin_all, fileno( STDIN ), 1 ) = 1;

while ($tty_open
       && $pipe_open
       && $stdin_open)
{
    $tty_read = &check_tty_data();
    $pipe_read = &check_pipe_data();
    $stdin_read  = &check_stdin_data();

    if (!$tty_read
	&& !$pipe_read
	&& !$stdin_read
	&& $tty_open
	&& $pipe_open
	&& $stdin_open)
    {
	# wait for *any* data
	print "waiting for any data\n"
	    if $debug;
	$n = select( $rout = $rin_all, undef, undef, undef );
    }
}

print "Signaling writer\n"
    if $debug;
kill 2, $writer_pid;
$rc = wait();
print "wait returned $rc\n"
    if $debug;

# FIXME: probably need to do something to close down the tty, but I don't know
# exactly what

# delete the fifo
unlink( $pipe_name );
--- 8< Cut here ---
END

From: Larry Clapp
Subject: Re: Lisp with Vim
Date: 
Message-ID: <ib7i4a.7k2.ln@rabbit.ddts.net>
In article <·············@rabbit.ddts.net>, Larry Clapp wrote:
[ 20k of text deleted ]

Sorry, should've just posted links.  Will do tomorrow.

Apologies.

-- Larry
From: Max Ischenko
Subject: Re: Lisp with Vim
Date: 
Message-ID: <jvri4a.3ps.ln@cvs>
 Larry Clapp wrote:

> Sorry, should've just posted links.  Will do tomorrow.

Hi Larry. Would you mind to post your files at vim.sf.net at scripts
section?
From: Larry Clapp
Subject: Re: Lisp with Vim
Date: 
Message-ID: <dsjs4a.l18.ln@rabbit.ddts.net>
In article <·············@cvs>, Max Ischenko wrote:
> Hi Larry. Would you mind to post your files at vim.sf.net at scripts
> section?

Thanks for the pointer, Max.

VIlisp.vim posted at http://vim.sourceforge.net/scripts/script.php?script_id=221

-- Larry
From: Bulent Murtezaoglu
Subject: Re: Lisp with Vim
Date: 
Message-ID: <87d6z7jll1.fsf@nkapi.internal>
>>>>> "LC" == Larry Clapp <·····@theclapp.org> writes:

    LC> ...  (For you Emacs fans: I know that Emacs/Ilisp does a
    LC> lot more than this; I'm sure it's the height of hubris for me
    LC> to consider duplicating it.  :) ...

Actually, if you will sink energy into this you might also want to look at 
Franz's lisp-emacs interface (LEP is the acronym AFAIK).   

cheers,

BM
   
From: George Brewster
Subject: Re: Lisp with Vim
Date: 
Message-ID: <a4m3ab$8mb$1@news.fas.harvard.edu>
On Fri, 15 Feb 2002 00:31:25 -0500, Larry Clapp wrote:

> ...
> However, Kent didn't quite inspire me to learn Emacs.  I fiddled with
> it, and purchased _Learning GNU Emacs_, so I can figure it out if I
> absolutely must, but I've been a vi/Vim user since I discovered Unix,
> and I figured I could skip Emacs if I could figure out how to get Vim to
> talk to Lisp.  I've implemented the absolute basics -- sending an
> expression from the editor to the interpreter -- and hope to do more, as
> time goes by.  (For you Emacs fans: I know that Emacs/Ilisp does a lot
> more than this; I'm sure it's the height of hubris for me to consider
> duplicating it.  :)

Have you checked out VIPER, the vi emulator for Emacs? I always preferred
vi keybindings, but envied the Lisp features of Emacs, and I've found
Emacs/VIPER to be pretty much the best of both worlds. So I certainly
don't want to discourage the work you're doing, but do check VIPER out if
you haven't already... it sounds like the kind of thing you might be
looking for.

-George