On a 400MHz Pentium II, the minimum overhead for using a foreign
function with Lispworks seems to be about 700ns.
What I'm looking for is a general method of implementing faster code
in tight loops than what seems possible with the present version of
Lispworks. Of course 99% of a typical program does not need such
attention to speed, but I'm talking about the 1% that does. Ideally I
should be able to put some inline assembler language in the tightest
loops. Or I might be able to find some way to make the FFI faster.
Another option would be to write more of the code in the foreign
functions, so they would do more per invocation. But that partly
defeats the purpose of using a higher level language such as Lisp.
One possibility might be to have the Lisp program generate some
machine code and store it somewhere in memory, then somehow invoke it
from tight loops. I can figure out how to generate and store it, but
how can I invoke it? Do I have to modify the compiled code generated
by Lispworks? Has anyone done that?
Of course whatever I do along those lines will be a mess, but I can
solve that problem by encapsulating it, such that whenever I want to
use fast code in tight loops in the future I can do it neatly and
easily.
And of course there are lots of things Lispworks does so fast there
would be no point in trying to speed them up further. But there are
also lots of things it does slowly, which is why this kind of thing is
desirable.