From: Howard R. Stearns
Subject: Re: Integrating personal application with existing software
Date:
Message-ID: <362388D7.42E393C8@elwood.com>
Devin Currie wrote:
>
> Hello all. :-)
>
> I am just starting out in CL.
>
> Is it possible to integrate personal programs to existing software that does
> not provides CL support? Thanks.
>
> --
> Devin
It depends on what you are trying to do, i.e., what do you mean by
integrating personal programs.
There are several ways to build an application out of separately
written pieces. As much as implementers may accidently or
deliberately obfuscate this, these methods are mostly independent of
program language or implementation support. (Although, of course,
explict support by some product can make your job easier, this support
does not necessarilly change the underlying technology.)
The first step in integrating applications is to define the nature of
the required interaction and examine the different available
technologies with respect to these requirements. Try not to let
familiarity with one technique force you into using an inappropriate
technology. Here are some different integration technologies:
1. Most languages, including Lisp, allow you to put different parts of
your application in different files, which are somehow combined. This
is simplest when all the files are written in the same language.
In C, the combination process is linking. To use this technique, one
must have machine-compiled object files representing the different
parts of the application. Some Lisp implementations provide libraries
in this format, and/or provide ways to convert user code to this
format. Examples include generation of C code or direct creation of
..dlls.
Within a Lisp development environment, files are simply LOADed,
possibly after compiling them first. Some Lisp implementations allow
you to LOAD compiled files that were originally written in languages
other than Lisp. In this case, you can load your machine-compiled
object files directly into a Lisp system. Consult your
implementation's documentation for the LOAD function.
Both of these techniques assume that the different parts of the
application are all combined into a single executable, running within
a single operating system process. Some operating systems also allow
one to share memory between different processes. This is a special
case of the above, and is an opering system, not a language issue.
However, it is difficult to use this technique with most garbage
collected systems (including Lisp).
2. Regardless of how the code is combined into memory, there may be
issues involved in directly using one language's functions and data
from within another language.
+ Some implementations allow (some specified set of) different
languages to call each other directly and to pass data directly, with
differing levels of restrictions.
+ In other cases, there is a "foreign-function" interface.
+ A third, even more remote technique is to provide a primitive
"engine" callable from one languae that can be used to run code from
another language. Often, there is some way to call "native" code from
within the code run by the engine. (This is what byte-coded
implementations often provide.)
Some issues to watch out for are differing calling conventions,
restrictions on acceptable data types and/or data conversion, and with
garbage collected languages like Lisp, unexpected relocation and
tracing of data. Other issues involve mixing control structure
between the two languages. For example, in Lisp one can define
dynamic exits (catch/throw), dynamic binding of global variables, and
cleanup forms that are executed during exits. Some
language-implementations/operating systems provide facilities for
asynchronous interupts and/or threads. You need to beware of bad
interactions between interconnected code written in two different
languages. Most Lisp implementations document how this is done in the
case of Lisp-C interactions.
3. There are several ways to have two (or more) separate executables
communicate with each other.
One traditional method on Unix systems is through pipes and/or
sockets. A simple example of this is when the text output of one
program is connected to the text input of another. Most Lisp
implementations provide some way to participate in such a chain, as
well as more complicated tees, sockets, etc. A traditional method to
communicate textually represented data on Windows systems is through
DDE.
Operating systems also often provide different ways for one program to
start another program running, using shared memory and/or pipes and
such for communication.
These communications paths are not really language issues, but
operating system issues. Some Lisp implementations provide direct
support for these techniques. In others, one can use operating system
libraries to provide whatever capabilities are required, using the
"in-process" program integration techniques described earlier.
4. One issue with pipe and DDE communication is that, in the simplest
uses of these techniques, communicated data must be represented as
text. Lisp provides many easy ways to get default and customized
textual representations of system and user defined data, and similar
default and customized ways to read it. Nonetheless, for efficiency
or accuraccy reasons, it may be desirable to use some binary encoding.
Such binary communication paths are often refered to as "transports",
and include (if I understand things correctly) XDR/EDR, TCP/IP, UDP.
5. Transports deal directly with primitive data, not function calls or
application-specific data types. Using either these or
semaphors/flags in shared or single-process memory, it is possible to
build protocols for more complex data, function calling, etc. One can
do this by hand, on and application specific basis, or use one of the
many protocols that have been developed for this.
Some simple such protocols are OLE/COM, HTTP, RPC, and Courier. If
your implementations of the different parts of your application
provide direct support for these, they may be used to communicate
data, and in some cases, events or operations without having to deal
with the lower level issues.
Finally, general inter-language, inter-process communications
protocols have been developed on top of these, including CORBA and ILU
on TCP, etc., and various protocols on top of OLE/COM. These provide
some of the same (variously restricted) data passing and function
calling capabilities mentioned in item 2, above. The difference is
that these protocols work when the two (or more) parts of an
application are in different processes, or even on different machines.
(Many implementations of these protocols still allow one to use them
within a single executable program.)
These highest level protocols are very general and can be used in wide
variety of systems and languages. Implementations of the various
pieces necessarry to make them work (server and client code) can be
obtained independent of programming language or language
implementation, and integrated into your application using the earlier
mentioned technologies if not directly supported by your vendor. The
drawback is that they are often reduced to a lowest common denominator
of application capabilities. If some more specific capability is
required, one must often use one of the underlying technologies.