Hi,
I have some questions concerning CMUCL's FFI. The CMUCL manual says that
CMUCL can only
load 8 MB of foreign code. Pointers to Lisp arrays can be passed as
arguments to foreign functions so
that much bigger arrays as 8MB can be used with foreign functions.
But does the 8 MB limit imply that no object file bigger than 8 MB can
be loaded and what happens when
the foreign function allocates memory of more than 8 MB?
Thanks in advance.
Rolf
>>>>> "Rolf" == Rolf Wester <···········@t-online.de> writes:
Rolf> I have some questions concerning CMUCL's FFI. The CMUCL
Rolf> manual says that CMUCL can only load 8 MB of foreign
Rolf> code. Pointers to Lisp arrays can be passed as arguments to
Rolf> foreign functions so that much bigger arrays as 8MB can be
Rolf> used with foreign functions.
If you mean malloc'd space, this is no longer true. The C heap space
is about 256 MB on all(?) platforms now. Still significantly smaller
than the Lisp heap of a GB or more in some cases, but probably more
than adequate.
If you really mean code, I'm not sure about that. There is a large
gap in the memory space for shared libs and stuff.
Rolf> But does the 8 MB limit imply that no object file bigger
Rolf> than 8 MB can be loaded and what happens when the foreign
Rolf> function allocates memory of more than 8 MB?
If the foreign code malloc'd too much, it would fail. If you tried to
malloc an alien object, Lisp would return an error, I think.
Ray
Raymond Toy wrote:
> >>>>> "Rolf" == Rolf Wester <···········@t-online.de> writes:
>
> Rolf> I have some questions concerning CMUCL's FFI. The CMUCL
> Rolf> manual says that CMUCL can only load 8 MB of foreign
> Rolf> code. Pointers to Lisp arrays can be passed as arguments to
> Rolf> foreign functions so that much bigger arrays as 8MB can be
> Rolf> used with foreign functions.
>
> If you mean malloc'd space, this is no longer true. The C heap space
> is about 256 MB on all(?) platforms now.
Is this documented anywhere?
> Still significantly smaller
> than the Lisp heap of a GB or more in some cases, but probably more
> than adequate.
>
I have a program written in C++ that I want to use from Lisp. The C++
programe
allocates memory of up to 500 MB. So the 256 MB are a serious limitation
in this case.
>
> If you really mean code, I'm not sure about that. There is a large
> gap in the memory space for shared libs and stuff.
>
> Rolf> But does the 8 MB limit imply that no object file bigger
> Rolf> than 8 MB can be loaded and what happens when the foreign
> Rolf> function allocates memory of more than 8 MB?
>
> If the foreign code malloc'd too much, it would fail. If you tried to
> malloc an alien object, Lisp would return an error, I think.
>
> Ray
Thanks for your reply.
Rolf
>>>>> "Rolf" == Rolf Wester <···········@t-online.de> writes:
Rolf> Raymond Toy wrote:
>> >>>>> "Rolf" == Rolf Wester <···········@t-online.de> writes:
>>
Rolf> I have some questions concerning CMUCL's FFI. The CMUCL
Rolf> manual says that CMUCL can only load 8 MB of foreign
Rolf> code. Pointers to Lisp arrays can be passed as arguments to
Rolf> foreign functions so that much bigger arrays as 8MB can be
Rolf> used with foreign functions.
>>
>> If you mean malloc'd space, this is no longer true. The C heap space
>> is about 256 MB on all(?) platforms now.
Rolf> Is this documented anywhere?
Not really. But if you have the sources, look in
src/lisp/*-validate.h to see. For x86, BSD systems have:
* 0x00000000->0x0E000000 224M C program and memory allocation.
* 0x0E000000->0x10000000 32M Foreign segment.
* 0x20000000->0x28000000 128M Reserved for shared libraries.
* 0xE0000000-> 256M C stack - Alien stack.
Linux says:
* 0x08000000->0x10000000 128M C program and memory allocation.
* 0x40000000->0x48000000 128M Reserved for shared libraries.
Sparc says:
* 0x00000000->0x10000000 256M C code and stuff(?)
* 0xc0000000->0xffffffff 1024M C stack, dynamic libs, etc.
>> Still significantly smaller
>> than the Lisp heap of a GB or more in some cases, but probably more
>> than adequate.
>>
Rolf> I have a program written in C++ that I want to use from
Rolf> Lisp. The C++ programe allocates memory of up to 500 MB. So
Rolf> the 256 MB are a serious limitation in this case.
If the C++ routines actually use 500 MB at one time, you are out of
luck with stock versions of CMUCL. If you have the option of passing
in pre-allocated stuff, then you can allocate the memory from Lisp
heap and pass them into the routines. (This is how matlisp works.)
The User's manual has an example, and if you need more help, please
ask on the CMUCL mailing lists.
Alternatively, it might be possible to build a new version for you
with a larger space by moving around the others. Of course, your Lisp
heap space will probably be much smaller. I think this is relatively
straight-forward but you need to do a cross-compile to do it.
Again, if you're interested, please ask on the CMUCL mailing lists.
Ray
Raymond Toy wrote:
>
> Rolf> Is this documented anywhere?
>
> Not really. But if you have the sources, look in
> src/lisp/*-validate.h to see. For x86, BSD systems have:
>
> * 0x00000000->0x0E000000 224M C program and memory allocation.
> * 0x0E000000->0x10000000 32M Foreign segment.
> * 0x20000000->0x28000000 128M Reserved for shared libraries.
> * 0xE0000000-> 256M C stack - Alien stack.
>
> Linux says:
>
> * 0x08000000->0x10000000 128M C program and memory allocation.
> * 0x40000000->0x48000000 128M Reserved for shared libraries.
>
> Sparc says:
>
> * 0x00000000->0x10000000 256M C code and stuff(?)
> * 0xc0000000->0xffffffff 1024M C stack, dynamic libs, etc.
>
>
Thank you for that information.
>
> If the C++ routines actually use 500 MB at one time, you are out of
> luck with stock versions of CMUCL. If you have the option of passing
> in pre-allocated stuff, then you can allocate the memory from Lisp
> heap and pass them into the routines. (This is how matlisp works.)
That would be a little bit cumbersome to do with the C++ program I mentioned but
may be an possibility for future developments.
> The User's manual has an example, and if you need more help, please
> ask on the CMUCL mailing lists.
>
I tried it, it worked (almost) as expected.
>
> Alternatively, it might be possible to build a new version for you
> with a larger space by moving around the others. Of course, your Lisp
> heap space will probably be much smaller. I think this is relatively
> straight-forward but you need to do a cross-compile to do it.
>
> Again, if you're interested, please ask on the CMUCL mailing lists.
>
> Ray
Thank you for your help. It was very instructive.
Rolf