Hi all, I am a newbie to Lisp, but have around 8 years of C experiance.
I'm learning Lisp by reading (LOTS! :) and implementing a simple 3D
engine as I go. I've done some small toy engines before, so this is
personally a good way for me to compare languages.
Anyhow, I intend to implement very low level data structures first -
but I don't know how to go about doing this.
For example, in C I would usually have
struct vector3
{
float x,y,z;
}
vector3 points[8]; // the points in my cube
I've taken the SDL examples & made a simple app showing a mesh, but my
mesh is composed of lists & is rendered as
(dolist (i indices)
(apply #'gl:vertex3f (nth i verticies)))
Which (even aside from the nth call) is slow. Normally in OGL I setup
a bunch of property arrays (points, normals, etc), enable them and draw
with a single call to glDrawElements (indicies);
So my question is - how do I go about creating these arrays such that
they match the underlying format that OGL expects? Is it possible to
do this in a Lisp agnostic way? Any readings that are particularly
relevant to me here?
At the moment I am using CMUCL on Linux & don't know enough to choose
the correct Lisp for me.
Thanks
Brad
Brad,
I have found (at least on CMUCL and Lispworks) that the fastest
representation for vectors is
(make-array 3 :element-type 'float) (or 'single-float or
'double-float)
This will allow the compiler to do unboxed floating point operations
(if you put the appropriate declarations in your routines)
I do the same thing as you do in C for speeding up the drawing calls to
OpenGL. I put all the data in arrays and use glDrawElements(indices).
To do this, you need to ensure you have the data in properly typed
arrays
(e.g. your vertices array would be: (make-array (* points 3)
:element-type 'single-float))
indices would be (make-array number-of-indices :element-type
'(unsigned-byte 16)) ; shorts)
In CMUCL, to pass the pointers to the data to OpenGL you can do:
(gl-draw-elements mode num-points GL_UNSIGNED_SHORT (sys:sap-int
(sys:vector-sap points)))
you might want to macro those dereferences. Its been a while since i
looked at the CMUCL ffi, there may be other ways to specify the
parameters where the system will do the automatic dereferencing instead
of requiring you to do it.
Thanks a lot for that. Am I best to just macro up the (sys:sap-int
....) call to something like (address-of points), and then make the
macro portable later?
I am a little concerned for portability - I generally try to write
fairly portable C code - should I try to write these kinds of things
using UFFI/CFFI, macro wrappers (my own portability layer), or not even
bother at all for now?
Thanks again - I'll probably have many more naive questions :)
Brad
I would say to understand the ffi before using UFFI/CFFI is probably a
good thing.
I did find that using foreign structures in CMUCL to be very slow and
went to using malloc and arranging things directly in memory for one
performance bottleneck that I had.
I'm not familiar with uffi/clrfi/whichever derivitive is most recent
except that they are a portable layer. I don't know if they handle
issues like LispWorks requiring any memory passed to a foreign function
has to be allocated in the static area.
Another thing to think about with CMUCL, that you ought to halt garbage
collection before changing your arrays to pointers and passing them to
the foreign function so that they don't get moved by the gc between the
time you converted them to pointers and the time you pass them to the
foreign function.
I read the juicy bits of the CMUCL manual last night & managed to
convert my simple list drawing code into simple array drawing code.
When you say "I did find that using foreign structures in CMUCL to be
very slow and went to using malloc and arranging things directly in
memory for one performance bottleneck that I had."
Were you creating alien objects, manipulating them & then passing them
to C? I am planning on trying to use Lisp data (such as make-array) of
the correct type and then passing them to C as you previously described
- though I guess that I am probably best to try and keep all this
pretty abstract for now.
Good tip about the GC - I guess it is common for Lisps to use copying
collectors?
Cheers
Brad
I was referring to structure handling. This was a while ago so I don't
remember the details but I think it was kevent structures. When I
hammered my app hard with network requests, the bottleneck seemed to be
in the foreign structure accesses.
Looking at the code now, i see a static buffer allocated with malloc
that uses sys:sap-ref-32 to pull the values out of memory and into lisp
before/after my kevent calls.
The speed difference was an order (or several) of magnitude, otherwise
I would have left the cleaner code in place if it was only a minimal
hit.
Brad Might wrote:
> I would say to understand the ffi before using UFFI/CFFI is probably a
> good thing.
Back in the day, I found UFFI helpful in that it somehow got me going on
FFI a little easier than staring at my native FFI. I guess by being a
least common denominator of multiple FFIs it ended up simpler, albeit
less powerful. Or maybe KMR's doc was more approachable, again by having
fewer complexities to document.
Likewise, I think CFFI will be easier to start with, and gives you
portability to boot. Because CFFI is currently being extended/refined
and the paint is so fresh, I wager support even for users on cffi-devel
will be excellent. Not that there is anything wrong with uffi support.
And lets not forget Verrazano (vzn). That can write the bindings for
you. I recommended to someone doing a sockets package they switch from
UFFI to CFFI. Five minutes later I emailed them the winsock API
bindings, written by vzn from winsock.h. Now vzn also has an UFFI
backend, but....
> I did find that using foreign structures in CMUCL to be very slow and
> went to using malloc and arranging things directly in memory for one
> performance bottleneck that I had.
>
> I'm not familiar with uffi/clrfi/whichever derivitive is most recent
> except that they are a portable layer.
UFFI was a breakthru for CL but even its maintainer and fans know it
needs work, and the consensus also is that this should be an UFFI-2 that
cannot be compatible with UFFI, because of the changes required.
CFFI-Luis includes AFAIK what UFFI-2 would be, and perhaps obviates the
need for it. Note that it also includes an UFFI compatibility layer.
I don't know if they handle
> issues like LispWorks requiring any memory passed to a foreign function
> has to be allocated in the static area.
CFFI-luis (make sure you get the right branch until it all gets merged)
comes with an extensive regression test which succeeds nicely on
Lispworks and other systems. See the web page for deets. I think any
problems will be eagerly tackled by Luis.
--
Kenny
Why Lisp? http://wiki.alu.org/RtL_Highlight_Film
"I've wrestled with reality for 35 years, Doctor, and I'm happy to state
I finally won out over it."
Elwood P. Dowd, "Harvey", 1950
bradb wrote:
> At the moment I am using CMUCL on Linux & don't know enough to choose
> the correct Lisp for me.
You may well find SBCL to be faster.
I'd be amazed if there aren't already OpenGL bindings for CMUCL/SBCL.
Haskell has HOpenGL and OCaml has LablGL...
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
I managed to get some basics going, but I've been distracted... :)
I decided to learn a bit more about Lisp by writing an ncurses text
editor. It has been enlightening. I intend to get back to 3D soonish
- and then I will have plenty of questions :)
Brad