I was talking with a fellow engineer at work about why CMUCL and SBCL
have not been fully ported to Windows. He mentioned that it is
difficult to port most Lisp implementations because of difference
between the Linux and Windows memory models and how they are used for
garbage collection. Can someone shed some light on what these
differences are.
Thanks,
Mike
"skibud2" <·············@gmail.com> writes:
> I was talking with a fellow engineer at work about why CMUCL and SBCL
> have not been fully ported to Windows. He mentioned that it is
> difficult to port most Lisp implementations because of difference
> between the Linux and Windows memory models and how they are used for
> garbage collection. Can someone shed some light on what these
> differences are.
http://www.google.com/search?q=comparison+%22memory+management%22+unix+windows
--
__Pascal Bourguignon__
"skibud2" <·············@gmail.com> writes:
> I was talking with a fellow engineer at work about why CMUCL and SBCL
> have not been fully ported to Windows. He mentioned that it is
> difficult to port most Lisp implementations because of difference
> between the Linux and Windows memory models and how they are used for
> garbage collection.
I wouldn't claim to be an expert in Windows programming, but my
understanding is that the difference in memory models is far from the
difficult bit in a port of a Unixoid Lisp implementation to Windows;
exception/signal handling seems harder to deal with. That said, I
also dispute your premise; both SBCL and CMUCL have been "fully"
ported to windows, and though it is likely that bugs remain in the
port you can test this for yourself in the case of SBCL.
Christophe
Christophe Rhodes wrote:
> I also dispute your premise; both SBCL and CMUCL have been "fully"
> ported to windows, and though it is likely that bugs remain in the
> port you can test this for yourself in the case of SBCL.
The SBCL ports page (http://sbcl.sourceforge.net/platform-table.html)
claims the Windows port is "in progress". Is that page out of date?
I have it on good authority that the SBCL Windows port (0.9.12),
as it stands, prints a disclaimer on start up which says
"This is an experimental prerelease support of the Windows platform:
use at your own risk. Your Kitten of Death awaits!"
Did they fix the bugs and forget to remove the disclaimer?
Just curious. I'd like to see a Windows port as much as anybody.
Robert Dodier
"Robert Dodier" <·············@gmail.com> writes:
> Christophe Rhodes wrote:
>
>> I also dispute your premise; both SBCL and CMUCL have been "fully"
>> ported to windows, and though it is likely that bugs remain in the
>> port you can test this for yourself in the case of SBCL.
>
> The SBCL ports page (http://sbcl.sourceforge.net/platform-table.html)
> claims the Windows port is "in progress". Is that page out of date?
No, not at all, except in as much as the binary linked to there is
four months old (and yet can be used to build a new one). But if you
stop and think a little bit, you don't get any less of a warranty and
so on from the Windows version than you do from any other one.
The linked binary is mostly flagged as "in progress" because it was
incapable of running SBCL's own regression test suite, because not
enough of the infrastructure was working. Although I don't follow the
progress of the Windows work terribly closely, my understanding is
that this is no longer the case: on Windows SBCL runs its test suite
and passes "most" of the tests.
> I have it on good authority that the SBCL Windows port (0.9.12),
> as it stands, prints a disclaimer on start up which says
> "This is an experimental prerelease support of the Windows platform:
> use at your own risk. Your Kitten of Death awaits!"
You are talking about a four-month-old version. (Not that that has
changed, mind you, but ignoring four months of work isn't terribly
helpful in this discussion.)
> Did they fix the bugs
How do you know that there are bugs?
> and forget to remove the disclaimer?
no; it stays flagged as "experimental" until enough people send
success reports that we can feel confident that it doesn't make
anyone's screen explode.
> Just curious. I'd like to see a Windows port as much as anybody.
If you actually mean that, you can help out by trying to use it and
reporting problems or the lack of them. (If instead you mean "I'd
like to see a Windows port but not enough to do any work" then by all
means don't do anything.)
Christophe
On Thu, 31 Aug 2006 10:15:48 +0200, Christophe Rhodes <·····@cam.ac.uk>
wrote:
>
> If you actually mean that, you can help out by trying to use it and
> reporting problems or the lack of them. (If instead you mean "I'd
> like to see a Windows port but not enough to do any work" then by all
> means don't do anything.)
>
> Christophe
I summarize this as fun to experiment with, but too unstable
for production work. I think I will get a CVS version to play with.
My experience with open-source is that they show interest for a while
then move on to new turf leaving things half done. I am glad this is
not the case for the windows version (yet). All the same I would be happy
to try some production code on it and report any bugs..
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Christophe Rhodes wrote:
> > Just curious. I'd like to see a Windows port as much as anybody.
>
> If you actually mean that, you can help out by trying to use it and
> reporting problems or the lack of them.
Tell you what. If you or someone posts a new Windows binary, I'll try
it.
Robert Dodier
skibud2 wrote:
> I was talking with a fellow engineer at work about why CMUCL and SBCL
> have not been fully ported to Windows. He mentioned that it is
> difficult to port most Lisp implementations because of difference
> between the Linux and Windows memory models and how they are used for
> garbage collection. Can someone shed some light on what these
> differences are.
When i have the todo list right in mind the largest problem is that
under Win32 you can't allocate a fixed address range and so loaded
compiled code must be able relocateable. The current implementation of
the SBCL compiler is not.
Signal handling isn't a problem because Win32 has no signals. Using
signals is a bad unix coding style and should be covered with higher
level abstractions.
Exception handling was also one on the todo list marked as "insane".
But there are dozens of other things i exepct to have problems serious
enough to kall the windows port a joke. When i see all the problems
with unicode then it is far away from professional usage.
It's a pitty, there is still no good cost-free lisp for the worlds most
important plattform.
On 2006-08-31, llothar <·······@web.de> wrote:
> When i have the todo list right in mind the largest problem is that
> under Win32 you can't allocate a fixed address range and so loaded
> compiled code must be able relocateable. The current implementation of
> the SBCL compiler is not.
It is true that Alastair Bridgewater listed that as a potential problem.
Although I do not know the technical reasons for his comment, let me
note that so far nobody has turned up who actually had problems with
SBCL on Windows because of this, so it does not seem to be a big problem
at this point.
Also note that there is a patch for relocation that mostly works, both
on Windows and Unix (well, Linux and BSD).
d.
David Lichteblau <········@lichteblau.com> wrote:
+---------------
| llothar <·······@web.de> wrote:
| > When i have the todo list right in mind the largest problem is that
| > under Win32 you can't allocate a fixed address range and so loaded
| > compiled code must be able relocateable. The current implementation of
| > the SBCL compiler is not.
|
| It is true that Alastair Bridgewater listed that as a potential problem.
|
| Although I do not know the technical reasons for his comment, let me
| note that so far nobody has turned up who actually had problems with
| SBCL on Windows because of this, so it does not seem to be a big problem
| at this point.
+---------------
Actually, it depends on exactly what *other* Windows programs
you have loaded on your system and in what order they start!!
Carl Shapiro ran into this problem with his CMUCL port: there was
some [name escapes me] Windows add-on that, when started at boot
time due to a registry entry, grabbed a hunk of absolute addresses
right in the *middle* of the address space, thus preventing CMUCL
from ever loading. [Part of CMUCL's "lisp.core" image has to load
at an absolute virtual address.]
He had some hacky solution [starting a CMUCL before the other tool?]
which I don't remember off-hand, but in any case it *has* been a
"big problem" at least for one person.
-Rob
p.s. Yes, that was CMUCL and not SBCL, but I dare say the problem
would likely be the same with the latter.
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
On 2006-09-01, Rob Warnock <····@rpw3.org> wrote:
> Actually, it depends on exactly what *other* Windows programs
> you have loaded on your system and in what order they start!!
> Carl Shapiro ran into this problem with his CMUCL port: there was
> some [name escapes me] Windows add-on that, when started at boot
> time due to a registry entry, grabbed a hunk of absolute addresses
> right in the *middle* of the address space, thus preventing CMUCL
> from ever loading. [Part of CMUCL's "lisp.core" image has to load
> at an absolute virtual address.]
Thank you for that information. However, it would be much more useful
together with instructions for reproducing it. What add-on was that?
What exact addresses did it grab?
Even if my relocation patch went in, without a proper bug report on,
say, sbcl-devel, nobody would know whether such user problem has been
fixed by it. For example, the patch implements relocation of dynamic
space but not of the linkage table.
And I doubt anyone is going to implement and commit a patch to record
fixups to the linkage table and re-build the linkage table on startup
unless it is known to be necessary.
d.
On 31 Aug 2006 07:46:09 -0700, "llothar" <·······@web.de> wrote:
>
>skibud2 wrote:
>> I was talking with a fellow engineer at work about why CMUCL and SBCL
>> have not been fully ported to Windows. He mentioned that it is
>> difficult to port most Lisp implementations because of difference
>> between the Linux and Windows memory models and how they are used for
>> garbage collection. Can someone shed some light on what these
>> differences are.
>
>When i have the todo list right in mind the largest problem is that
>under Win32 you can't allocate a fixed address range and so loaded
>compiled code must be able relocateable. The current implementation of
>the SBCL compiler is not.
??? The first argument to VirtualAlloc() is the desired starting
address for the memory block. It will be rounded to the nearest page
boundary but that shouldn't be any problem.
I don't know enough Lisp to be useful to the project, but if what you
say about their problems is true, somebody there should definitely
read up on Window's memory management API - it's orders of magnitude
more flexible than the Unix API. Memory management is one thing
Window's does *far* better than Unix.
>Signal handling isn't a problem because Win32 has no signals. Using
>signals is a bad unix coding style and should be covered with higher
>level abstractions.
Windows has Posix signals but I don't know why anyone would want to
use them. Windows uses native exceptions or events for all the stuff
that Unix signals - the Posix signal emulation is a messy abstraction
layer on top of it.
George
--
for email reply remove "/" from address
On 2006-09-01 19:27:37 +0100, George Neuner <·········@comcast.net> said:
> Memory management is one thing
> Window's does *far* better than Unix.
I suspect you know little of the memory management strategies and APIs
of many modern Unices then (no, *not* Linux though it may have good
APIs too). When you've had a huge virtual address space, significantly
large physical address space, and large numbers of processors you
better design something that doesn't suck.
--tim
On Fri, 1 Sep 2006 21:59:17 +0100, Tim Bradshaw <···@tfeb.org> wrote:
>On 2006-09-01 19:27:37 +0100, George Neuner <·········@comcast.net> said:
>
>> Memory management is one thing
>> Window's does *far* better than Unix.
>
>I suspect you know little of the memory management strategies and APIs
>of many modern Unices.
I suspect you're wrong. I'm quite familiar with all the advanced
capabilities of mmap, mprotect, shm_, et al.
Windows' provides all the same functionality. Additionally Windows
allows debugger-like manipulation of the memory map of any process for
which you have a valid handle and security descriptor (using the same
calls), allows separate address space reservation and backing page
allocation, simple checking of access permissions for any virtual
address within the process, userland monitoring and retrieval of
"dirty" page status for writable pages, and notification when global
high/low memory usage watermarks are passed.
I know of no Unix that separates page allocation from address
reservation, nor any which simply and easily reports page "dirty"
status to userland code - if you want that information you must catch
the write access fault for each page and track it yourself. Windows
does it transparently and more efficiently for you.
Windows reports faults by exception rather than by out-of-band signals
so in many cases it is possible to write simpler local error handling
code. When more a global view is necessary to handle the error, SEH
provides debugger-like context information which enables a handler to
restore the (possibly modified) context and restart/continue the
interrupted code.
Obviously YMMV. If you prefer Unix, by all means use it. I like Unix
just fine for a lot of things but really complicated memory management
tasks are not among them.
>When you've had a huge virtual address space, significantly
>large physical address space, and large numbers of processors you
>better design something that doesn't suck.
Touche'. Windows doesn't (yet) run on big multiprocessors. But
recall that the NT kernel was designed by people from DEC who had lots
of experience with multiprocessors. Micro$qish artificially limited
NT to 4 cpus - there was no real technical reason that the NT *kernel*
could not have supported more - in fact Intel ran NT on their 16 cpu
server noting that the kernel required only minor modifications to
support it. The various real limitations of Windows' non-kernel code
are subjects for other discussions.
The memory management API changed little in 2K/XP/2003 adding
essentially only watermarks, user mode dirty page tracking and
windowing extensions for >4GB memory access. Underneath the API,
multiprocessor performance was significantly increased. The 2K/XP
kernel removed many of the spin-locks that limited NT's performance on
multiprocessors, reduced both lock contention and lock hold time on
all code paths, and introduced smarter VM page caching algorithms.
2003 reduced lock contention even further and again improved VM page
caching.
Moreover, I didn't say Windows was better than Unix ... I said
Windows' memory management was better than Unix's memory management -
a comparison of respective subsystems. By "better" I mean more
flexible and easier to use for complicated tasks (performance is
another issue but meaningful comparisons are extremely complicated and
depend on what exactly is being measured). I stand by my "better"
judgement with regard to 32-bit systems - I will concede that I
haven't yet worked enough with 64-bit versions of either to draw
comparisons there.
>--tim
George
--
for email reply remove "/" from address
George Neuner wrote:
> .... a lot interesting stuff ....
Do have some API calls for the userland dirty page access so that i can
find my way through the MSDN ?
I only want to add the windows kernel is far superior to the
linux/bsd/solaris kernels in the was that windows events, file access,
thread and IPC mechanisms are mixed together.
And also the asynchronous API seems to be much more efficient.
On 2 Sep 2006 02:07:49 -0700, "llothar" <·······@web.de> wrote:
>George Neuner wrote:
>> .... a lot interesting stuff ....
>
>Do have some API calls for the userland dirty page access so that i can
>find my way through the MSDN ?
Reserve/Commit the region using VirtualAlloc() with the
MEM_WRITE_WATCH flag. Then use GetWriteWatch() to read the dirty
status of the region's pages and ResetWriteWatch() to clear it. You
can also read and clear in one step with a flag to GetWriteWatch().
>I only want to add the windows kernel is far superior to the
>linux/bsd/solaris kernels in the was that windows events, file access,
>thread and IPC mechanisms are mixed together.
>
>And also the asynchronous API seems to be much more efficient.
I don't know whether Windows is more efficient than Unix but I
definitely think it is easier to use for adding background I/O to an
otherwise linear task - the completion event model meshes very well
with threading.
In the case of the more traditional asynchronous "server" ... handling
multiple I/O streams in a single thread ... Windows overlapped I/O
quickly becomes more complex than using select() in Unix because of
the extra programmer data structure needed for each stream and because
Windows distinguishes sockets from files.
However, it's is not terribly difficult to make wrapper classes to
hide the differences and reduce the complexity of Windows' overlapped
I/O to that of Unix's non-blocking I/O.
George
--
for email reply remove "/" from address
llothar wrote:
> George Neuner wrote:
> > .... a lot interesting stuff ....
>
> Do have some API calls for the userland dirty page access so that i can
> find my way through the MSDN ?
I think you have unconciously hit the nail on the head. No-one person
really understands Windows and the various Unix flavours treatment of
this well enough to be able to make a sound judgement. All the
information is hidden in the back of the MSDN or obscure man pages and
has never really been brought together.
I remember Linus Torvalds trying to debate this once with someone and
largely failing because both parties didn't know enough about each
others systems.
On 4 Sep 2006 03:00:34 -0700, "Rob Thorpe"
<·············@antenova.com> wrote:
>llothar wrote:
>> George Neuner wrote:
>> > .... a lot interesting stuff ....
>>
>> Do have some API calls for the userland dirty page access so that i can
>> find my way through the MSDN ?
>
>I think you have unconciously hit the nail on the head. No-one person
>really understands Windows and the various Unix flavours treatment of
>this well enough to be able to make a sound judgement. All the
>information is hidden in the back of the MSDN or obscure man pages and
>has never really been brought together.
At least MSDN provides 99.44% of the information the application
programmer needs, cross indexed, hyperlinked, and with examples.
Though you still need to look in the component development kits for
some of the esoteric stuff, it's safe to assume anything not in MSDN
is for mature audiences only.
In contrast, the man pages on many Unix and Unix-like systems are hit
and miss. Frequently you look up some lesser used function and find
just a sentence about its use and a listing of its flags and error
codes with no explanations of what they mean. Or better yet, you find
some data structure required to use the function only partially
documented (and that in a different page).
The real Unixes are, IME, uniformly better than the free ones WRT man
page quality (AT&T and Solaris stand out in my recollection), but even
in the best of them you can find numerous omissions if you look.
In fairness, MSDN was intended to *be* the main source of Windows
documentation whereas Unix's man pages were intended to be only a
quick reference. The problem is that nowadays the in-depth
documentation is either missing entirely (BSD/Linux) or, in many
cases, provided in a flat electronic form that can't easily be
searched.
George
--
for email reply remove "/" from address
George Neuner wrote:
> On 4 Sep 2006 03:00:34 -0700, "Rob Thorpe"
> <·············@antenova.com> wrote:
>
> >llothar wrote:
> >> George Neuner wrote:
> >> > .... a lot interesting stuff ....
> >>
> >> Do have some API calls for the userland dirty page access so that i can
> >> find my way through the MSDN ?
> >
> >I think you have unconciously hit the nail on the head. No-one person
> >really understands Windows and the various Unix flavours treatment of
> >this well enough to be able to make a sound judgement. All the
> >information is hidden in the back of the MSDN or obscure man pages and
> >has never really been brought together.
>
> At least MSDN provides 99.44% of the information the application
> programmer needs, cross indexed, hyperlinked, and with examples.
> Though you still need to look in the component development kits for
> some of the esoteric stuff, it's safe to assume anything not in MSDN
> is for mature audiences only.
MSDN is very through, though slow to use if you use it from the MS site
and extremely large if you install it on your own machine.
> In contrast, the man pages on many Unix and Unix-like systems are hit
> and miss. Frequently you look up some lesser used function and find
> just a sentence about its use and a listing of its flags and error
> codes with no explanations of what they mean. Or better yet, you find
> some data structure required to use the function only partially
> documented (and that in a different page).
Yes, unfortunately.
> The real Unixes are, IME, uniformly better than the free ones WRT man
> page quality (AT&T and Solaris stand out in my recollection), but even
> in the best of them you can find numerous omissions if you look.
In "Linux" system the GNU Info pages, and their HTML mirrors, are the
best documentation of the Gnu parts of the system, and they are much
more informative than man pages are anywhere. But that's only one part
of the distribution, not as large a part as once it was.
The rest is hit and miss, many of the small single person stuff has OK
documentation. KDE and GNOME have some good documentation, but with
holes. X's documentation is OK if not quite up to date.
The Kernel and the utilities around it have the most irritating docs in
my view. They're often incomplete and out of date. Often the
documentation is considered to be the code, which is a poor situation.
> In fairness, MSDN was intended to *be* the main source of Windows
> documentation whereas Unix's man pages were intended to be only a
> quick reference. The problem is that nowadays the in-depth
> documentation is either missing entirely (BSD/Linux) or, in many
> cases, provided in a flat electronic form that can't easily be
> searched.
Yes.
On 2006-09-02 08:42:16 +0100, George Neuner <·········@comcast.net> said:
>
> I suspect you're wrong. I'm quite familiar with all the advanced
> capabilities of mmap, mprotect, shm_, et al.
> Windows' provides all the same functionality. Additionally Windows
> allows debugger-like manipulation of the memory map of any process for
> which you have a valid handle and security descriptor (using the same
> calls), allows separate address space reservation and backing page
> allocation, simple checking of access permissions for any virtual
> address within the process, userland monitoring and retrieval of
> "dirty" page status for writable pages, and notification when global
> high/low memory usage watermarks are passed.
Ok, I think we're talking at cross purposes, sorry. It probably is the
case that Windows offers, at least portably (though I'm not sure what
that means in this context), a better memory-management API than Unix
systems. However I suspect if you're willing to use nonportable APIs
on, say, Solaris, you can do an awful lot of what you're after (for
instance there is a whole per-process overcommit API for Solaris
somewhere, which is all about separating page allocation from address
reservation, though I'm not sure how fine-grained it is).
But..
> Touche'. Windows doesn't (yet) run on big multiprocessors. But
> recall that the NT kernel was designed by people from DEC who had lots
> of experience with multiprocessors.
I don't think they did, or if they did they were rather horrible
machines such as multiprocessor vaxen. Machines like modern
multiprocessor boxes just didn't really exist in that timescale
(multiprocessors existed of course, but the performance tradeoffs were
entirely different than they are now). I suspect that only Sun and SGI
have ever done a fully commercial reasonably scalable OS, though
probably IBM should be added to that list.
> Moreover, I didn't say Windows was better than Unix ... I said
> Windows' memory management was better than Unix's memory management -
> a comparison of respective subsystems. By "better" I mean more
> flexible and easier to use for complicated tasks (performance is
> another issue but meaningful comparisons are extremely complicated and
> depend on what exactly is being measured). I stand by my "better"
> judgement with regard to 32-bit systems - I will concede that I
> haven't yet worked enough with 64-bit versions of either to draw
> comparisons there.
So I think what I'm trying to say is that: yes it's probably the case
that Windows' MM API is better than Unix's, and certainly it is better
than what you have to use if you want to write portable Unix
applications. But the underlying MM system is nothing like as
sophisticated. It remains to be seen whether they've exposed things in
the API which will hurt them later, of course, by committing them to
particular implementation strategies.
--tim
> Ok, I think we're talking at cross purposes, sorry. It probably is the
Well i don't think that cross platform means lowest common denominator.
It should be the best you could get from the best plattfrom and the try
to get at least something comparable from the other.
But cross platform can also mean a lot of if defs. It's the fucking
java that made a one source (in text or compiled binary form) for all
systems popular, but this was never the meaning of crsss platform.
Greetings, from a totally drunken programmer
On Sat, 2 Sep 2006 19:23:37 +0100, Tim Bradshaw <···@tfeb.org> wrote:
>On 2006-09-02 08:42:16 +0100, George Neuner <·········@comcast.net> said:
>>
>
>> Touche'. Windows doesn't (yet) run on big multiprocessors. But
>> recall that the NT kernel was designed by people from DEC who had lots
>> of experience with multiprocessors.
>
>I don't think they did, or if they did they were rather horrible
>machines such as multiprocessor vaxen. Machines like modern
>multiprocessor boxes just didn't really exist in that timescale
I don't see how that's relevant. IFAICT, it would have been harder to
write a decent multiprocessor OS for VAX than for the i386 - VAX had
many more non-interruptible and non-restartable instructions. VAX
Ultrix didn't have kernel threads but DecStation Ultrix (MIPS/x86) did
so I expect they had experience with that too.
>So I think what I'm trying to say is that: yes it's probably the case
>that Windows' MM API is better than Unix's, and certainly it is better
>than what you have to use if you want to write portable Unix
>applications. But the underlying MM system is nothing like as
>sophisticated. It remains to be seen whether they've exposed things in
>the API which will hurt them later, of course, by committing them to
>particular implementation strategies.
This one I'll give you with the caveat that Windows has consistently
improved its VMM system in every professional 32-bit version ... that
is in NT/2K/XP/2003.
The VMM system in 2003 server edition is self tuning and performs
quite well. 2K server also had some self tuning ability but it didn't
work nearly so well as 2003. In all the other versions you have to
tune the VMM manually. It's a real chore but worth doing for a server
with a stable application mix. It's essentially impossible for a
general system. There are some end-user tools like CacheSet
(www.sysinternals.com) that along with performance monitoring will
help an advanced user tweak her system, but really dialing in optimal
performance is a black registry art.
I've heard, but not confirmed, that Vista will have the self tuning
VMM in all versions (including the home editions). I hope it is true.
I know Tim is aware of this, but for others who may be interested, I
will also mention that, out of the box, the VMM in server editions is
tuned differently than in workstation editions so *some* of the
comparisons that have been made between Unix and Windows have been
unfair. Windows workstations are, after all, single user systems.
For a meaningful comparison between real Unix (e.g., Solaris, Ultrix,
AIX, etc.) and Windows, you really have to use a multiuser server
edition of Windows - NT4 Terminal Server, 2K server or 2003 server.
George
--
for email reply remove "/" from address
George Neuner wrote:
> I don't see how that's relevant. IFAICT, it would have been harder to
> write a decent multiprocessor OS for VAX than for the i386 - VAX had
> many more non-interruptible and non-restartable instructions. VAX
> Ultrix didn't have kernel threads but DecStation Ultrix (MIPS/x86) did
> so I expect they had experience with that too.
The tradeoffs when designing (commercial) multiprocessor OSs have
changed radically over time - as memory and peripherals become
inevitably `further away' from processors, as the execution models of
processors become more complex, and finally as the number of processors
or cores has increased dramatically. a 4 CPU VAX system is nothing like
a big modern system.
(I'm saying `commercial' above to avoid the riposte - not from you - of
`oh yes, but look at this weird thousand-CPU machine from 1979' (like
the CM1 or something) - those boxes were typically special-purpose
`scientific' (in the broad sense, not just physics-simulation machines)
systems where application programmers were willing to jump through lots
of hoops to get their code to run.)
--tim
George Neuner wrote:
> On 31 Aug 2006 07:46:09 -0700, "llothar" <·······@web.de> wrote:
>
> >
> >skibud2 wrote:
> >> I was talking with a fellow engineer at work about why CMUCL and SBCL
> >> have not been fully ported to Windows. He mentioned that it is
> >> difficult to port most Lisp implementations because of difference
> >> between the Linux and Windows memory models and how they are used for
> >> garbage collection. Can someone shed some light on what these
> >> differences are.
> >
> >When i have the todo list right in mind the largest problem is that
> >under Win32 you can't allocate a fixed address range and so loaded
> >compiled code must be able relocateable. The current implementation of
> >the SBCL compiler is not.
>
> ??? The first argument to VirtualAlloc() is the desired starting
> address for the memory block. It will be rounded to the nearest page
> boundary but that shouldn't be any problem.
VirtualAlloc is allowed to return null if it cannot reserve the desired
pages.
In any case, the lisp heap isn't allocated dynamically like that. It
resides in a file, so you would use the file mapping functions. If you
look at `MapViewOfFileEx', you'll see this caveat:
``While it is possible to specify an address that is safe now (not
used by the operating system), there is no guarantee that the address
will remain safe over time. Therefore, it is better to let the
operating system choose the address. In this case, you would not store
pointers in the memory mapped file, you would store offsets from the
base of the file mapping so that the mapping can be used at any
address.''
There are obviously many solutions to this problem. Making the SBCL
compiler emit position independent code is a good start.
> I don't know enough Lisp to be useful to the project, but if what you
> say about their problems is true, somebody there should definitely
> read up on Window's memory management API - it's orders of magnitude
> more flexible than the Unix API. Memory management is one thing
> Window's does *far* better than Unix.
Agreed. Windows also excels in process control.
On 5 Sep 2006 09:45:29 -0700, "Joe Marshall" <··········@gmail.com>
wrote:
>
>George Neuner wrote:
>
>> ??? The first argument to VirtualAlloc() is the desired starting
>> address for the memory block. It will be rounded to the nearest page
>> boundary but that shouldn't be any problem.
>
>VirtualAlloc is allowed to return null if it cannot reserve the desired
>pages.
>
>In any case, the lisp heap isn't allocated dynamically like that. It
>resides in a file, so you would use the file mapping functions. If you
>look at `MapViewOfFileEx', you'll see this caveat:
> ``While it is possible to specify an address that is safe now (not
>used by the operating system), there is no guarantee that the address
>will remain safe over time. Therefore, it is better to let the
>operating system choose the address. In this case, you would not store
>pointers in the memory mapped file, you would store offsets from the
>base of the file mapping so that the mapping can be used at any
>address.''
My gripe is that file mapping is incompatible with other uses of
VirtualAlloc(), including dirty page monitoring. But it seems to me
that for implementing a GC'd heap, I would care more about using the
dirty page monitoring than about transparent writeback to the disk. In
fact, I think I would deliberately *not* use a file mapping for a
modifiable heap because that might cause disk writes for creation of
or updates to objects that may not survive the next collection.
Better to allocate the heap area dynamically, use all the facilities
of the VMM, and read/write the image file deliberately.
YMMV.
George
--
for email reply remove "/" from address
George Neuner wrote:
> On 5 Sep 2006 09:45:29 -0700, "Joe Marshall" <··········@gmail.com>
> wrote:
>
> >
> >George Neuner wrote:
> >
> >> ??? The first argument to VirtualAlloc() is the desired starting
> >> address for the memory block. It will be rounded to the nearest page
> >> boundary but that shouldn't be any problem.
> >
> >VirtualAlloc is allowed to return null if it cannot reserve the desired
> >pages.
> >
> >In any case, the lisp heap isn't allocated dynamically like that. It
> >resides in a file, so you would use the file mapping functions. If you
> >look at `MapViewOfFileEx', you'll see this caveat:
> > ``While it is possible to specify an address that is safe now (not
> >used by the operating system), there is no guarantee that the address
> >will remain safe over time. Therefore, it is better to let the
> >operating system choose the address. In this case, you would not store
> >pointers in the memory mapped file, you would store offsets from the
> >base of the file mapping so that the mapping can be used at any
> >address.''
>
> My gripe is that file mapping is incompatible with other uses of
> VirtualAlloc(), including dirty page monitoring. But it seems to me
> that for implementing a GC'd heap, I would care more about using the
> dirty page monitoring than about transparent writeback to the disk. In
> fact, I think I would deliberately *not* use a file mapping for a
> modifiable heap because that might cause disk writes for creation of
> or updates to objects that may not survive the next collection.
> Better to allocate the heap area dynamically, use all the facilities
> of the VMM, and read/write the image file deliberately.
For the dynamic heap, yes, but the saved heap image would mostly be
`pure' and wouldn't need collection.