With the talk about Prevalence, I was curious about a more persistent system
(this could easily apply to Smalltalk as well).
Right now, we simply snapshot the image at whatever time we deem fit. The
system dumps it's state out to a file that can be read later.
Later, on startup, that image is loaded and read entirely into RAM.
This is all well and good in its simplicity, but for large (or perhaps I
should say HUGE images), restarting starts to get burdensome.
Today, particularly with 32 bit machines, we have machines capable of
hosting processes that easily meet the limits of the CPU in terms of process
sizes.
So, using the subtle blows of the sledge hammer, I was hoping someone could
share the problems with something like this.
Take, for example, CMUCL.
Set it up so that the CMUCL image is, say, a 2GB image file.
This image file at start up is simply mmaped into the CMUCL space as a
whole, and now CMUCL has a 2GB memory space mapped read/write/exec. The
detail here, of course, is that we can get an absolute starting address for
CMUCL that will map the entire space. (I think CMUCL actually already does
something like this, at least with Linux -- malloc(1073741824))
Now, the CMUCL user just uses the system as they normally would. When they
quit, CMUCL simply munmaps the space, and in the process this flushes out
any pending page changes to the image.
This kind of thing allows "quick" start up and, ideally, quick shutdown for
huge images filled with millions of objects. Of course you're always going
to have the price of paging in bits of the image at startup.
If your machine has an adequate amount of RAM, swapping should be minimal.
I'm sure there are several downsides. One, is the ever popular Full GC
paging the entire space. Another is the fact that perhaps the Lisp system
can more intelligently manage paging data in and out of RAM than the OSs VM.
The darkest problems of course is that if something "runs amok", if will
happily persist and potentially corrupt the image, or even worse should the
image (or machine) crash, it may only partially be flushed to the disk
image.
Of course, should you have it set to save core files, it can be argued that
the core file is the "real" image, and perhaps you can recover a running
image from that.
None of this tomfoolery would prevent the creation of "fresh, empty" images
of course, those could be done on some routine basis (perhaps this would be
a more full, deeper GC).
But, I'm just considering the concept that with modern machines, we
shouldn't have to continually cross the boundary between the file system and
the memory system of the image. Wasting time not only loading and saving
data, but converting it to a "portable" format...day in and day out.
I don't know if it's practical, possibe, or efficient to create a
transaction system to prevent the image for being corrupted should the
process/machine crash, I suppose it depends on what kind of control one has
over the VM of the OS.
Anyway, just something for the campfire.
Regards,
Will Hartung
(·····@msoft.com)
"Will Hartung" <·····@msoft.com> writes:
> But, I'm just considering the concept that with modern machines, we
> shouldn't have to continually cross the boundary between the file system and
> the memory system of the image. Wasting time not only loading and saving
> data, but converting it to a "portable" format...day in and day out.
>
> I don't know if it's practical, possibe, or efficient to create a
> transaction system to prevent the image for being corrupted should the
> process/machine crash, I suppose it depends on what kind of control one has
> over the VM of the OS.
>
> Anyway, just something for the campfire.
Have a look at Eros and its persistent memory system.
http://www.eros-os.org/
--
__Pascal_Bourguignon__
http://www.informatimago.com/
Do not adjust your mind, there is a fault in reality.
Lying for having sex or lying for making war? Trust US presidents :-(
Will Hartung wrote:
> With the talk about Prevalence, I was curious about a more persistent system
> (this could easily apply to Smalltalk as well).
>
> Right now, we simply snapshot the image at whatever time we deem fit. The
> system dumps it's state out to a file that can be read later.
>
> ..snip...
>
> But, I'm just considering the concept that with modern machines, we
> shouldn't have to continually cross the boundary between the file system and
> the memory system of the image. Wasting time not only loading and saving
> data, but converting it to a "portable" format...day in and day out.
>
> I don't know if it's practical, possibe, or efficient to create a
> transaction system to prevent the image for being corrupted should the
> process/machine crash, I suppose it depends on what kind of control one has
> over the VM of the OS.
Hmm, actually many of the issues you raised might be solved by the EROS
operating system. You might want to have a look at the following paper
titled "Design Evolution of the EROS Single-Level Store"
<http://www.eros-os.org/papers/storedesign2002.pdf>
The issues of image corruption and security could be solved by
implementing a capability system.
The paper titled "Verifying Operating System Security"
<http://www.eros-os.org/papers/MS-CIS-97-26.ps> explains most
of the terminology.
It does need a Lisp. Just to quote the web pages:
"EROS does not yet have a self-hosted development environment. We are
actively working on bringing up a POSIX-compatible environment, but do
not expect that this will make it into the first release. The irony is
that we may get Java development support in first (!)."
Now that would be sad indeed.
-fix
--
------- Markus Fix http://www.bookfix.com/ --------
Don't fight forces, use them. -R.Buckminster Fuller