modern ram allowing servers to have 500Gig ram? to enable a all in ram
database that is super fast, and can recover from stoppages by having a
transaction log written to disk, and every once in a while a whole
image dump to disk?
From: Tim Bradshaw
Subject: Re: all in ram data (al la prevayler or all in ram database) can this be doible with
Date:
Message-ID: <ekq9g8$rio$1$8300dec7@news.demon.co.uk>
On 2006-12-01 21:37:36 +0000, "gavino" <········@yahoo.com> said:
> modern ram allowing servers to have 500Gig ram? to enable a all in ram
> database that is super fast, and can recover from stoppages by having a
> transaction log written to disk, and every once in a while a whole
> image dump to disk?
That pretty much is how modern databases work, except they make serious
efforts to roll the transaction log into the store incrementally, as
you don't want some awful stop-and-copy thing (or even `slow down the
machine and copy' thing) - just the same way GCs try and avoid long
stop-the-world events. Recovery time is also very important - if you
lose the system then whatever picks up the bits needs to be able to do
that very fast, so you don't want to have to roll some gynormous log
forward to recreate state, taking 20 mins.
Unfortunately anything that is transactional still tends to drive disk
systems very hard because you need the transactions to be in
nonvolatile store when they commit (which generally means NV store in
the disk system somewhere and only later on the disks, of course). One
interesting thing is that Sun's ZFS is a copy-on-write filesystem,
meaning that what looks like random writes all over a file might well
actually turn out to be sequential writes to the disk, so it would be
interesting to see how that does for performance for db loads. Other
filesystems may be as well already do this of course, and lots of dbs
still really want to write to raw disk as they don't want some FS
getting in the way with all its annoying extra caching.
But the main thing is that yes, of course they keep everything in core
that they possibly can do, and they can benefit from lots and lots of
real memory - I dunno what big machines have now, the last biggish
machines I dealt with had over 100G, and they weren't very big by,
well, big standards.
--tim