Hi,
I'm just curious if these issues are addressed for developing high
traffic web sites in CL:
1) Scaling out: have a distributed in-memory cache (eg. memcached)
that sits between the app and the db.
2) Hardware fail-safe: Make use of the said cache/db to store session
objects.
I remember Marco Baringer mentioned in an old thread that he was going
to implement fast session serialization, which could eventually lead
to these things. Is it available in UCW already? Is it done in other
servers/frameworks?
Thanks!
Bob
(message (Hello 'bob)
(you :wrote :on '(28 Apr 2007 12:26:48 -0700))
(
b> 1) Scaling out: have a distributed in-memory cache (eg. memcached)
b> that sits between the app and the db.
personally i have no experience with building high traffic web sites yet,
but i strongly believe that it would be better to use in-memory simple
cache, like just some hash table.
recently there was a question on Y Combinator news: "What are the best
non-database solutions you've seen? What did Viaweb use?" --
http://news.ycombinator.com/comments?id=14605
Paul Graham replied: "Keep everything in memory in the usual sort of data
structures (e.g. hash tables). Save changes to disk, but never read from
disk except at startup." then he also made a note that it's was easy to
scale this solution to multiple servers simply partitioning servers among
clients.
Paul Graham seems to have experience with high-traffic web sites :)
memcached thing is actually PHP style -- PHP is very limited, so it needs an
external entity for caching.
with Common Lisp you can just use internal data structures for it, and you
can make it easily scalable w/o complex 'distibuted hash table' thing (that
introduces some overhead by the way) if you'll analyse your data processing.
b> I remember Marco Baringer mentioned in an old thread that he was going
b> to implement fast session serialization,
b> which could eventually lead to these things. Is it available in UCW
b> already? Is it done in other servers/frameworks?
most likely Marco Baringer was going to implement CONTIUNATION
serialization, that's kinda much more complex that session serialization if
you are not useing continuations.
are there really any problems with session serialization?
and actually i don't think there's really any need in session serialization.
simply do not keep anything important in the session. if you'll have a
hardware failure and sesssion will be lost, person using it will need to
re-login, for example. i don't think it's a big deal, web applications were
never meant to be anyhow reliable.. and actually do you really have hardware
failures each day, so it's an issue?
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"I am everything you want and I am everything you need")
On Apr 29, 5:34 am, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> (message (Hello 'bob)
> (you :wrote :on '(28 Apr 2007 12:26:48 -0700))
> (
>
> b> 1) Scaling out: have a distributed in-memory cache (eg. memcached)
> b> that sits between the app and the db.
>
> personally i have no experience with building high traffic web sites yet,
> but i strongly believe that it would be better to use in-memory simple
> cache, like just some hash table.
>
> recently there was a question on Y Combinator news: "What are the best
> non-database solutions you've seen? What did Viaweb use?" -- http://news.ycombinator.com/comments?id=14605
>
> Paul Graham replied: "Keep everything in memory in the usual sort of data
> structures (e.g. hash tables). Save changes to disk, but never read from
> disk except at startup." then he also made a note that it's was easy to
> scale this solution to multiple servers simply partitioning servers among
> clients.
> Paul Graham seems to have experience with high-traffic web sites :)
>
> memcached thing is actually PHP style -- PHP is very limited, so it needs an
> external entity for caching.
> with Common Lisp you can just use internal data structures for it, and you
> can make it easily scalable w/o complex 'distibuted hash table' thing (that
> introduces some overhead by the way) if you'll analyse your data processing.
>
> b> I remember Marco Baringer mentioned in an old thread that he was going
> b> to implement fast session serialization,
> b> which could eventually lead to these things. Is it available in UCW
> b> already? Is it done in other servers/frameworks?
>
> most likely Marco Baringer was going to implement CONTIUNATION
> serialization, that's kinda much more complex that session serialization if
> you are not useing continuations.
> are there really any problems with session serialization?
>
> and actually i don't think there's really any need in session serialization.
> simply do not keep anything important in the session. if you'll have a
> hardware failure and sesssion will be lost, person using it will need to
> re-login, for example. i don't think it's a big deal, web applications were
> never meant to be anyhow reliable.. and actually do you really have hardware
> failures each day, so it's an issue?
>
> )
> (With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
> "I am everything you want and I am everything you need")
Thanks for the reply! :)
I guess I could use sticky session on the load balancer. However,
there's still the problem with caching database queries in memory once
they grow too big for in-memory data structures and have to be stored
on disk constantly. Caching them in local hash tables would result in
a lot of duplications. This is a fairly common problem now a days.
There could be another solution, but I don't know if it would be an
overkill for web farms; We could use OS-level distributed shared
memory, such as in a beowulf cluster. So basically, each thread would
be locked onto its own CPU, while the user data can be shared among
the different machines transparently. From the outside it would
appear as a single multi-processor machine with a single memory
address space. DragonFly BSD has some features that makes it easy to
do this over a slow and loosely-connected ethernet (they aim to do
this efficiently over the internet in the future!). Theoretically,
right now you can run all the NetBSD ports on it, but I'm not sure how
well the SBCL threads work in this manner, or if it runs at all.
Cheers!
Bob
(message (Hello 'bob)
(you :wrote :on '(29 Apr 2007 12:48:00 -0700))
(
b> I guess I could use sticky session on the load balancer. However,
b> there's still the problem with caching database queries in memory once
b> they grow too big for in-memory data structures and have to be stored
b> on disk constantly. Caching them in local hash tables would result in
b> a lot of duplications. This is a fairly common problem now a days.
lot of duplications where? in memory of different machines, that is actually
what caching is meant to do -- have data closer. if duplications on same
machine, that should be eliminated..
b> address space. DragonFly BSD has some features that makes it easy to
b> do this over a slow and loosely-connected ethernet (they aim to do
b> this efficiently over the internet in the future!).
i think HT over network will introduce at least order-of-magnitude overhead
comparing to direct hash tables.
but it might still acceptable if it introduces benefits comparing to RDBMS
performance.
i didn't evaluate memcached performance, but i saw that RDBMS performance
was killed with network communications -- sending each packet introduces
some overhead, and applications was sending megatons of those packets.. so
time was spent not on web server or database, but on OS system time..
certainly, network HT might have more optimal protocol than RDBMS over
network, or you can fine-tune requests better..
but you are going to check automated shared memory. i don't see how it can
work fast :)
if you'll do testing, please post here some benchmarking results.
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"I am everything you want and I am everything you need")
On Apr 29, 2:07 pm, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> (message (Hello 'bob)
> (you :wrote :on '(29 Apr 2007 12:48:00 -0700))
> (
>
> b> I guess I could use sticky session on the load balancer. However,
> b> there's still the problem with caching database queries in memory once
> b> they grow too big for in-memory data structures and have to be stored
> b> on disk constantly. Caching them in local hash tables would result in
> b> a lot of duplications. This is a fairly common problem now a days.
>
> lot of duplications where? in memory of different machines, that is actually
> what caching is meant to do -- have data closer. if duplications on same
> machine, that should be eliminated..
>
> b> address space. DragonFly BSD has some features that makes it easy to
> b> do this over a slow and loosely-connected ethernet (they aim to do
> b> this efficiently over the internet in the future!).
>
> i think HT over network will introduce at least order-of-magnitude overhead
> comparing to direct hash tables.
> but it might still acceptable if it introduces benefits comparing to RDBMS
> performance.
> i didn't evaluate memcached performance, but i saw that RDBMS performance
> was killed with network communications -- sending each packet introduces
> some overhead, and applications was sending megatons of those packets.. so
> time was spent not on web server or database, but on OS system time..
> certainly, network HT might have more optimal protocol than RDBMS over
> network, or you can fine-tune requests better..
>
> but you are going to check automated shared memory. i don't see how it can
> work fast :)
> if you'll do testing, please post here some benchmarking results.
>
> )
> (With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
> "I am everything you want and I am everything you need")
On Apr 29, 2:07 pm, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> (message (Hello 'bob)
> (you :wrote :on '(29 Apr 2007 12:48:00 -0700))
> (
>
> b> I guess I could use sticky session on the load balancer. However,
> b> there's still the problem with caching database queries in memory once
> b> they grow too big for in-memory data structures and have to be stored
> b> on disk constantly. Caching them in local hash tables would result in
> b> a lot of duplications. This is a fairly common problem now a days.
>
> lot of duplications where? in memory of different machines, that is actually
> what caching is meant to do -- have data closer. if duplications on same
> machine, that should be eliminated..
The idea of memcached is to do make use of existing memory for as
cheaply as possible, retrieving data from multiple machines through a
centralized interface, while offering orders of magnitudes of
performance over disks. By duplication I mean each machine stores
disk data locally in memory where you could have a lot of overlaps
between machines. In many memcached setups, each existing web server
sets aside a slice of the memory for memcached, to be shared among all
servers. news.yc is targeted toward a niche group, and Paul Graham
mentioned that he hope to never expose news.yc to reddit kind of
traffic.
>
> b> address space. DragonFly BSD has some features that makes it easy to
> b> do this over a slow and loosely-connected ethernet (they aim to do
> b> this efficiently over the internet in the future!).
>
> i think HT over network will introduce at least order-of-magnitude overhead
> comparing to direct hash tables.
> but it might still acceptable if it introduces benefits comparing to RDBMS
> performance.
> i didn't evaluate memcached performance, but i saw that RDBMS performance
> was killed with network communications -- sending each packet introduces
> some overhead, and applications was sending megatons of those packets.. so
> time was spent not on web server or database, but on OS system time..
> certainly, network HT might have more optimal protocol than RDBMS over
> network, or you can fine-tune requests better..
DragonFly BSD locks each thread to its own CPU, so data won't be
flying all over the place unneccessarily. Only the data that need to
be shared are passed around (eg. a global hash table frequently
accessed by multiple machines). The main advantage of this over
memcached is that you don't have to serialize objects, and just leave
them as they are. Since you can't serialize things like closures
directly, this can save you some headaches during development.
>
> but you are going to check automated shared memory. i don't see how it can
> work fast :)
> if you'll do testing, please post here some benchmarking results.
>
I don't have spare machines to play around at the moment. But if
anyone has done something similar before, any info would be
appreciated.
> )
> (With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
> "I am everything you want and I am everything you need")
Cheers!
Bob
On Apr 29, 11:09 pm, bob <··········@gmail.com> wrote:
> The idea of memcached is to do make use of existing memory for as
> cheaply as possible, retrieving data from multiple machines through a
> centralized interface, while offering orders of magnitudes of
> performance over disks.
My experience, based on having worked at a significantly large site
was that the best approach by far was to a customised traditional web
cache (a distributed squid in some form or other) to cache everything
that could "easily" be cached. This typically is more than 90% of
your traffic. For the remaining traffic I think that non-stupid
design of the system would mean that a decent database can cope. (in
fact I know this is true).
The front-end cache needs significant thought - we had a very smart
guy who did ours, basically replacing some expensive commercial thing
which killed the site more-or-less every day with a customised squid
which worked very well. Even then there were things it was critical
to get right - for instance we had some imagemap which caused URLs to
come in which had coordinates in which were essentially always a cache
miss and this used to kill the site frequently. Fixing this made a
huge difference.
The non-stupid design thing also matters. At some point I worked out
that we were averaging 1000 requests a second at the front end, well
over 90% of which were satisfied from cache, and the back end database
was sustaining 4000 IOPS. So that's worse than 400 IOPS per uncached
request. I kind of realised we were doomed at that point.
--tim
> My experience, based on having worked at a significantly large site
> was that the best approach by far was to a customised traditional web
> cache (a distributed squid in some form or other) to cache everything
> that could "easily" be cached. This typically is more than 90% of
> your traffic...
Just curious, how does squid refresh its cache? Is it possible for
the web server to tell it to expire an url explicitly? This could be
useful if the site uses predictable url (eg. a REST interface) to
manage user data.
On Apr 30, 3:28 pm, bob <··········@gmail.com> wrote:
> Just curious, how does squid refresh its cache? Is it possible for
> the web server to tell it to expire an url explicitly? This could be
> useful if the site uses predictable url (eg. a REST interface) to
> manage user data.
I don't know the details, but I think that yes, it is, both in the can-
set-ttls sense and in the invalidate-this-stuff-now sense. We
certainly did the latter.
On Apr 30, 10:08 am, Tim Bradshaw <··········@tfeb.org> wrote:
>
> The non-stupid design thing also matters. At some point I worked out
> that we were averaging 1000 requests a second at the front end, well
> over 90% of which were satisfied from cache, and the back end database
> was sustaining 4000 IOPS. So that's worse than 400 IOPS per uncached
> request. I kind of realised we were doomed at that point.
That's not the first time I've made that mistake: 40 IOs per uncached
request not 400, let alone 400 IOPS per request which not only is
wrong but doesn't make sense.
Sorry
On Sun, 29 Apr 2007 15:34:22 +0300, Alex Mizrahi wrote:
> memcached thing is actually PHP style -- PHP is very limited, so it needs an
> external entity for caching.
> with Common Lisp you can just use internal data structures for it, and you
> can make it easily scalable w/o complex 'distibuted hash table' thing (that
> introduces some overhead by the way) if you'll analyse your data processing.
You may want to actually research memcached before spouting rubbish:
http://www.danga.com/memcached/
On May 1, 4:54 am, Brian Adkins <·················@gmail.com> wrote:
> You may want to actually research memcached before spouting rubbish:
>
> http://www.danga.com/memcached/
Right. memcached is doing something really quite non-trivial and
something which is not offered by Lisp out of the box.