In-memory compression using blosc

17 Mar 2010

Just read:

Why Modern CPUs Are Starving and What Can Be Done About
The summary is that CPUs are way faster than memory, so memory access
is the performance bottleneck. Ok great, now what.

The author proposes using in-memory compression. We ain't talking
gzip either. It's some new fangled algorithm called "blosc". Read
all about it here

I haven't kicked the tires on it yet, but if you are use shared memory
for a ton of data (sound familiar wink wink), this might allow you to
put more into memory and access it faster.