- Recent
- Popular
- Tags (1)
- Subscribers (21)
- Micron’s 80k Write IOPS PCI Express BoardNovember 28
-
This makes two vendors shipping NAND on PCI-E …
Fusion IO has a competitor. Hopefully this leads to a price war.
Intel’s X-25 card looks pale by comparison.
- Violin’s Flash DeviceNovember 13
-
The flash version has an eight times larger capacity of 4TB, starting at 320GB and consists single level cell (SLC) NAND flash. Its latency is around 23 times slower at 70 microseconds. It supports more than 100,000 sustained random write IOPS and 200,000 read IOPS (4K blocks) and can do so for ten years. According to Violin, users would need 500 15,000 rpm Fibre Channel drives to deliver this level of performance.
The biggest barrier to running MySQL/InnoDB on a device like this is that it’s going to be 100% CPU bound.
My advice to the guys at Fusion IO or Violin is to give/loan one of your lower end machines to Percona, MySQL AB, or even just reach out to one of the alpha geeks.
Of course over time this stuff will just get fixed.
Your devices are going to be CPU bound at first but with motivation they will quickly fix the problem and you’ll sell more customers.
- Google, Bigtable, Compression, Zippy and BMDiffOctober 13
-
A few months ago, when I was heads down finalizing the distributed database in Spinn3r, I was exceedingly curious about what other DBs are using for compression.
GZip seems to be the obvious choice but its compression speed isn’t very good when compared to LZO.
Your disks are almost certainly going to be bottlenecked on IO (if you have a good DB design) so compressing the data means you can trade CPU (which will almost certainly idle)
I remembered some notes about compression in the original Bigtable paper and decided to dig a bit deeper.
Apparently, there isn’t much information about what Google uses for compression in Bigtable, GFS, etc.
These notes were compiled from Jeff Dean’s talk in 2005 but I haven’t seen anything else referencing the subject.
Skip to 46:00 in the Dean talk to see the compression notes.
Andrew Hitchcock also took some notes on the talk:
There is a lot of redundant data in their system (especially through time), so they make heavy use of compression. He went kind of fast and I only followed part of it, so I’m just going to give an overview. Their compression looks for similar values along the rows, columns, and times. They use variations of BMDiff and Zippy. BMDiff giv
- Google Showing Feedburner Redirect URLs in Search ResultsOctober 2
-
Google is showing Feedburner redirect URLs in their search results.
They’re using the link:
http://feeds.latimes.com/~r/topoftheticket/~3/408704356/palin-couric.html
which is a Feedburner redirect URL which they use in RSS feeds to help in tracking.
Google owns Feedburner so it’s a bit embarrassing that they’re making such an obvious mistake.
This might be distorting the stats for the LA Times. Feedburner may in fact be ignoring these when they see that the HTTP referrer is in fact from Google (and not empty or from a web based reader).
The biggest problem with the implementation that Feedburner is using is that it’s impossible to reconstruct the original URL from the redirect URL once it’s in the wild.
A few years ago (when they were just a newly hatched startup and I was working on Rojo) I proposed that they use a URL that encodes the target URL and only adds about 30 additional characters.
The template would be:
http://feeds.latimes.com/r/$nonc
- Pretty InnoDB Buffer Pool StatsSeptember 14
-
The output from SHOW INNODB STATUS isn’t very pretty or easy to use. Seriously, other than Heikki who thinks in 16k block sizes? :-)
A few lines of bash magic fixes this problem.
Now I can quickly see buffer pool stats in the following format:
buffer pool size: 27999076352 bytes (27G) used: 16563568640 bytes (16G) (59.00%) modified db pages: 4747952128 bytes (4G) (16.00%)Here’s what the code looks like:
#!/bin/sh # Pretty print InnoDB buffer stats. # # SHOW INNODB STATUS looks like: # # Buffer pool size 1708928 # Free buffers 1142066 # Database pages 565676 # Modified db pages 123467 pp() { value=$1 if [ $value -gt 1000000000 ]; then value=$(expr $value / 1000000000)G elif [ $value -gt 1000000 ]; then value=$(expr $value / 1000000)M elif [ $value -gt 1000 ]; then value=$(expr $value / 1000)K fi echo $value } perc() { nr=$1 total=$2 echo "scale = 2; ($nr / $total ) * 100" | bc } # don't let bash screw up multiline parsing (I hate this bug) IFS= stats=$(echo "SHOW ENGINE INNODB STATUS\G" | mysql -N) #echo $stats set -o noglob buffer_pool_size=$(echo $stats | grep -E '^Buffer pool size' | grep -Eo '[0-9]+$') buffer_pool_size=$(expr $buffer_pool_size * 16384) free_pool_size=$(echo $stats | grep -E '^Free buffers' | grep -Eo '[0-9]+$') free_pool_size=$(expr $free_pool_size * 16384) modified_db_pages=$(echo $stats | grep -E '^Modified db pages' |
