Jeff Dean

Increasing Index Size and Query Capacity

• Huge increases in index size in ’99, ’00, ’01, ...
  – From ~50M pages to more than 1000M pages
• At same time as huge traffic increases
  – ~20% growth per month in 1999, 2000, ...
  – ... plus major new partners (e.g. Yahoo in July 2000 doubled traffic overnight)
• Performance of index servers was paramount
  – Deploying more machines continuously, but...
  – Needed ~10-30% software-based improvement every month