As our search traffic grew, we were desperately trying to avoid melting every Tuesday around noon (peak traffic for the week), while simultaneously improving the service. Making the index bigger (30M->70M->200M->500M->1B pages and beyond in a short span), …
Scaling Search Index from Millions to Billions of Pages
By
–