Real-time search at Twitter (Keynote)

This is a summary of the keynote talk delivered at ECIR 2014. Twitter’s search engine faces some of the most unique challenges in information retrieval and distributed systems today. On the scaling front, it’s a relatively young system with a massive user base, billions of queries daily, and many billions of indexed documents - with thousands being added every second. On the ranking side, the combination of realtime and social requires new solutions to relevance estimation for newly-created documents, blending different types of live content, evaluation in the absence of direct user feedback, and more. On top of this, the dynamic nature of a nascent company and product leads to a multitude of operational challenges and opportunities. In this talk, I’ll cover some of these challenges and how we approach them at Twitter. © Springer International Publishing Switzerland 2014.