I attended a Silicon Valley Cloud Meetup this week where Siddharth Anand (Sid) gave a great talk on how Netflix has moved to the cloud from applications hosted in their own datacenter. In particular he focused on moving applications from using a traditional RDMS (Oracle in this case) to first SimpleDB (an Amazon offering) and then Cassandra (an open source Key/Value store). SriSatish Ambati (DataStax, @srisatish) hsd given a great overview of Cassandra’s history and capabilities (slides) as a warm up talk.
Sid focused down on the actual issues their engineering team ran into when moving to the could, including:
- Data Model changes required
- Living without SQL support
- No joins
- No transactions
- No triggers
The gist of it amounts to a lot of work in your client code to handle these differences. Netflix accomplishes this by a common layer in their architecture that deals with the NoSQL and hides some of these idiosyncrasies from the application layers above. This is a lot of work (as evidenced by their recent hiring spree – they do these talks to recruit), but is necessary if they are to outsource their resource management and planning to the cloud so that they can scale efficiently.
Sid will give a followup part 2 talk that deals in more depth with the Cassandra issues that they are still working on solving. His slides and a whitepaper are available if you want more details. Updated: There is now a video of the talk.