How does a tweet go from one person to hundreds of millions of people? How does the data process so quickly? In this video of Architecting with Google Cloud, Priyanka chats with Gary and Saurabh from Twitter about how data from over 200 million users goes through the Twitter data center and Google Cloud. Watch along and learn how data stored across tens of thousands of BigQuery tables in Google Cloud runs millions of queries each month.
Click on the video below to watch it in detail:
Chapters:
0:00 - Intro
1:39 - How does Twitter process data?
2:20 - How does Twitter’s data get to Google Cloud?
3:34 - Twitter’s data migration architecture
4:40 - Use cases for BigQuery at Twitter
6:00 - How does Twitter manage storage?
7:49 - How does Twitter organize data?
10:31 - Data storage hierarchy & provisioning
12:06 - How does Twitter secure & monitor data?
13:24 - Twitter’s challenges for data at scale
14:21 - BigQuery at Twitter
15:10 - Wrap up
Extra Credit:
- Twitter Google collab on BigQuery → https://goo.gle/3fJW7Qd
