How Twitter maximizes performance with BigQuery | C2C Community

How Twitter maximizes performance with BigQuery

  • 10 October 2022
  • 0 replies
  • 3 views

Userlevel 7
Badge +17

 

How does a tweet go from one person to hundreds of millions of people? How does the data process so quickly? In this video of Architecting with Google Cloud, Priyanka chats with Gary and Saurabh from Twitter about how data from over 200 million users goes through the Twitter data center and Google Cloud. Watch along and learn how data stored across tens of thousands of BigQuery tables in Google Cloud runs millions of queries each month.

Click on the video below to watch it in detail:

Chapters:

0:00 - Intro

1:39 - How does Twitter process data?

2:20 - How does Twitter’s data get to Google Cloud?

3:34 - Twitter’s data migration architecture

4:40 - Use cases for BigQuery at Twitter

6:00 - How does Twitter manage storage?

7:49 - How does Twitter organize data?

10:31 - Data storage hierarchy & provisioning

12:06 - How does Twitter secure & monitor data?

13:24 - Twitter’s challenges for data at scale

14:21 - BigQuery at Twitter

15:10 - Wrap up

 

Extra Credit:


0 replies

Be the first to reply!

Reply