Datastream Drove the Conversation at the C2C Connect: France Session on June, the 29th | C2C Community

Datastream Drove the Conversation at the C2C Connect: France Session on June, the 29th


Userlevel 6
Badge +15

The C2C Connect: France session’s momentum, C2C members @antoine.castex and @guillaume.blaquiere hosted a powerful session with Etai Margolin, PM of Datastream, as guest for France (and beyond!) in the cloud space. These sessions intend to bring together a community of cloud experts and customers to connect, learn, and shape the future of cloud. 

 

60 Minutes Summed Up in 60 Seconds

  1. Datastream is a CDC, Change Data Capture, with all the best practices of Google Cloud: serverless, pay-as-you-use, secured and automatically scalable.

  2. Today, only Oracle and MySQL are compliant. But several sources will be added in the future.

  3. Sources can be databases but also SaaS applications, such as SAP or Salesforce (no ETA but planned).

  4. No additional tools or installation is required on the source product. Datastream reuses the native CDC component of the product and plugs them to GCP.

  5. Destination is only Cloud Storage and PubSub is coming soon!

  6. Dataflow templates can be use to sync the data with BigQuery, Spanner and Cloud SQL

  7. A more integrated and transparent integration with the databases is in the roadmap (to avoid the Dataflow configuration step).

  8. A lot of customers asked for this tool. And the IP (Intellectual Property) of Alooma (acquired by Google in 2019) reuse to speed up the development.

  9. Cloud SQL migration tool and Datastream are similar. But Datastream is more designed for long term usage, Cloud SQL migration more for one shot/short term usage.

  10. Datafusion integrates a Datastream component to use both easily. Only BigQuery destinations are possible with this pattern for now.

 

Get In on the Datastream Conversation!

 

Datastream is a hot topic, and it certainly stole the show. The group spent time on CDC (Change Data Capture) flow from the capture to the ingestion, such as: 

  1. Dataflow and streaming capabilities

  2. Cloud Storage and PubSub as Datastream destination

  3. Google Cloud Databases (BigQuery, Spanner, Cloud SQL, BigTable) as changes sink

They also shared what they liked and didn’t. For example, Etai explained that “Kafka source requires high volume and low latency capacity, that is not the perfect match with Datastream.”

 

Preview What's Next

 

Join us for the next session, after the summer break, where we’ll cover the following topics that came up but didn’t make it to the discussion floor: 

  1. Analytics Hub 

  2. Dataplex 

We’ll have the amazing Valliappa Lakshmanan (aka Lak) to tell us the ins and outs.

Interested? Be sure to sign up to get in touch with the group!

 

Extra Credit

 

Looking for more Google Cloud products news and resources? We got you. 

The following links were shared with attendees and are now available to you!


0 replies

Be the first to reply!

Reply