Configure highly available data and real-time analytics to optimize performance, improve decision-making, and unlock new value.
- 86 Topics
- 72 Replies
Join the Data Engineer Spotlight to connect with your fellow data engineers and Google Cloud experts like Priyanka Vergadia, learn best practices of Google Cloud’s data analytics tools, play a hands-on Cloud Hero game (with prizes at stake!), and more. Want to show how proud you are to be in the data cloud community? Reserve your seat now! Topics include:How Google Cloud is helping the data engineer community solve for increasing data complexity Manage complex SQL workflows in BigQuery using Dataform CLI Build unified batch and streaming pipelines on popular ML frameworks Data Warehouse migrations to BigQuery made easy with BigQuery Migration Service Manage and govern distributed data with Dataplex
Often called the sexiest career of the twenty-first century and the fastest-growing field in tech right now – the effect of Data Science in today's world may seem more mysterious and a difficult field to venture into for those outside the bounds technical field. Let us discover the best sources to learn data science online. To begin let us first explain what data science is and what the job entails.Data Science explainedData science is a subject that entails the gathering and analysis of data – both structured and unstructured – in order to generate insights and information that can be used by businesses to develop effective strategies. Patterns can assist data scientists to detect trends and provide recommendations to stakeholders that will help them locate new market opportunities, improve efficiency, lower costs, and gain a competitive advantage in their sector by collecting and analyzing data over time.Why should you study data science?A large amount of data is generated every day
Hi,We are building an open source Data Observability tool dqo.ai and we wondering what kind of data quality issues we should focus on. So far we are able to monitor timeliness (if the data is fresh), validity (values in columns not meeting requirements), consistency (anomalies how the number of rows is growing over time), uniqueness.I will appreciate the feedback about the typical data quality issues that are worth continuous monitoring.The code is open source on github: https://github.com/dqoai/dqo Best Regards, Piotr
Hi C2C community! I’m Grace, an Associate Product Marketing Manager on the Google Cloud Data Analytics team, flying in with the second Google Cloud Data Heroes Series blog post ☁️In the Google Cloud Data Heroes series, we share stories of the everyday heroes who use our data analytics tools to do incredible things ⭐️ 🤩 In this edition, you'll meet our hero, Antonio, a lead data engineer from Peru 🇵🇪 who built an e-commerce recommendation system on BigQuery, Dataflow, and Cloud Pub/Sub. Outside of work, he is building the next generation of data enthusiasts in Peru through teaching at universities, organizing Data Day conferences, and writing Medium articles.Interested in being the next Google Cloud Data Hero? Fill out this form!
Hello, I am collecting some data for analytics and the intent is to store it in the BQ DB. Need some guidance on structuring the table so that I can utilize BQ ML for analytics.I have reviewed the best practices document of do’s and don’ts. What I am interested is in guidance on table design to optimize some repetitive data and some of the data is in JSON. Please reach out if you are in a position to help!https://www.linkedin.com/in/srinivasan-vanamali-2507004/
💡 Join Google Cloud Industry experts for a half-day dedicated to the possibilities of “Data”. Discover how you and your organization can store, understand, explore & analyze your data to make smarter decisions and solve complex business challenges.Embark on a journey with Google Cloud in building your business with a data-driven culture, and explore key topics like Operating with trust, Democratizing insights, Increasing business agility & Applying intelligence👉 Registration Link : https://inthecloud.withgoogle.com/know-your-data-22/register.htmlThrough a series of demos and hands-on workshops, you’ll explore:💡 Build data applications to deliver great customer experience💡 Understand how to gain deeper insights from your data using BI and embedded analytics💡 Secure, share & govern your datasets with controlled access LinkedIn Event Registration Page
Timed with the Data Cloud Summit, Frederic Lardinois from TechCrunch reports that Google Cloud announced the preview launch of BigLake, a new data lake storage engine that will help enterprises analyze the data in their data warehouses and data lakes.Lardinois notes Google also announced that Spanner will get a new feature called “changed streams,” which will allow users to easily track any changes to a database in real time.Image Credits: GoogleAdditionally, Lardinois notes that Google Cloud announced the general availability of Vertex AI Workbench and launched Connected Sheets for Looker, as well as the ability to access Looker data models in its Data Studio BI tool. Read more details here: https://techcrunch.com/2022/04/05/google-cloud-launches-biglake-a-new-cross-platform-data-storage-engine/?guccounter=1
Data Cloud Summit is around the corner!Have you registered? If not, click on this link.Join the Google Data Cloud Summit to gain insights from technology leaders and data professionals to capitalize on the next wave of data solutions. Discover how you and your organization can make smarter decisions and solve complex challenges with key innovations in AI, machine learning, analytics, and databases.Register today for this digital event.Located in EMEA (Europe, Middle East, Africa)? Visit the EMEA page to view summit events in your time zone and for captioning in your local language. @selavia will be here with us before - during and after the event. She will cycle the technologies that will be announced at the Data Summit.So, stay tuned!
Hi EveryoneWe are organizing a LinkedIn Live session on Big Data Essentials Big Data enables business to identify trends and patterns that may be used in the future. It can assist in identifying which clients are most likely to purchase items or in optimizing marketing campaigns by determining which advertising methods have the best return on investment.In this live session, we will walk you through what is big data, why big data and the data transformation vision for the organization.Event Registration Link: https://www.linkedin.com/video/event/urn:li:ugcPost:6914839113682624512/Event Date: 31March,2022Event Timing: 4:30PM-5:30PM IST
This Tuesday, March 28 we will have a more “Hands-on” approach in our Coffee Chat were we will be walking step by step into a Qwiklab called “Creating a Data Transformation Pipeline with Cloud Dataprep” with the help of one of our community members @MarkusK.Join us to just watch or actively participate in the chat and share!And remember you can join each Tuesday at the same time for an informal discussion about GCP
Hi all,There is an interesting game going on on Google Cloud Skills Boost right now about Bigquery and AI/MLAll you need is an account on Google Cloud Skills Boost to access, you will find more information and the code to access in the following linkhttps://www.cloudskillsboost.google/games/2380Hurry up! the game only last for 2 days!. If you make it to the top post a screenshot here!
Dataplex, is an intelligent data fabric that enables you to centrally manage, monitor, and govern data across data lakes, data warehouses, and data marts, and make this data securely accessible to a variety of analytics and data science tools.And it is Generally Available (GA) now.You can read more here
Google Data Cloud Summit EMEA Join the Google Data Cloud Summit EMEA to gain insights from technology leaders and capitalize on the next wave of data solutions. Discover how you and your organization can make smarter decisions and solve complex challenges with key innovations in AI, machine learning, analytics, and databases.Located in the Americas? Visit the Americas page to view summit events in your time zone.
New Google Expert Opinion Piece: What skills do you think are most important for data practitioners in 2022 and why?
Abdul Razack VP Solutions Enginnering, Technology Solutions and Strategy at Google Cloud and Bruno Aziza, Head of Data and Analytics at Google Cloud weigh in on the top data analytics skills to learn as a data practitioner in 2022. Give it a read and then let us know your thoughts…What skills do you think are most important for data practitioners in 2022 and why? In light of booming unified data clouds should data practitioners generalize their skillsets or specialize their skillsets?[As a bonus, if you’re interested in up-leveling your Google Cloud data analytics skills, be sure to register for the Latest Google Cloud data analytics innovations event happening 3/10. Register here.]
Google Cloud Data Heroes is a series where we share stories of the everyday heroes who use our data analytics tools to do amazing things. Like any good superhero tale, we explore our Google Cloud Data Heroes’ origin stories, how they moved from data chaos to a data-driven environment, what projects and challenges they are overcoming now, and how they give back to the community.Read the full story on Google Cloud Data Heroine Lynn Langit, written by @mesnaider, here! Also, don’t forget to save your seat for the Google Cloud Data Analytics Innovations Event on March 10th! -AJ
This month’s Google Open Source Live will highlight Running Spark Jobs in a Serverless Environment along with multiple sessions presented by Spark team members and the community. Join the Google team to learn How to use Kubernetes, How to Use Notebooks, as well as How to use Iceberg and Delta with Serverless Spark.March 3d - Spark Day on Google Open Source Live Also Google Data Team is hosting another event on March 10th, a live virtual event. “Latest Google Cloud data analytics innovations”
Join us for the Latest Google Cloud data analytics innovations event on March 10th, a live virtual event where Google Cloud’s data leaders and product engineers will focus on getting data professionals up to speed on all the new product updates, features, and solutions coming from Google’s unified data cloud platform in 2022. This event allows attendees to:Get their questions answered LIVE by Google Cloud Data Analytics VPs, leaders, and developer advocates Hear from Google Cloud Data Analysis Innovator Champions Hear about the latest BigQuery, Dataplex, Dataproc, Dataflow, Spark, and Design Patterns product updates Interact with never-before-seen product demos about our latest releases! Connect with the community, and more!Register to save your spot here
In this video and sketchnote I summarize the 6 steps of data science and point you to the tools in Google Cloud that help you achieve the outcome of these steps in data science. This will give you an overview of data science on Google Cloud in 10 minutes. We cover the six steps of data science:1. Data Engineering2. Data Analysis3. Model Development4. ML Engineering5. Insights activation More sketches here
Login to the community
Social LoginLogin With Your C2C Credentials
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.