Google Cloud Pub/Sub Strategies For Beginners | C2C Community

Google Cloud Pub/Sub Strategies For Beginners

Categories: Applications and DevOps
Google Cloud Pub/Sub Strategies For Beginners

In essence, Google Cloud Pub/Sub is a real-time messaging system, perfect for those large companies with modern microservices that want to integrate their services. GC Pub/Sub scales with platform growth and helps you distribute analytics and data to or from some or more stakeholders, any time, anywhere. Publishers post messages to stakeholders “fork-down,” meaning from one to many. Subscribers, namely the stakeholders or those who subscribe to the platform, post “many to one”––namely to the Publisher. 

With GC Pub/Sub, you can reach huge amounts of people synchronously, or decouple your posts for targeted recipients. Projects get done faster, more reliably, and more effectively, with no loss of downtime or jitter to the company. Today, more than 173 companies use Google Cloud Pub/Sub, including HENNGE K.K., Client Platform, and PLAID.

 

Why Do We Need GC Pub/Sub?

GC Pub/Sub is the only model that accommodates both push- and pull-based supply chain models.

Push-based models are those industries that use analytics to forecast customer demand and supply accordingly. Pull-based models, in contrast, wait for customer demand to flow in before supplying for those needs.

 

The two most notable push or pull-based models on the market are:

GC Pub/Sub services both niches.

Alternately, vendors/ operators can use synchronous (or asynchronous) remote procedure calls (RPCs) for communicating between their different microservices. But while GC sends events to all that need to receive them without regard to how or when these events will be processed, RPCs tend to be time-consuming, since publishers must wait for subscribers to receive the data. 

 

Vocabulary

  • Topic – The shared string where members discourse through a common thread.

  • Publish – How publishers “push” (or “publish”) a message to a Cloud Pub/Sub topic.

  • Subscription – How subscribers “subscribe” to a topic, either by pulling messages from the “subscription” or setting up automated “webhooks” (i.e. notifications) to receive these subscriptions.

 

The GC Pub/Sub Process

The Publisher drafts a “message” that falls under a specific Topic. This “message” is either stored in Message Storage until it's picked up by one of the subscribers who subscribe to that topic, or gets pushed to the entire pool, or to one or more pools of appropriate subscribers. The recipients acknowledge the receipt of this message. A non-success response means GC Pub/Sub redelivers the message until receipt is confirmed. 

 

Example

Nike Inc. sells its products to more than 170 countries. Its microservices include checkout, track activity, and read data. To make sure the apps deliver a good experience and to build Nike's brand, Nike's Albert Perez, director of database and data services, needs a scalable Pub/Sub system where he sends messages to subscribers (namely to his more than 30,000 employees across microservices and to Nike’s stakeholders). The ideal Pub/Sub system would help Nike’s microservices inter-and intra-communicate in a timely manner. The best type of messaging service should also be impervious to hacking, resilient to downtime, and fully automated and smooth.

That’s what the Google Cloud Pub/Sub system provides!

 

The Pub-Sub Distribution System

Publishers have the following three options. 

  1. One-to-many (fan-out) – Publishers can mail their posts in different subscription boxes that will then reach individual subscribers.

  2. Many-to-one (fan-in) – Various publishers can post on the same topic that will then reach one collective pool of subscribers.

  3. Many-to-many - Various publishers can post on the same topic to diverse subscribers.

Example

Returning to Nike Inc., Albert Perez could use these different Pub-Sub options to send messages to one or more of Nike’s microservices at the same time. By the same token, one of its microservices––such as the data analytics service––could publish its own update or event ticket that would then get distributed to one or more targeted subscribers who would acknowledge receipt of the data.

Subscribers could also access these messages through the Pull process, where they select that particular topic from the Message Storage folder. Once retrieved, the post would be eliminated from Storage.

 

Google Cloud Pub/Sub Pros

  • Fully-managed and highly scalable messaging middleware service – The system scales to meet demand. You can send millions of messages at one time with latencies less than 100 milliseconds, which is considered decent.

  • Provides good support for asynchronous processes – GC Pub/Sub provides an ideal decoupling service, where microservices function separately to the rest of the company enabling for internal control, testing, or management of a single point of failure (for example), while the company continues to function. 

  • A dedicated topic for unprocessed messages (dead-letter topic) – The delivery keeps resending until the recipient acknowledges receipt of the message.

  • HIPAA-compliant - GC Pub/Sub’s platform security ensures the privacy, security, and integrity of protected health information.

Google Cloud Pub/Sub Cons

  • Message ordering is not guaranteed. GC Pub/Sub may stall on collecting (or sending) extra-bulky messages that could stall the system.

  • Multiple transfer services can be complicated, with some messages becoming orphaned or mistakenly unacknowledged.

GC Pub/Sub User-Cases

Companies typically use GC Pub/Sub for the following case-scenarios:

  • Refreshing distributed caches – For example, an application can publish invalidation events to update the object identifications of those objects that have changed.

  • Distributing event notifications – GC Pub/Sub allows you to gather events from many clients simultaneously.

  • Parallel processing and workflows – You can efficiently distribute a large batch of tasks among multiple workers––such as compressing text files, sending email notifications, evaluating AI models, or reformatting images.

  • Real-time event distribution – Events can be made available to multiple applications across your team and organization for real-time processing.

  • Replicating data among databases – n order, for instance, to inform all divisions of updates in real-time or to distribute changed events from databases.

  • Load balancing for reliability – If the service fails in any one area, (for example, if one of your critical employees is missing), the others that have subscribed to this common topic can pick up the load automatically.

  • Data streaming from the Internet of Things (IoT) devices – For example, a residential sensor can stream data to backend servers hosted in the cloud.
     

Bottom Line

For large companies like Nike Inc. that use microservices for agile, fast-moving, and lean communication, Google Cloud Pub/Sub helps them send millions of messages in real-time to various or more of their microservices with the acknowledgment that these messages have been received. GC Pub/Sub is the only Push/Pull option on the market that’s also flexible and smooth. It’s scalable, secure, and reliable, and provides managers with a great option for effective storage, data analytics, and data streaming. 

Want to learn the basics of GC Pub/Sub and perhaps distribute your own messages? This free Google Cloud Pub/Sub Qwik Start Lab lets you do all this and more…

 

Let’s Connect!

Leah Zitter, PhD, has a Masters in Philosophy, Epistemology and Logic and a PhD in Research Psychology.

 

@Leah Zitter  Awesome write-up on GCP Pub/Sub..!!