In this discussion, I will focus on the reference architecture for technical decision makers who want to connect devices and build Cloud IoT apps on Google Cloud using Intelligent Products Essentials.
At this moment, Manufacturers want to continuously improve their products by adding intelligent capabilities that delight customers and monetize new features. In this article, I will describe how google helps architects tasked with designing intelligent product systems on Google Cloud that are scalable, reliable, secure, and cost-effective.
Google Cloud provides capabilities for connecting, ingesting, storing, analyzing, and retrieving data from products to build in artificial intelligence and machine learning capabilities, such as a personalized product ownership app or a digital twin simulation. Here I will present an overview of architecture, its high-level components, integration topics, and general design considerations. The architecture includes the following:
The architecture includes the following:
- Firebase for user app hosting
- BigQuery and Looker for analytics and business intelligence (BI)
- Cloud Run and Firestore for microservices infrastructure
Architecture
The following diagram shows the overall architecture, including the following components:
- Device telemetry streaming
- Microservices
- Analytics and business intelligence
- User app
- Anomaly detection
The diagram uses the following terms:
- Edge: The constrained-device edge or smart device edge as defined in the Linux Foundation continuum.
- Telemetry: Read-only data about the environment, usually collected through sensors. This data is often referred to as eyes-and-ears data (data that comes from facial recognition, image recognition, gesture recognition, speech recognition, natural language processing, and emotion recognition).
- CRM: Customer relationship management.
- ERP: Enterprise resource planning.
The next sections go over in depth the numbered components in this architecture diagram.
The architecture lays the groundwork for analysing device data (including fleets of devices), detecting product anomalies, doing predictive maintenance, and improving the product owner experience.
Although in this dicussions I wil provides some integration recommendations, it doesn't provide details such as device power and hardware, local operating system, compute and storage, and LAN/WAN communications.
Component 2: Microservices
The microservices component contains the device, customer, and user app microservices, as illustrated in the figure below:
Device API microservice
The Device API microservice enables device interactions such as device provisioning by the manufacturer, updating device configuration, and viewing device status. The Device Registry API runs on Cloud Run and stores device data in a Firestore database instance.
Customer API microservice
The Customer API microservice provides an API for customer information relevant to the product ownership experience, such as a unique customer ID for reference and customer name. The Customer API runs on Cloud Run. Data is stored in a Firestore database instance.
User app API microservice
The User app API microservice provides an API for use by user apps. It runs on Cloud Run and combines ownership data which is stored in Firestore with customer and device data from the customer and device API microservices respectively.
Ingestion
The Ingestion component is a function that loads microservice data stored in Firestore into the data warehouse, BigQuery. It runs periodically in Cloud Run triggered by Cloud Scheduler.
Component 3: Analytics and business intelligence
The following diagram shows the analytics and business intelligence (BI) component:
Dataflow is used to extract data, using ELT or ETL, from various sources such as device registry and device telemetry components and optionally other enterprise data sources such as CRM, ERP, and service management. The architecture uses BigQuery as the data warehouse and Looker for visualization.
Component 4: User app
The following diagram shows the subcomponents of the user app component, which is a web and mobile app for the product user:
Firebase provides app development capabilities such as hosting, messaging, authentication, application analytics, and A/B testing. The app communciates with the user app API microservice described above.
Component 5: Anomaly detection
An anomaly detection service runs in Cloud Run and uses Timeseries Insights API or other backends to analyze telemetry data. When an anomaly is detected, the services generates a records in the data warehouse as well as alerts in Pub/Sub that can be used by various subscribers.
Integrating with other systems
This section recommends options from Google Cloud partners and other parties for user edge functions such as IoT clients and OTA solutions.
MLOps with Intelligent products
With device telemetry and other data in BigQuery and Cloud Storage, implement MLOps with intelligent products with Vertex AI for an end-to-end machine-learning lifecycle: prepare, build, validate, deploy, and operate ML models.
Existing and third-party IoT providers
The architecture integrates with existing and third-party IoT deployments. For example, you might already have an IoT implementation with IoT hardware, edge software, and an IoT backend either on-premises or in the cloud.
The following diagram shows how you can integrate existing IoT services with the architecture:
Telemetry data is sent from the service provider edge to Pub/Sub messaging and pre-processed with a service like Dataflow. The device registry information is periodically ingested in batch (ETL/ELT) operations into the data warehouse (BigQuery) for analytics.
Edge IoT client
Cloud IoT supports MQTT and HTTP messages sent from devices to Cloud IoT cloud services. Clients can publish telemetry events with MQTT by using a standard MQTT client.
Extra credit:
- MLOps with Intelligent Products Essentials
- Explore reference architectures, diagrams, tutorials, and best practices about Google Cloud. Take a look at Google Cloud Architectures
