We know how difficult it is to solve problems like fraud detection, ad targeting, and recommendation engines, which require near real-time predictions. The accuracy of these predictions is heavily reliant on access to the most recent data, with delays of just a few seconds making all the difference. However, it is difficult to set up the infrastructure required to support high-throughput updates and low-latency data retrieval.
Erwin Huizenga and Kaz Sato, Developer Advocate, Google Cloud wrote an article based on the latest Vertex AI updates. Vertex AI Matching Engine and Feature Store will support real-time Streaming Ingestion as Preview features. With Streaming Ingestion for Matching Engine, a fully managed vector database for vector similarity search, items in an index are updated continuously and reflected in similarity search results immediately. With Streaming Ingestion for Feature Store, you can retrieve the latest feature values with low latency for highly accurate predictions, and extract real-time datasets for training.
Still there is the issue of the few seconds latency
The latency issue, not only relies on the server but also on the client side at times. In addition to internet speed and bandwidth, CDN, routing, and background running applications, irresponsible browser activity can lead to latency issues. Yes, it is still good, and it will get better in the future.
Thank you for bringing up a good point for discussion.