We've come across a challenge using an internal HTTP load balancer (L7) and could use some advice.
We're working on a project in Canada that requires GPUs (which GCP does not offer in Canada). Our Cloud Run project needs to stay in Canada due to speed/latency and data laws, however, this is forcing us to use VMs (GCE) in the US to gain access to GPUs.
We want to internal load balance (HTTP) on our VPC (Cloud Run as the client in the Canada region and GCE+GPUs as the backend in a US region), but this doesn't seem possible. It appears the only way to load balance across regions is to use a TCP load balancer, which doesn't work as well (doesn't allow us to scale on metrics like number of requests or requests/second).
We've considered setting up an Nginx proxy and other types of proxies that would allow us to cross regions, but it would be so much easier to use a GCP-native solution that autoscales.
Best answer by seijimanoanView original
@ljzazmic , really Interesting architecture question, I internally tagged some Google Engineers as long as Google Partners so they can provide a suggestion for you!
Thank you for your help! At this point, we’re open to all suggestions :-)
@ljzazmic. How are you?
I had success with that by using an internal TCP/IP load balancer, so I can request from other VPC and region. Because I can set up the global access. It's still regional, but it has cross regional access.
So you can create a VPC peering between them.
Have you tried that?
Thanks for the response @seijimanoan.
It sounds like you are using an internal TCP/IP LB to bridge the regions.
We specifically need to use an L7 (https) load balancer in order to balance based on metrics like number of requests or requests/second.
Are you suggesting that you are combining an internal TCP/IP LB *and* an L7 internal LB in a way that offers L7 balancing across regions?
It depends on design. We have some ways to think about it.
Does your architecture have Serverless VPC Access?
In docs, I found an interesting example:
So, based on this example, you could have two subnets in different regions in the same VPC network.
I can’t try it right now. But I think you can check it out carefully.
@seijimanoan for the reference to the serverless VPC connector. This may provide a workable solution. I’ve passed this info on to our main tech person to see if this is a viable option!