In Vertex AI Jupyter Notebook while trying to do online prediction on my Pytorch model deployed on Vertex endpoint for a Text classification use case , I am getting below error -
ServiceUnavailable: 503 502:Bad Gateway
I have found there seems to be a limit on the text character and it’s capped to 2500 characters. Please correct me if I am wrong in this. While I am getting output for Input length less than 2500 characters, but for any input more than that I am getting the above error. Please help.
Thank you for the questions. There are many reasons behind the error code. Please see below for details. This may assist you in overcoming this problem.
Can't access the terminal in a managed notebooks instance
Terminal access must be enabled when you create a managed notebooks instance.
If you're unable to access the terminal or can't find the terminal window in the launcher, it could be because your managed notebooks instance does not have terminal access enabled.
502 error when opening JupyterLab
A 502 error might mean that your managed notebooks instance is not ready yet. Wait a few minutes, refresh the Google Cloud console browser tab, and try again.
Opening a notebook results in a 524 (A Timeout Occurred) error
A 524 error is usually an indication that the Inverting Proxy agent isn't connecting to the Inverting Proxy server or the requests are taking too long on the backend server side (Jupyter). Common causes of this error include networking issues, the Inverting Proxy agent is not running, or the Jupyter service is not running.
If you can't access a notebook, verify that your managed notebooks instance is started.
Notebook is unresponsive
If your managed notebooks instance isn't executing cells or appears to be frozen, first try restarting the kernel by clicking Kernel from the top menu and then Restart Kernel. If that doesn't work, you can try the following:
GPU quota has been exceeded
Determine the number of GPUs available in your project by checking the quotas page. If GPUs are not listed on the quotas page, or you require additional GPU quota, you can request a quota increase. See Requesting additional quota on the Compute Engine Resource Quotas page.
Using container images
Container image doesn't appear as a kernel in JupyterLab
Container images that do not have a valid kernelspec don't successfully load as kernels in JupyterLab. For more information, see the custom container requirements.
Using the executor
Package installations not available to the executor
The executor runs your notebook code in a separate environment from the kernel where you run your notebook file's code. Because of this, some of the packages you installed might not be available in the executor's environment. To resolve this issue, see Ensure package installations are available to the executor.
401 or 403 errors when running the notebook code using the executor
A 401 or 403 error when you run the executor can mean that the executor is not able to access resources. See the following for possible causes:
The executor runs your notebook code in a tenant project separate from your managed notebooks instance's project. Therefore, when you access resources through code run by the executor, the executor might not connect to the correct Google Cloud project by default. To resolve this issue, use explicit project selection.
By default, your managed notebooks instance can have access to resources that exist in the same project, and therefore, when you run your notebook file's code manually, these resources do not need additional authentication. However, because the executor runs in a separate tenant project, it does not have the same default access. To resolve this issue, authenticate access using service accounts.
The executor cannot use end-user credentials to authenticate access to resources, for example, the
gcloud auth logincommand. To resolve this issue, authenticate access using service accounts.
exited with a non-zero status of 127error when using the executor
exited with a non-zero status of 127error, or "command not found" error, can happen when you use the executor to run code on a custom container that does not have the
To ensure that your custom container has the
nbexecutorextension, you can create a derivative container image from a Deep Learning Containers image. Deep Learning Containers images include the
Invalid service networking configuration error message
No free blocks were found in the allocated IP ranges of your network. Use a subnet mask of
/24or lower. For more information, see Set up a network.
Unable to install third party JupyterLab extension
Attempting to install a third party JupyterLab extension results in an
Third party JupyterLab extensions are not supported in managed notebooks instances.
Also, you can check the following links .
Thanks for another great answer,
@Dimitris Petrakis .
Thanks for another great answer,
Thanks for your answer. I have tried, but it didn’t help solving the issue. The same error is popping again and again. It’s working if the input text is of 2550 characters, once it exceeds it’s throwing the error. For reference notebook, we are following this - vertex-ai-samples/pytorch-text-classification-vertex-ai-train-tune-deploy.ipynb at main · GoogleCloudPlatform/vertex-ai-samples · GitHub
In this notebook at the bottom in this part ‘'Sending an online prediction request’ when I am trying to give a very long input , it is throwing the error as I mentioned.
Thank you so much for reply. Please allow some time I will check it for you and come back with an update.
Did you get a chance to look at the repo?
I apologised profusely for the delay. Actually, I was preoccupied with another important task, so I was unable to focus on this topic.
But I tried putting the code into action to see what happened, but I couldn't find the time for another experiment.
However, did you solve your problem or are you still having trouble?
If you have already solved the problem, please share your solution here so that others can learn from your experience.
Otherwise, I'll try again.
@Abhishek111 , did you see malamin’s reply? Perhaps you managed to find a solution in the meantime? Or are you perhaps still facing trouble?
Please kindly let us know.
A kind reminder,
you could ask i think google directly to be capp at 5k they could tell you how but im not so sure if its the same