[Ray[serve] gRPC] Unable to pickle protobuf objects #45351
Assignees
Comments
mgierada
added
bug
shrekris-anyscale
added
P2
|
@mgierada I just tried it again on Ray master and it has no issue. The only difference in the dependencies is I'm on Otherwise you can probably try those files generated from my version to see if it solves the issue |
|
Hey @GeneDer Thanks for taking a look. This is still not working for me. I made sure I am running the same version of |
|
oh actually another difference, I'm running on Python 3.10.12. Maybe try it out. Also my pickle version is 4.0 not sure if that makes a difference. |
|
I tired with python 3.10.12 and with pickle 4.0 @GeneDer but I still got the same error. Here's what I am running, maybe I missed something obvious.
python -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. ./src/protos/user_defined_protos.proto
ray start --head
serve start \
--grpc-port 9000 \
--grpc-servicer-functions protos.user_defined_protos_pb2_grpc.add_UserDefinedServiceServicer_to_serverThat correctly starts the ray cluster.
import time
from typing import Generator
from protos.user_defined_protos_pb2 import (
UserDefinedMessage,
UserDefinedMessage2,
UserDefinedResponse,
UserDefinedResponse2,
)
import ray
from ray import serve
@serve.deployment
class GrpcDeployment:
def __call__(self, user_message: UserDefinedMessage) -> UserDefinedResponse:
greeting = f"Hello {user_message.name} from {user_message.origin}"
num = user_message.num * 2
user_response = UserDefinedResponse(
greeting=greeting,
num=num,
)
return user_response
@serve.multiplexed(max_num_models_per_replica=1)
async def get_model(self, model_id: str) -> str:
return f"loading model: {model_id}"
async def Multiplexing(self, user_message: UserDefinedMessage2) -> UserDefinedResponse2:
model_id = serve.get_multiplexed_model_id()
model = await self.get_model(model_id)
user_response = UserDefinedResponse2(
greeting=f"Method2 called model, {model}",
)
return user_response
def Streaming(
self, user_message: UserDefinedMessage
) -> Generator[UserDefinedResponse, None, None]:
for i in range(10):
greeting = f"{i}: Hello {user_message.name} from {user_message.origin}"
num = user_message.num * 2 + i
user_response = UserDefinedResponse(
greeting=greeting,
num=num,
)
yield user_response
time.sleep(0.1)
g = GrpcDeployment.bind()
app1 = "app1"
serve.run(target=g, name=app1, route_prefix=f"/{app1}")
|
|
What if you don't put those in a sub-directory and instead just generate in the same root? |
|
Nah, unfortunately that is not helping much, |
|
Oh I finally solved that issue. Docs are very misleading on this. The cause was that my proto imports were out of scope. Moving them close to the place they are used made a difference. So instead of importing on top of the file like this import time
from typing import Generator
from protos.user_defined_protos_pb2 import (
UserDefinedMessage,
UserDefinedMessage2,
UserDefinedResponse,
UserDefinedResponse2,
)
import ray
from ray import serve
# ... ray server logicdo this # file name ./src/deploy.py
import time
from typing import Generator
from ray import serve
@serve.deployment
class GrpcDeployment:
def __call__(self, user_message):
from user_defined_protos_pb2 import UserDefinedMessage, UserDefinedResponse
greeting = f"Hello {user_message.name} from {user_message.origin}"
num = user_message.num * 2
user_response = UserDefinedResponse(
greeting=greeting,
num=num,
)
return user_response
@serve.multiplexed(max_num_models_per_replica=1)
async def get_model(self, model_id: str) -> str:
return f"loading model: {model_id}"
async def Multiplexing(self, user_message):
from user_defined_protos_pb2 import UserDefinedMessage2, UserDefinedResponse2
model_id = serve.get_multiplexed_model_id()
model = await self.get_model(model_id)
user_response = UserDefinedResponse2(
greeting=f"Method2 called model, {model}",
)
return user_response
def Streaming(self, user_message) -> Generator:
from user_defined_protos_pb2 import UserDefinedMessage, UserDefinedResponse
for i in range(10):
greeting = f"{i}: Hello {user_message.name} from {user_message.origin}"
num = user_message.num * 2 + i
user_response = UserDefinedResponse(
greeting=greeting,
num=num,
)
yield user_response
time.sleep(0.1)
g = GrpcDeployment.bind()
app1 = "app1"
serve.run(target=g, name=app1, route_prefix=f"/{app1}")Then, assuming your ray is available on the localhost or whatever, deploy by running the file python src/deploy.py |
8 tasks
|
The following PR fixes the docs: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What happened + What you expected to happen
I went through the documentation with 1 to 1 proto file but I keep getting the following errors when running the app.
Similar issue was posted here by someone else https://discuss.ray.io/t/keep-getting-error-typeerror-cannot-pickle-classmethod-descriptor-object/10153
Versions / Dependencies
but I tired so many version and combination including the latest grpcio-tools, grpcio and protobuf as well as the very old one. The issue is still the same.
I am running python 3.11.9 and apple silicon.
Reproduction script
Please follow the official documentation https://docs.ray.io/en/latest/serve/advanced-guides/grpc-guide.html
Issue Severity
High: It blocks me from completing my task.
The text was updated successfully, but these errors were encountered: