Use a Python client

Brief introduction

If we want to access the TensorFlow Serving service, the Python applications also need to implement the corresponding gRPC client.

TensorFlow Serving official documentation provides examples of generating an MNIST Python client, but because of reliance on Bazel compilation, the compiled Python script cannot execute directly.

Complete example

Here is an example of a Python gRPC client. The Proto code is generated manually. With no dependencies, users can directly execute https://github.com/tobegit3hub/deep_recommend_system/tree/master/python_predict_client .

./predict_client.py --host 127.0.0.1 --port 9000 --model_name linear --model_version 1

The input formats of different models are different. We only need to directly modify the Python script to build TensorProto code, and It is recommended to use the project to compile generated files and code examples.

Implementation principles

To implement the Python gRPC client, we need to compile TensorFlow Serving and TensorFlow Proto files. Since the official project relies on Bazel compilation, we need to modify the dependency path in the Proto file.

Compiled Python files can be imported directly. Then refer to the TensorFlow document to generate the TensorProto object. Refer to gRPC documentation to invoke the gRPC server.

Read an image file to generate a TensorProto

In such scenarios as image classification, we need to convert the image to a Numpy or TensorProto object in order to request TensorFlow Serving service through gRPC. Here is an example of using Scipy Ndimage. The test already supports JPG and PNG files.

Here is a complete example of using CNN to train image classification models and then using a Python client to load JPG or PNG files for predictions.https://github.com/tobegit3hub/deep_cnn/blob/master/python_predict_client/predict_client.py .

from scipy import ndimage

# Create numpy ndarray for batch images
features = np.ndarray(shape=(5, 32, 32, 3), dtype=np.float32)

image_filepaths = ["../data/inference/Blastoise.png",
                   "../data/inference/Charizard.png",
                   "../data/inference/Mew.png",
                   "../data/inference/Pikachu.png",
                   "../data/inference/Venusaur.png"]

for index, image_filepath in enumerate(image_filepaths):
  image_ndarray = ndimage.imread(image_filepaths[0], mode="RGB")
  features[index] = image_ndarray

# Create TensorProto
features_tensor_proto = tf.contrib.util.make_tensor_proto(features,
                                                          dtype=tf.float32)