如何在Tensorflow服务中进行批处理？

Question

部署的Tensorflow为Inception-V3服务并运行测试。工作正常。

现在，想为Inception-V3服务的批处理。 例如想要发送10个图像进行预测而不是一个。

如何做到这一点？要更新哪些文件（inception_saved_model.py或inception_client.py）？这些更新是什么样子的？以及图像如何传递给服务 - 它是以包含图像的文件夹的形式传递的？

感谢您对此问题的深入了解。任何与此相关的代码片段都会非常有用。

=================================

更新inception_client.py

# Copyright 2016 Google Inc. All Rights Reserved. 
# 
# Licensed under the Apache License, Version 2.0 (the "License"); 
# you may not use this file except in compliance with the License. 
# You may obtain a copy of the License at 
# 
#  http://www.apache.org/licenses/LICENSE-2.0 
# 
# Unless required by applicable law or agreed to in writing, software 
# distributed under the License is distributed on an "AS IS" BASIS, 
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
# See the License for the specific language governing permissions and 
# limitations under the License. 
# ============================================================================== 

#!/usr/bin/env python2.7 

"""Send JPEG image to tensorflow_model_server loaded with inception model. 
""" 

from __future__ import print_function 

"""Send JPEG image to tensorflow_model_server loaded with inception model. 
""" 

from __future__ import print_function 

# This is a placeholder for a Google-internal import. 

from grpc.beta import implementations 
import tensorflow as tf 
from tensorflow.python.platform import flags 
from tensorflow_serving.apis import predict_pb2 
from tensorflow_serving.apis import prediction_service_pb2 


tf.app.flags.DEFINE_string('server', 'localhost:9000', 
          'PredictionService host:port') 
tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format') 
FLAGS = tf.app.flags.FLAGS 


def main(_): 
    host, port = FLAGS.server.split(':') 
    channel = implementations.insecure_channel(host, int(port)) 
    stub = prediction_service_pb2.beta_create_PredictionService_stub(channel) 
    # Send request 
    #with open(FLAGS.image, 'rb') as f: 
    # See prediction_service.proto for gRPC request/response details. 
    #data = f.read() 
    #request = predict_pb2.PredictRequest() 
    #request.model_spec.name = 'inception' 
    #request.model_spec.signature_name = 'predict_images' 


# request.inputs['images'].CopyFrom(
#  tf.contrib.util.make_tensor_proto(data, shape=[1])) 
# result = stub.Predict(request, 10.0) # 10 secs timeout 
# print(result) 


# Build a batch of images 

    request = predict_pb2.PredictRequest() 
 request.model_spec.name = 'inception' 
request.model_spec.signature_name = 'predict_images' 
   
  image_data = [] 
  for image in FLAGS.image.split(','): 
   with open(image, 'rb') as f: 
     image_data.append(f.read()) 
   
  request.inputs['images'].CopyFrom(
      tf.contrib.util.make_tensor_proto(image_data, shape=[len(image_data)])) 
   
  result = stub.Predict(request, 10.0)  # 10 secs timeout 
  print(result) 
if __name__ == '__main__': 
    tf.app.run() 

Answer 1

您应该能够计算对inception_client.py中的请求构造代码进行较小更改的一批图像的预测。在该文件中以下行创建具有“间歇”含有单一图像的请求（注意shape=[1]，这意味着“长度为1的矢量”）：

with open(FLAGS.image, 'rb') as f: 
    # See prediction_service.proto for gRPC request/response details. 
    data = f.read() 
    request = predict_pb2.PredictRequest() 
    request.model_spec.name = 'inception' 
    request.model_spec.signature_name = 'predict_images' 
    request.inputs['images'].CopyFrom(
     tf.contrib.util.make_tensor_proto(data, shape=[1])) 
    result = stub.Predict(request, 10.0) # 10 secs timeout 
    print(result) 

可以在相同载体中传递多个图像以对一批数据运行预测。例如，如果FLAGS.image是逗号分隔的文件名列表：

request = predict_pb2.PredictRequest() 
request.model_spec.name = 'inception' 
request.model_spec.signature_name = 'predict_images' 

# Build a batch of images. 
image_data = [] 
for image in FLAGS.image.split(','): 
    with open(image, 'rb') as f: 
    image_data.append(f.read()) 

request.inputs['images'].CopyFrom(
    tf.contrib.util.make_tensor_proto(image_data, shape=[len(image_data)])) 

result = stub.Predict(request, 10.0) # 10 secs timeout 
print(result) 

if __name__ == '__main__': 
    tf.app.run()