2019-03-04 02:52:26.610387: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: resnet model_base_path: /models/resnet 2019-03-04 02:52:26.618200: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models. 2019-03-04 02:52:26.618628: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: resnet 2019-03-04 02:52:26.745813: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: resnet version: 1538687457} 2019-03-04 02:52:26.745901: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: resnet version: 1538687457} 2019-03-04 02:52:26.745935: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: resnet version: 1538687457} 2019-03-04 02:52:26.747590: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/resnet/1538687457 2019-03-04 02:52:26.747705: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/resnet/1538687457 2019-03-04 02:52:26.795363: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve } 2019-03-04 02:52:26.828614: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-03-04 02:52:26.923902: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:162] Restoring SavedModel bundle. 2019-03-04 02:52:28.098479: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:138] Running MainOp with key saved_model_main_op on SavedModel bundle. 2019-03-04 02:52:28.144510: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:259] SavedModel load for tags { serve }; Status: success. Took 1396689 microseconds. 2019-03-04 02:52:28.146646: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /models/resnet/1538687457/assets.extra/tf_serving_warmup_requests 2019-03-04 02:52:28.168063: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: resnet version: 1538687457} 2019-03-04 02:52:28.174902: I tensorflow_serving/model_servers/server.cc:286] Running gRPC ModelServer at 0.0.0.0:8500 ... [warn] getaddrinfo: address family for nodename not supported 2019-03-04 02:52:28.186724: I tensorflow_serving/model_servers/server.cc:302] Exporting HTTP/REST API at:localhost:8501 ... [evhttp_server.cc : 237] RAW: Entering the event loop ...
$ saved_model_cli show --dir /tmp/resnet/1538687457/ --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['predict']: The given SavedModel SignatureDef contains the following input(s): inputs['image_bytes'] tensor_info: dtype: DT_STRING shape: (-1) name: input_tensor:0 The given SavedModel SignatureDef contains the following output(s): outputs['classes'] tensor_info: dtype: DT_INT64 shape: (-1) name: ArgMax:0 outputs['probabilities'] tensor_info: dtype: DT_FLOAT shape: (-1, 1001) name: softmax_tensor:0 Method name is: tensorflow/serving/predict
signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['image_bytes'] tensor_info: dtype: DT_STRING shape: (-1) name: input_tensor:0 The given SavedModel SignatureDef contains the following output(s): outputs['classes'] tensor_info: dtype: DT_INT64 shape: (-1) name: ArgMax:0 outputs['probabilities'] tensor_info: dtype: DT_FLOAT shape: (-1, 1001) name: softmax_tensor:0 Method name is: tensorflow/serving/predict
# The server URL specifies the endpoint of your server running the ResNet # model with the name "resnet" and using the predict interface. SERVER_URL = 'http://localhost:8501/v1/models/resnet:predict'
# The image URL is the location of the image we should send to the server IMAGE_URL = 'https://tensorflow.org/images/blogs/serving/cat.jpg'
defmain(): # Download the image dl_request = requests.get(IMAGE_URL, stream=True) dl_request.raise_for_status()
# Send few requests to warm-up the model. for _ in range(3): response = requests.post(SERVER_URL, data=predict_request) response.raise_for_status()
# Send few actual requests and report average latency. total_time = 0 num_requests = 10 for _ in range(num_requests): response = requests.post(SERVER_URL, data=predict_request) response.raise_for_status() total_time += response.elapsed.total_seconds() prediction = response.json()['predictions'][0]
# This is a placeholder for a Google-internal import.
import grpc import requests import tensorflow as tf
from tensorflow_serving.apis import predict_pb2 from tensorflow_serving.apis import prediction_service_pb2_grpc
# The image URL is the location of the image we should send to the server IMAGE_URL = 'https://tensorflow.org/images/blogs/serving/cat.jpg'
tf.app.flags.DEFINE_string('server', 'localhost:8500', 'PredictionService host:port') tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format') FLAGS = tf.app.flags.FLAGS
defmain(_): if FLAGS.image: with open(FLAGS.image, 'rb') as f: data = f.read() else: # Download the image since we weren't given one dl_request = requests.get(IMAGE_URL, stream=True) dl_request.raise_for_status() data = dl_request.content