Skip to content

Deploy Paddle model with InferenceService

In this example, we use a trained paddle resnet50 model to classify images by running an inference service with Paddle predictor.

Deploy Paddle model with V1 protocol

Create the InferenceService

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "paddle-resnet50"
spec:
  predictor:
    model:
      modelFormat:
        name: paddle
      storageUri: "gs://kfserving-examples/models/paddle/resnet"

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "paddle-resnet50"
spec:
  predictor:
    paddle:
      storageUri: "gs://kfserving-examples/models/paddle/resnet"

Apply the above yaml to create the InferenceService 

kubectl apply -f paddle.yaml

Expected Output

$ inferenceservice.serving.kserve.io/paddle-resnet50 created

Run a Prediction

The first step is to determine the ingress IP and ports and set INGRESS_HOST and INGRESS_PORT.

You can see an example payload jay.json as the sample input to test the model. 

MODEL_NAME=paddle-resnet50
SERVICE_HOSTNAME=$(kubectl get inferenceservice ${MODEL_NAME} -o jsonpath='{.status.url}' | cut -d "/" -f 3)
curl -v -H "Host: ${SERVICE_HOSTNAME}" \
 -H "Content-Type: application/json" \
 -d @./jay.json \
 http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/${MODEL_NAME}:predict

Expected Output

*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /v1/models/paddle-resnet50:predict HTTP/1.1
> Host: paddle-resnet50.default.example.com
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 3010209
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 23399
< content-type: application/json; charset=UTF-8
< date: Mon, 17 May 2021 03:34:58 GMT
< server: istio-envoy
< x-envoy-upstream-service-time: 511
<
{"predictions": [[6.736678770380422e-09, 1.1535990829258935e-08, 5.142250714129659e-08, 6.647170636142619e-08, 4.094492567219277e-08, 1.3402451770616608e-07, 9.355561303436843e-08, 2.8935891904779965e-08, 6.845367295227334e-08, 7.680615965455218e-08, 2.0334689452283783e-06, 1.1085678579547675e-06, 2.3477592492326949e-07, 6.582037030966603e-07, 0.00012373103527352214, 
4.2878804151769145e-07, 6.419959845516132e-06, 0.9993496537208557, 7.372002437477931e-05, 3.101135735050775e-05, 5.6028093240456656e-06, 2.1862508674530545e-06, 1.9544044604913324e-08, 3.728893887000595e-07, 4.2903633357127546e-07, 1.8251179767503345e-07, 7.159925985433802e-08, 9.231618136595898e-09, 6.469241498052725e-07, 7.031690341108288e-09, 4.451231561120039e-08, 1.2455971898361895e-07, 9.44632745358831e-08, 4.347704418705689e-08, 4.658220120745682e-07, 6.797721141538204e-08, 2.1060276367279585e-07, 2.2605123106700376e-08, 1.4311490303953178e-07, 7.951298641728499e-08, 1.2341783417468832e-07, 1.0921713737843675e-06, 1.5243892448779661e-05, 3.1173343018053856e-07, 2.4152058131221565e-07, 6.863762536113427e-08, 8.467682022228473e-08, 9.4246772164297e-08, 1.0219210366813058e-08, 3.3770753304906975e-08, 3.6928835100979995e-08, 1.3694031508748594e-07, 1.0674284567357972e-07, 2.599483650556067e-07, 3.4866405940192635e-07, 3.132053549848024e-08, 3.574873232992104e-07, 6.64843895492595e-08, 3.1638955988455564e-07, 1.2095878219042788e-06, 8.66409024524728e-08, 4.0144172430700564e-08, 1.2544761318622477e-07, 3.3201178695208e-08, 1.9731444922399533e-07, 3.806405572959193e-07, 1.3827865075199952e-07, 2.300225965257141e-08, 7.14422512260171e-08, 2.851114544455413e-08, 2.982567437470607e-08, 8.936032713791064e-08, 6.22388370175031e-07, 6.478838798784636e-08, 1.3663023423760023e-07, 9.973181391842445e-08, 2.5761554667269593e-08, 4.130220077058766e-08, 3.9384463690339544e-08, 1.2158079698565416e-07, 4.302821707824478e-06, 1.8179063090428826e-06, 1.8520155435908237e-06, 1.6246107179540559e-06, 1.6448313544970006e-05, 1.0544916221988387e-05, 3.993061909568496e-06, 2.646479799750523e-07, 1.9193475964129902e-05, 4.803242745765601e-07, 1.696285067964709e-07, 4.550505764200352e-06, 4.235929372953251e-05, 4.443338639248395e-06, 5.104009687784128e-06, 1.3506396498996764e-05, 4.1758724478313525e-07, 4.494491463447048e-07, 3.156698369366495e-07, 1.0557599807725637e-06, 1.336463917311903e-08, 1.3893659556174498e-08, 6.770379457066156e-08, 1.4129696523923485e-07, 7.170518756538513e-08, 7.934466594861078e-08, 2.639154317307657e-08, 2.6134321373660896e-08, 7.196725881897237e-09, 2.1752363466021052e-08, 6.684639686227456e-08, 3.417795824134373e-08, 1.6228275967478112e-07, 4.107114648377319e-07, 6.472135396506928e-07, 2.951379372007068e-07, 5.653474133282543e-09, 4.830144462175667e-08, 8.887481861563629e-09, 3.7306168820805397e-08, 1.7784264727538357e-08, 4.641905082536368e-09, 3.413118676576232e-08, 1.937393818707278e-07, 1.2980176506971475e-06, 3.5641004814124244e-08, 2.149332445355867e-08, 3.055293689158134e-07, 1.5532516783878236e-07, 1.4520978766086046e-06, 3.488464628276233e-08, 3.825438398052938e-05, 4.5088432898410247e-07, 4.1766969616219285e-07, 6.770622462681786e-07, 1.4142248971893423e-07, 1.4235997696232516e-05, 6.293820433711517e-07, 4.762866865348769e-06, 9.024900577969674e-07, 9.058987870957935e-07, 1.5713684433649178e-06, 1.5720647184025438e-07, 1.818536503606083e-07, 7.193188622522939e-08, 1.1952824934269302e-06, 8.874837362782273e-07, 2.0870831463071227e-07, 9.906239029078279e-08, 7.793621747964607e-09, 1.0058498389753368e-07, 4.2059440374941914e-07, 1.843624630737395e-07, 1.6437947181202617e-07, 7.025352743994517e-08, 2.570448600636155e-07, 7.586877615040066e-08, 7.841313731660193e-07, 2.495309274763713e-07, 5.157681925993529e-08, 4.0674127177453556e-08, 7.531796519799627e-09, 4.797485431140558e-08, 1.7419973019627832e-08, 1.7958679165985814e-07, 1.2566392371127222e-08, 8.975440124459055e-08, 3.26965476915575e-08, 1.1208359751435637e-07, 3.906746215420753e-08, 4.6769045525252295e-08, 1.8523553535487736e-07, 1.4833052830454108e-07, 1.2279349448363064e-07, 1.0729105497375713e-06, 3.6538490011395197e-09, 1.6198403329781286e-07, 1.6190719875908144e-08, 1.2004933580556099e-07, 1.4800277448046018e-08, 4.02294837442696e-08, 2.15060893538066e-07, 1.1925696696835075e-07, 4.8982514044837444e-08, 7.608920071788816e-08, 2.3137479487900237e-08, 8.521050176568679e-08, 9.586213423062873e-08, 1.3351650807180704e-07, 3.021699157557123e-08, 4.423876376336011e-08, 2.610667060309879e-08, 2.3977091245797055e-07, 1.3192564551900432e-07, 1.6734931662654162e-08, 1.588336999702733e-07, 4.0643516285854275e-07, 8.753454494581092e-08, 8.366999395548191e-07, 3.437598650180007e-08, 7.847892646850596e-08, 8.526394701391382e-09, 9.601382799928615e-08, 5.258924034023948e-07, 1.3557448141909845e-07, 1.0307226716577134e-07, 1.0429813457335513e-08, 5.187714435805901e-08, 2.187001335585137e-08, 1.1791439824548888e-08, 2.98065643278278e-08, 4.338393466696289e-08, 2.9991046091026874e-08, 2.8507610494443725e-08, 3.058665143385042e-08, 6.441099031917474e-08, 1.5364101102477434e-08, 1.5973883549236234e-08, 2.5736850872704053e-08, 1.0903765712555469e-07, 3.2118737891551064e-08, 6.819742992547617e-09, 1.9251311300649832e-07, 5.8258109447706374e-08, 1.8765761922168167e-07, 4.0070790419122204e-07, 1.5791577823165426e-08, 1.950158434738114e-07, 1.0142063189277906e-08, 2.744815041921811e-08, 1.2843531571604672e-08, 3.7297493094001766e-08, 7.407496838141014e-08, 4.20607833007125e-08, 1.6924804668860816e-08, 1.459203531339881e-07, 4.344977000414474e-08, 1.7191403856031684e-07, 3.5817443233554513e-08, 8.440249388286247e-09, 4.194829728021432e-08, 2.514032360068086e-08, 2.8340199520471288e-08, 8.747196034164517e-08, 8.277125651545703e-09, 1.1676293709683705e-08, 1.4548514570833504e-07, 7.200282148289716e-09, 2.623600948936655e-06, 5.675736929333652e-07, 1.9483527466945816e-06, 6.752595282932816e-08, 8.168475318370838e-08, 1.0933046468153407e-07, 1.670913718498923e-07, 3.1387276777650186e-08, 2.973524537708272e-08, 5.752163900751839e-08, 5.850877471402782e-08, 3.2544622285968217e-07, 3.330221431951941e-08, 4.186786668469722e-07, 1.5085906568401697e-07, 2.3346819943981245e-07, 2.86402780602657e-07, 2.2940319865938363e-07, 1.8537603807544656e-07, 3.151798182443599e-07, 1.1075967449869495e-06, 1.5369782602192572e-07, 1.9237509718550427e-07, 1.64044664074936e-07, 2.900835340824415e-07, 1.246654903752642e-07, 5.802622027317739e-08, 5.186220519703966e-08, 6.0094205167615655e-09, 1.2333241272699524e-07, 1.3798474185477971e-07, 1.7370231830682314e-07, 5.617761189569137e-07, 5.1604470030497396e-08, 4.813277598714194e-08, 8.032698417537176e-08, 2.0645263703045202e-06, 5.638597713186755e-07, 8.794199857220519e-07, 3.4785980460583232e-06, 2.972389268052211e-07, 3.3904532870110415e-07, 9.469074058188198e-08, 3.754845678827223e-08, 1.5679037801419327e-07, 8.203105039683578e-08, 6.847962641387539e-09, 1.8251624211984563e-08, 6.050240841659615e-08, 3.956342808919544e-08, 1.0699947949888156e-07, 3.2566634899922065e-07, 3.5369430406717584e-07, 7.326295303755614e-08, 4.85765610847011e-07, 7.717713401689252e-07, 3.4567779749750116e-08, 3.246204585138912e-07, 3.1608601602783892e-06, 5.33099466792919e-08, 3.645687343123427e-07, 5.48158936908294e-07, 4.62306957160763e-08, 1.3466177506415988e-07, 4.3529482240955986e-08, 1.6404105451783835e-07, 2.463695381038633e-08, 5.958712634424046e-08, 9.493651020875404e-08, 5.523462576206839e-08, 5.7412357534758485e-08, 1.1850350347231142e-05, 5.8263944993086625e-06, 7.4208674050169066e-06, 9.127966222877149e-07, 2.0019581370434025e-06, 1.033498961078294e-06, 3.5146850763112525e-08, 2.058995278275688e-06, 3.5655509122989315e-07, 6.873234070781109e-08, 2.1935298022413008e-09, 5.560363547374436e-08, 3.3266996979364194e-07, 1.307369217329324e-07, 2.718762992515167e-08, 1.0462929189714032e-08, 7.466680358447775e-07, 6.923166040451179e-08, 1.6145664361033596e-08, 8.568521003837759e-09, 4.76221018175238e-09, 1.233977116044116e-07, 8.340628632197422e-09, 3.2649041248333788e-09, 5.0632489312363305e-09, 4.0704994930251814e-09, 1.2043538610839732e-08, 5.105608380517879e-09, 7.267142887457112e-09, 1.184516307262129e-07, 7.53557927168913e-08, 6.386964201965384e-08, 1.6212936770898523e-08, 2.610429419291904e-07, 6.979425393183192e-07, 6.647513117741255e-08, 7.717492849224072e-07, 6.651206945207377e-07, 3.324495310152997e-07, 3.707282019149716e-07, 3.99564243025452e-07, 6.411632114122767e-08, 7.107352217872176e-08, 1.6380016631956096e-07, 6.876800995314625e-08, 3.462474467141874e-07, 2.0256503319160402e-07, 6.19610148078209e-07, 2.6841073363925716e-08, 6.720335363752383e-07, 1.1348340649419697e-06, 1.8397931853542104e-06, 6.397251581802266e-07, 7.257533241045167e-08, 4.2213909523525217e-07, 3.9657925299252383e-07, 1.4037439655112394e-07, 3.249856774800719e-07, 1.5857655455420172e-07, 1.1122217102865761e-07, 7.391420808744442e-08, 3.42322238111592e-07, 5.39796154441774e-08, 8.517296379295658e-08, 4.061009803990601e-06, 1.4478755474556237e-05, 7.317032757470088e-09, 6.9484960008026064e-09, 4.468917325084476e-08, 9.23141172393116e-08, 5.411982328951126e-08, 2.2242811326123046e-07, 1.7609554703312824e-08, 2.0906279374344194e-08, 3.6797682678724186e-09, 6.177919686933819e-08, 1.7920288541972695e-07, 2.6279179721200308e-08, 2.6988200119149042e-08, 1.6432807115052128e-07, 1.2827612749788386e-07, 4.468908798571647e-08, 6.316552969565237e-08, 1.9461760203398626e-08, 2.087125849925542e-08, 2.2414580413965268e-08, 2.4765244077684656e-08, 6.785398465325443e-09, 2.4248794971981624e-08, 4.554979504689527e-09, 2.8977037658250993e-08, 2.0402325162649504e-08, 1.600950270130852e-07, 2.0199709638291097e-07, 1.611188515937556e-08, 5.964113825029926e-08, 4.098318573397819e-09, 3.9080127578472457e-08, 7.511338218080255e-09, 5.965624154669058e-07, 1.6478223585636442e-07, 1.4106989354445432e-08, 3.2855584919389e-08, 3.3387166364917675e-09, 1.220043444050134e-08, 4.624639160510924e-08, 6.842309385746148e-09, 1.74262879681919e-08, 4.6611329906909305e-08, 9.331947836699328e-08, 1.2306078644996887e-07, 1.2359445022980253e-08, 1.1173199254699284e-08, 2.7724862405875683e-08, 2.419210147763806e-07, 3.451186785241589e-07, 2.593766978975509e-08, 9.964568192799561e-08, 9.797809674694236e-09, 1.9085564417764544e-07, 3.972706252852731e-08, 2.6639204619982593e-08, 6.874148805735558e-09, 3.146993776681484e-08, 2.4086594407890516e-07, 1.3126927456141857e-07, 2.1254339799270383e-07, 2.050203384840188e-08, 3.694976058454813e-08, 6.563175816154398e-07, 2.560050127442537e-08, 2.6882981174480847e-08, 6.880636078676616e-07, 2.0092733166166e-07, 2.788039665801989e-08, 2.628409134786125e-08, 5.1678345158734373e-08, 1.8935413947929192e-07, 4.61852835087484e-07, 1.1086777718105623e-08, 1.4542604276357451e-07, 2.8737009216683873e-08, 6.105167926762078e-07, 1.2016463379893594e-08, 1.3944705301582871e-07, 2.093712758721722e-08, 4.3801410498645055e-08, 1.966320795077081e-08, 6.654448991838535e-09, 1.1149590584125235e-08, 6.424939158478082e-08, 6.971554888934861e-09, 3.260019587614238e-09, 1.4260189473702667e-08, 2.7895078247297533e-08, 8.11578289017234e-08, 2.5995715802196173e-08, 2.2855578762914774e-08, 1.055962854934478e-07, 8.145542551574181e-08, 3.7793686402665116e-08, 4.881891513264236e-08, 2.342062366267328e-08, 1.059935517133681e-08, 3.604105103249822e-08, 5.062430830093945e-08, 3.6804440384230475e-08, 1.501580193519203e-09, 1.4475033367489232e-06, 1.076210423889279e-06, 1.304991315009829e-07, 3.073601462233455e-08, 1.7184021317007137e-08, 2.0421090596300928e-08, 7.904992216367646e-09, 1.6902052379919041e-07, 1.2416506933732308e-08, 5.4758292122869534e-08, 2.6250422280327257e-08, 1.3261367115546818e-08, 6.29807459517906e-08, 1.270998595259698e-08, 2.0171681569536304e-07, 4.386637186826192e-08, 6.962349630157405e-08, 2.9565120485131047e-07, 7.925131626507209e-07, 2.0868920103112032e-07, 1.7341794489311724e-07, 4.2942417621816276e-08, 4.213406956665722e-09, 8.824785169281313e-08, 1.7341569957807224e-08, 7.321587247588468e-08, 1.7941774288487977e-08, 1.1245148101579616e-07, 4.242405395871174e-07, 8.259573469615589e-09, 1.1336403105133286e-07, 8.268798978861014e-08, 2.2186977588489754e-08, 1.9539720952366224e-08, 1.0675703876472653e-08, 3.288517547161973e-08, 2.4340963022950746e-08, 6.639137239972115e-08, 5.604687380866835e-09, 1.386604697728444e-08, 6.675873720496384e-08, 1.1355886009312144e-08, 3.132159633878473e-07, 3.12451788886392e-08, 1.502181845580708e-07, 1.3461754377885882e-08, 1.8882955998833495e-07, 4.645742279762999e-08, 4.6453880742092224e-08, 7.714453964524637e-09, 3.5857155467056145e-08, 7.60832108426257e-09, 4.221501370693659e-08, 4.3407251126836854e-09, 1.340157496088068e-08, 8.565600495558101e-08, 1.7045413969185574e-08, 5.4221903411644234e-08, 3.021912675649219e-08, 6.153376119755194e-08, 3.938857240370908e-09, 4.135628017820636e-08, 1.781920389021252e-08, 4.3105885083605244e-08, 3.903354972578654e-09, 7.663085455078544e-08, 1.1890405993142394e-08, 9.304217840622186e-09, 1.0968062014171664e-09, 1.0536767902635802e-08, 1.1516804221400889e-07, 8.134522886393825e-07, 5.952623993721318e-08, 2.806350174466843e-08, 1.2833099027886874e-08, 1.0605690192733164e-07, 7.872949936427176e-07, 2.7501393162765453e-08, 3.936289072470345e-09, 2.0519442145428002e-08, 7.394815870753746e-09, 3.598397313453461e-08, 2.5378517065632877e-08, 4.698972233541099e-08, 7.54952989012736e-09, 6.322805461422831e-07, 5.582006412652163e-09, 1.29640980617296e-07, 1.5874988434916304e-08, 3.3837810775594335e-08, 6.474512037613067e-09, 9.121148281110436e-08, 1.3918511676536127e-08, 8.230025549949005e-09, 2.7061290097663004e-08, 2.6095918315149902e-08, 5.722363471960534e-09, 6.963475698285038e-07, 4.685091781198025e-08, 9.590579885809802e-09, 2.099205858030473e-07, 3.082160660028421e-08, 3.563162565001221e-08, 7.326312925215461e-07, 2.1759731225756695e-06, 2.407518309155421e-07, 2.974515780351794e-07, 2.529018416908002e-08, 7.667950718825978e-09, 2.663289251358947e-07, 3.4358880185436647e-08, 2.3130198201215535e-08, 3.1239693498719134e-08, 2.8691621878351725e-07, 3.895845068768722e-08, 2.4184130253956937e-08, 1.1582445225144511e-08, 5.1545349322168477e-08, 2.034345492063494e-08, 8.201963197507212e-08, 1.164153573540716e-08, 5.496356720868789e-07, 1.1682151246361627e-08, 4.7576914852243135e-08, 1.6349824605299546e-08, 4.090862759653646e-08, 2.1271189609706198e-07, 1.6697286753242224e-07, 3.989708119433999e-08, 2.852450279533514e-06, 1.2500372292834072e-07, 2.4846613655427063e-07, 1.245429093188477e-08, 2.9700272463628608e-08, 4.250991558762962e-09, 1.61443480806156e-07, 2.6386018703306036e-07, 7.638056409575711e-09, 3.4455793773702226e-09, 7.273289526210647e-08, 1.7631434090503717e-08, 7.58661311550668e-09, 2.1547013062672704e-08, 1.2675349125856883e-07, 2.5637149292379036e-08, 3.500976220038865e-08, 6.472243541111311e-08, 8.387915251262257e-09, 3.069512288789156e-08, 7.520387867998579e-08, 1.5724964441687916e-07, 1.9634005354873807e-07, 1.2290831818972947e-07, 1.112118730439704e-09, 1.546895944670723e-08, 9.91701032404535e-09, 6.882473257974198e-07, 8.267616635748709e-08, 4.469531234008173e-08, 2.075201344098332e-08, 8.649378457903367e-08, 5.202766573120243e-08, 4.5564942041664835e-08, 2.0319955496006514e-08, 8.705182352741758e-09, 6.452066969586667e-08, 2.1777438519166026e-08, 1.030954166481024e-08, 3.211904342492744e-08, 2.3336936294526822e-07, 8.054096056753224e-09, 1.9623354319264763e-07, 1.2888089884199871e-07, 1.5392496166555247e-08, 1.401903038100727e-09, 5.696818305978013e-08, 6.080025372057207e-09, 1.0782793324892737e-08, 2.4260730313585555e-08, 1.9388659566743627e-08, 2.2970310453729326e-07, 1.9971754028347277e-08, 2.8477993296860404e-08, 5.2273552597625894e-08, 2.7392806600801123e-07, 9.857291161097237e-08, 3.12910977129377e-08, 4.151442212219081e-08, 5.251196366629074e-09, 1.580681100676884e-06, 8.547603442821128e-07, 1.068913135782168e-08, 1.0621830597301596e-06, 7.737313012512459e-08, 6.394216711669287e-08, 1.1698345758759388e-07, 1.0486609625104393e-07, 2.1161000063329993e-07, 1.53396815250062e-08, 5.094453570109181e-08, 1.4005379966874898e-08, 2.6282036102998063e-08, 8.778433624456738e-08, 7.772066545896905e-09, 4.228875383205377e-08, 3.3243779284930497e-07, 7.729244799747903e-08, 7.636901111496286e-10, 5.989500806435899e-08, 1.326090597331131e-07, 1.2853634245857393e-07, 8.844242671557367e-09, 1.0194374766570036e-07, 2.493779334145074e-07, 1.6547971881664125e-07, 1.1762754326127833e-08, 1.1496195639892903e-07, 2.9342709240154363e-07, 1.326124099421122e-08, 8.630262726683213e-08, 5.7394842656322e-08, 1.1094081031615133e-07, 2.2933713239581266e-07, 3.4706170026765903e-07, 1.4751107357824367e-07, 1.502495017291494e-08, 6.454319390059027e-08, 5.164533689594464e-08, 6.23741556182722e-08, 1.293601457064142e-07, 1.4052071506398534e-08, 5.386946000385251e-08, 2.0827554791935654e-08, 1.3040637902861363e-08, 1.0578981601838677e-07, 1.5079727688771527e-08, 8.92632726845477e-07, 4.6374381668101705e-08, 7.481006036869076e-07, 5.883147302654379e-09, 2.8707685117979054e-09, 8.381598490814213e-07, 7.341958596640552e-09, 1.4245998158912698e-08, 1.0926417104428765e-07, 1.1308178216040687e-07, 2.52339901862797e-07, 1.1782835684925885e-07, 4.6678056975224536e-08, 2.7959197179683315e-09, 3.4363861090014325e-08, 1.4674496640054713e-07, 3.5396915620822256e-08, 2.0581127557761647e-07, 7.18387909159901e-08, 2.7693943138729082e-08, 4.5493386835460115e-08, 1.9559182717898693e-08, 1.5359708172013598e-08, 1.2336623278486059e-08, 2.9570605519779747e-08, 2.877552560676122e-07, 9.051845495378075e-07, 2.3732602016934834e-07, 1.6521676471370483e-08, 1.5478875070584763e-08, 3.526786329643983e-08, 3.616410637619083e-08, 1.61590953950963e-08, 7.65007328595857e-08, 1.9661483108279754e-08, 4.917534823789538e-08, 1.1712612746350715e-07, 1.0889253054813253e-08, 1.494120169809321e-06, 1.018585660261806e-08, 3.7575969003000864e-08, 2.097097784314883e-08, 3.368558054717141e-08, 4.845588819080149e-09, 6.039624622644624e-07, 1.037331109898787e-08, 2.841650257323636e-07, 4.4990630954089283e-07, 3.463186004637464e-08, 7.720684180867465e-08, 1.471122175189521e-07, 1.1601575522490748e-07, 4.007488030310924e-07, 3.025649775167949e-08, 6.706784461130155e-08, 2.0128741340386114e-08, 1.5987744461654074e-09, 4.1919822280078733e-08, 1.3167154477855547e-08, 3.231814815762846e-08, 9.247659704669786e-08, 1.3075300842047e-07, 1.0574301256838226e-07, 3.762165334819656e-08, 1.0942246575496029e-07, 7.001474955359299e-08, 2.742706151082075e-08, 2.0766625752344225e-08, 4.5403403703403455e-08, 3.39040298058535e-08, 1.0469661759771043e-07, 2.8271578855765256e-08, 3.406226767310727e-07, 5.146206945028098e-07, 6.740708613506285e-07, 6.382248063374618e-09, 3.63878704945364e-08, 3.626059807970705e-08, 1.6065602892467723e-07, 3.639055989879125e-07, 6.232691696084203e-09, 4.805490050330263e-08, 3.372633727849461e-08, 6.328880317596486e-07, 6.480631498106959e-08, 2.1165197949812864e-07, 8.38779143919055e-08, 1.7589144363228115e-08, 2.729027670511641e-09, 2.144795097080987e-08, 7.861271456022223e-08, 2.0118186228046397e-08, 2.8407685093156942e-08, 2.4922530883486615e-07, 2.0156670998972004e-08, 2.6551649767725394e-08, 2.7848242822869906e-08, 6.907123761834555e-09, 1.880543720744754e-08, 1.3006903998302732e-08, 3.685918272822164e-07, 3.967941211158177e-07, 2.7592133022835696e-08, 2.5228947819755376e-08, 1.547002881352455e-07, 3.689306637966183e-08, 1.440177199718562e-09, 2.1504929392790473e-08, 5.068111263994979e-08, 5.081711407228795e-08, 1.171875219085905e-08, 5.409278358570191e-08, 7.138276600926474e-07, 2.5237213208129106e-07, 7.072044638789521e-08, 7.199763984999663e-08, 1.2525473103153217e-08, 3.4803417747752974e-07, 1.9591827538079087e-07, 1.2404700555634918e-07, 1.234617457157583e-07, 1.9201337408958352e-08, 1.9895249181445251e-07, 3.7876677794201896e-08, 1.0629785052174157e-08, 1.2437127772102485e-08, 2.1861892207653e-07, 2.6181456291851646e-07, 1.112900775979142e-07, 1.0776630432474121e-07, 6.380325157095967e-09, 3.895085143312826e-09, 1.5762756788717525e-07, 2.909027019271093e-09, 1.0381050685737137e-08, 2.8135211493918177e-08, 1.0778002490496874e-08, 1.3605974125141529e-08, 2.9236465692861202e-08, 1.9189795352758665e-07, 2.199506354827463e-07, 1.326399790002597e-08, 4.9004846403022384e-08, 2.980837132682268e-09, 8.926045680368588e-09, 1.0996975774446582e-08, 7.71560149104289e-09, 7.454491246505768e-09, 5.086162246925596e-08, 1.5129764108223753e-07, 1.1960075596562092e-08, 1.1323334270230134e-08, 9.391332156383214e-09, 9.585701832293125e-08, 1.905532798218701e-08, 1.8105303922766325e-08, 6.179227796110354e-08, 6.389401363549041e-08, 1.1853179771037503e-08, 9.37277544466042e-09, 1.2332148457971925e-07, 1.6522022860954166e-08, 1.246116454467483e-07, 4.196171854431441e-09, 3.996593278543514e-08, 1.2554556505506298e-08, 1.4302138140465104e-08, 6.631793780798034e-09, 5.964224669696705e-09, 5.556936244488497e-09, 1.4192455921602232e-07, 1.7613080771639034e-08, 3.380189639301534e-07, 7.85651934620546e-08, 2.966783085867064e-08, 2.8992105853831163e-06, 1.3787366697215475e-06, 5.313622430946907e-09, 2.512852859126724e-08, 8.406627216572815e-08, 4.492839167369311e-08, 5.408793057881667e-08, 2.4239175999696272e-08, 4.016805235096399e-07, 4.1083545454512205e-08, 5.4153481698904216e-08, 8.640767212853007e-09, 5.773256717134245e-08, 2.6443152023603034e-07, 8.953217047746875e-07, 2.7994001783326894e-08, 5.889480014786841e-09, 4.1788819515886644e-08, 2.8880645430717777e-08, 2.135752907861388e-08, 2.3024175277441827e-07, 8.786625471657317e-08, 2.0697297209437693e-09, 2.236410523437371e-08, 3.203276310870251e-09, 1.176874686592555e-08, 6.963571053120177e-08, 2.271932153519174e-08, 7.360382525689602e-09, 6.922528772435044e-09, 3.213871480056696e-08, 1.370577820125618e-07, 1.9815049157045905e-08, 1.0578956377571558e-08, 2.7049420481262132e-08, 2.9755937713815683e-09, 2.1773699288019088e-08, 1.09755387001087e-08, 1.991872444762066e-08, 2.3882098076910552e-08, 2.1357365653784655e-08, 6.109098560358461e-09, 1.1890497475519624e-08, 1.1459891702259029e-08, 3.73173456580389e-08, 1.572620256240498e-08, 3.404023374287135e-08, 3.6921580459647885e-08, 9.281765045443535e-08, 1.2323201303843234e-07, 4.2347593876002065e-08, 1.7423728237986325e-08, 5.8113389656000436e-08, 3.931436154402945e-08, 2.3690461148362374e-08, 1.792850135018398e-08, 1.440664210150544e-08, 7.019830494670032e-09, 6.041522482291839e-08, 4.867479930226182e-08, 1.0685319296044327e-08, 1.0051243393149889e-08, 4.2426261614991745e-08, 2.607815297039906e-08, 5.136670200300 841e-09, 1.69729952315123e-09, 1.9131586981302462e-08, 2.111743526711507e-07, 1.337269672774255e-08, 2.0002481448955223e-08, 1.0454256482717028e-07, 2.8144228281234973e-08, 2.1344791889532644e-07, 2.1046110632028103e-08, 1.9114453664315079e-07, 3.957693550660224e-08, 2.931631826186276e-08, 1.105203111251285e-07, 4.84007678380749e-08, 5.583606110803885e-08, 1.2130111315400427e-07, 1.77621615193857e-08, 2.5610853882085394e-08, 1.203865309662433e-07, 4.674859610531712e-09, 1.5916098661250544e-08, 3.147594185293201e-08, 6.147686093527227e-08, 2.204641802450169e-08, 3.257763410147163e-07, 1.198914532096751e-07, 2.3818989802748547e-07, 1.4909986134625797e-08, 5.10168831624469e-08, 5.5142201915714395e-08, 2.288550327023131e-08, 5.714110073995471e-08, 5.185095801607531e-07, 4.977285783525076e-08, 1.1049896109227575e-08, 1.264099296349741e-07, 8.174881571676451e-08]]}
* Connection #0 to host localhost left intact

Deploy the model with Open Inference Protocol

Test the Model locally

Once you've got your model serialised model.pdmodel, we can then use KServe Paddle Server to spin up a local server.

Note

This step is optional and just meant for testing, feel free to jump straight to deploying with InferenceService.

Using KServe PaddleServer

Pre-requisites

Firstly, to use KServe Paddle server locally, you will first need to install the paddleserver runtime package in your local environment.

  1. Clone the KServe repository and navigate into the directory. 
    git clone https://github.com/kserve/kserve
  2. Install paddleserver runtime. Kserve uses Poetry as the dependency management tool. Make sure you have already installed poetry. 
    cd python/paddleserver
poetry install

Serving model locally

The paddleserver package takes two arguments.

  • --model_dir: The model directory path where the model is stored.
  • --model_name: The name of the model deployed in the model server, the default value is model. This is optional.

With the paddleserver runtime package installed locally, you should now be ready to start our server as:

python3 paddleserver --model_dir /path/to/model_dir --model_name paddle-v2-resnet50

Deploy the Model with REST endpoint through InferenceService

Lastly, you will use KServe to deploy the trained model onto Kubernetes. For this, you will just need to use version v1beta1 of the InferenceService CRD and set the protocolVersion field to v2.

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "paddle-v2-resnet50"
spec:
  predictor:
    model:
      modelFormat:
        name: paddle
      protocolVersion: v2
      runtime: kserve-paddleserver
      storageUri: "gs://kfserving-examples/models/paddle/resnet"

Apply the InferenceService yaml to get the REST endpoint

kubectl apply -f paddle-v2-resnet.yaml

Test the Deployed Model

You can now test your deployed model by sending a sample request.

Note that this request needs to follow the Open Inference Protocol.

You can use the example payload jay-v2.json as the sample input to test the model.

Determine the ingress IP and port and set INGRESS_HOST and INGRESS_PORT. Now, you can use curl to send the inference request as:

SERVICE_HOSTNAME=$(kubectl get inferenceservice paddle-v2-resnet50 -o jsonpath='{.status.url}' | cut -d "/" -f 3)

curl -v \
  -H "Host: ${SERVICE_HOSTNAME}" \
  -H "Content-Type: application/json" \
  -d @./jay-v2.json \
  http://${INGRESS_HOST}:${INGRESS_PORT}/v2/models/paddle-v2-resnet50/infer

Expected Output

{"model_name":"paddle-v2-resnet50","model_version":null,"id":"afa5ec0b-a5c7-454b-a464-53ba931b22df","parameters":null,"outputs":[{"name":"output-0","shape":[1,1000],"datatype":"FP32","parameters":null,"data":[6.736678770380422e-9,1.1535990829258935e-8,5.142250714129659e-8,6.647170636142619e-8,4.094492567219277e-8,1.3402451770616608e-7,9.355561303436843e-8,2.8935891904779965e-8,6.845367295227334e-8,7.680615965455218e-8,0.0000020334689452283783,0.0000011085678579547675,2.3477592492326949e-7,6.582037030966603e-7,0.00012373103527352214,4.2878804151769145e-7,0.000006419959845516132,0.9993496537208557,0.00007372002437477931,0.00003101135735050775,0.0000056028093240456656,0.0000021862508674530545,1.9544044604913324e-8,3.728893887000595e-7,4.2903633357127546e-7,1.8251179767503345e-7,7.159925985433802e-8,9.231618136595898e-9,6.469241498052725e-7,7.031690341108288e-9,4.451231561120039e-8,1.2455971898361895e-7,9.44632745358831e-8,4.347704418705689e-8,4.658220120745682e-7,6.797721141538204e-8,2.1060276367279585e-7,2.2605123106700376e-8,1.4311490303953178e-7,7.951298641728499e-8,1.2341783417468832e-7,0.0000010921713737843675,0.000015243892448779661,3.1173343018053856e-7,2.4152058131221565e-7,6.863762536113427e-8,8.467682022228473e-8,9.4246772164297e-8,1.0219210366813058e-8,3.3770753304906975e-8,3.6928835100979995e-8,1.3694031508748594e-7,1.0674284567357972e-7,2.599483650556067e-7,3.4866405940192635e-7,3.132053549848024e-8,3.574873232992104e-7,6.64843895492595e-8,3.1638955988455564e-7,0.0000012095878219042788,8.66409024524728e-8,4.0144172430700564e-8,1.2544761318622477e-7,3.3201178695208e-8,1.9731444922399533e-7,3.806405572959193e-7,1.3827865075199952e-7,2.300225965257141e-8,7.14422512260171e-8,2.851114544455413e-8,2.982567437470607e-8,8.936032713791064e-8,6.22388370175031e-7,6.478838798784636e-8,1.3663023423760023e-7,9.973181391842445e-8,2.5761554667269593e-8,4.130220077058766e-8,3.9384463690339544e-8,1.2158079698565416e-7,0.000004302821707824478,0.0000018179063090428826,0.0000018520155435908237,0.0000016246107179540559,0.000016448313544970006,0.000010544916221988387,0.000003993061909568496,2.646479799750523e-7,0.000019193475964129902,4.803242745765601e-7,1.696285067964709e-7,0.000004550505764200352,0.00004235929372953251,0.000004443338639248395,0.000005104009687784128,0.000013506396498996764,4.1758724478313525e-7,4.494491463447048e-7,3.156698369366495e-7,0.0000010557599807725637,1.336463917311903e-8,1.3893659556174498e-8,6.770379457066156e-8,1.4129696523923485e-7,7.170518756538513e-8,7.934466594861078e-8,2.639154317307657e-8,2.6134321373660896e-8,7.196725881897237e-9,2.1752363466021052e-8,6.684639686227456e-8,3.417795824134373e-8,1.6228275967478112e-7,4.107114648377319e-7,6.472135396506928e-7,2.951379372007068e-7,5.653474133282543e-9,4.830144462175667e-8,8.887481861563629e-9,3.7306168820805397e-8,1.7784264727538357e-8,4.641905082536368e-9,3.413118676576232e-8,1.937393818707278e-7,0.0000012980176506971475,3.5641004814124244e-8,2.149332445355867e-8,3.055293689158134e-7,1.5532516783878236e-7,0.0000014520978766086046,3.488464628276233e-8,0.00003825438398052938,4.5088432898410247e-7,4.1766969616219285e-7,6.770622462681786e-7,1.4142248971893423e-7,0.000014235997696232516,6.293820433711517e-7,0.000004762866865348769,9.024900577969674e-7,9.058987870957935e-7,0.0000015713684433649178,1.5720647184025438e-7,1.818536503606083e-7,7.193188622522939e-8,0.0000011952824934269302,8.874837362782273e-7,2.0870831463071227e-7,9.906239029078279e-8,7.793621747964607e-9,1.0058498389753368e-7,4.2059440374941914e-7,1.843624630737395e-7,1.6437947181202617e-7,7.025352743994517e-8,2.570448600636155e-7,7.586877615040066e-8,7.841313731660193e-7,2.495309274763713e-7,5.157681925993529e-8,4.0674127177453556e-8,7.531796519799627e-9,4.797485431140558e-8,1.7419973019627832e-8,1.7958679165985814e-7,1.2566392371127222e-8,8.975440124459055e-8,3.26965476915575e-8,1.1208359751435637e-7,3.906746215420753e-8,4.6769045525252295e-8,1.8523553535487736e-7,1.4833052830454108e-7,1.2279349448363064e-7,0.0000010729105497375713,3.6538490011395197e-9,1.6198403329781286e-7,1.6190719875908144e-8,1.2004933580556099e-7,1.4800277448046018e-8,4.02294837442696e-8,2.15060893538066e-7,1.1925696696835075e-7,4.8982514044837444e-8,7.608920071788816e-8,2.3137479487900237e-8,8.521050176568679e-8,9.586213423062873e-8,1.3351650807180704e-7,3.021699157557123e-8,4.423876376336011e-8,2.610667060309879e-8,2.3977091245797055e-7,1.3192564551900432e-7,1.6734931662654162e-8,1.588336999702733e-7,4.0643516285854275e-7,8.753454494581092e-8,8.366999395548191e-7,3.437598650180007e-8,7.847892646850596e-8,8.526394701391382e-9,9.601382799928615e-8,5.258924034023948e-7,1.3557448141909845e-7,1.0307226716577134e-7,1.0429813457335513e-8,5.187714435805901e-8,2.187001335585137e-8,1.1791439824548888e-8,2.98065643278278e-8,4.338393466696289e-8,2.9991046091026874e-8,2.8507610494443725e-8,3.058665143385042e-8,6.441099031917474e-8,1.5364101102477434e-8,1.5973883549236234e-8,2.5736850872704053e-8,1.0903765712555469e-7,3.2118737891551064e-8,6.819742992547617e-9,1.9251311300649832e-7,5.8258109447706374e-8,1.8765761922168167e-7,
4.0070790419122204e-7,1.5791577823165426e-8,1.950158434738114e-7,1.0142063189277906e-8,2.744815041921811e-8,1.2843531571604672e-8,3.7297493094001766e-8,7.407496838141014e-8,4.20607833007125e-8,1.6924804668860816e-8,1.459203531339881e-7,4.344977000414474e-8,1.7191403856031684e-7,3.5817443233554513e-8,8.440249388286247e-9,4.194829728021432e-8,2.514032360068086e-8,2.8340199520471288e-8,8.747196034164517e-8,8.277125651545703e-9,1.1676293709683705e-8,1.4548514570833504e-7,7.200282148289716e-9,0.000002623600948936655,5.675736929333652e-7,0.0000019483527466945816,6.752595282932816e-8,8.168475318370838e-8,1.0933046468153407e-7,1.670913718498923e-7,3.1387276777650186e-8,2.973524537708272e-8,5.752163900751839e-8,5.850877471402782e-8,3.2544622285968217e-7,3.330221431951941e-8,4.186786668469722e-7,1.5085906568401697e-7,2.3346819943981245e-7,2.86402780602657e-7,2.2940319865938363e-7,1.8537603807544656e-7,3.151798182443599e-7,0.0000011075967449869495,1.5369782602192572e-7,1.9237509718550427e-7,1.64044664074936e-7,2.900835340824415e-7,1.246654903752642e-7,5.802622027317739e-8,5.186220519703966e-8,6.0094205167615655e-9,1.2333241272699524e-7,1.3798474185477971e-7,1.7370231830682314e-7,5.617761189569137e-7,5.1604470030497396e-8,4.813277598714194e-8,8.032698417537176e-8,0.0000020645263703045202,5.638597713186755e-7,8.794199857220519e-7,0.0000034785980460583232,2.972389268052211e-7,3.3904532870110415e-7,9.469074058188198e-8,3.754845678827223e-8,1.5679037801419327e-7,8.203105039683578e-8,6.847962641387539e-9,1.8251624211984563e-8,6.050240841659615e-8,3.956342808919544e-8,1.0699947949888156e-7,3.2566634899922065e-7,3.5369430406717584e-7,7.326295303755614e-8,4.85765610847011e-7,7.717713401689252e-7,3.4567779749750116e-8,3.246204585138912e-7,0.0000031608601602783892,5.33099466792919e-8,3.645687343123427e-7,5.48158936908294e-7,4.62306957160763e-8,1.3466177506415988e-7,4.3529482240955986e-8,1.6404105451783835e-7,2.463695381038633e-8,5.958712634424046e-8,9.493651020875404e-8,5.523462576206839e-8,5.7412357534758485e-8,0.000011850350347231142,0.0000058263944993086625,0.0000074208674050169066,9.127966222877149e-7,0.0000020019581370434025,0.000001033498961078294,3.5146850763112525e-8,0.000002058995278275688,3.5655509122989315e-7,6.873234070781109e-8,2.1935298022413008e-9,5.560363547374436e-8,3.3266996979364194e-7,1.307369217329324e-7,2.718762992515167e-8,1.0462929189714032e-8,7.466680358447775e-7,6.923166040451179e-8,1.6145664361033596e-8,8.568521003837759e-9,4.76221018175238e-9,1.233977116044116e-7,8.340628632197422e-9,3.2649041248333788e-9,5.0632489312363305e-9,4.0704994930251814e-9,1.2043538610839732e-8,5.105608380517879e-9,7.267142887457112e-9,1.184516307262129e-7,7.53557927168913e-8,6.386964201965384e-8,1.6212936770898523e-8,2.610429419291904e-7,6.979425393183192e-7,6.647513117741255e-8,7.717492849224072e-7,6.651206945207377e-7,3.324495310152997e-7,3.707282019149716e-7,3.99564243025452e-7,6.411632114122767e-8,7.107352217872176e-8,1.6380016631956096e-7,6.876800995314625e-8,3.462474467141874e-7,2.0256503319160402e-7,6.19610148078209e-7,2.6841073363925716e-8,6.720335363752383e-7,0.0000011348340649419697,0.0000018397931853542104,6.397251581802266e-7,7.257533241045167e-8,4.2213909523525217e-7,3.9657925299252383e-7,1.4037439655112394e-7,3.249856774800719e-7,1.5857655455420172e-7,1.1122217102865761e-7,7.391420808744442e-8,3.42322238111592e-7,5.39796154441774e-8,8.517296379295658e-8,0.000004061009803990601,0.000014478755474556237,7.317032757470088e-9,6.9484960008026064e-9,4.468917325084476e-8,9.23141172393116e-8,5.411982328951126e-8,2.2242811326123046e-7,1.7609554703312824e-8,2.0906279374344194e-8,3.6797682678724186e-9,6.177919686933819e-8,1.7920288541972695e-7,2.6279179721200308e-8,2.6988200119149042e-8,1.6432807115052128e-7,1.2827612749788386e-7,4.468908798571647e-8,6.316552969565237e-8,1.9461760203398626e-8,2.087125849925542e-8,2.2414580413965268e-8,2.4765244077684656e-8,6.785398465325443e-9,2.4248794971981624e-8,4.554979504689527e-9,2.8977037658250993e-8,2.0402325162649504e-8,1.600950270130852e-7,2.0199709638291097e-7,1.611188515937556e-8,5.964113825029926e-8,4.098318573397819e-9,3.9080127578472457e-8,7.511338218080255e-9,5.965624154669058e-7,1.6478223585636442e-7,1.4106989354445432e-8,3.2855584919389e-8,3.3387166364917675e-9,1.220043444050134e-8,4.624639160510924e-8,6.842309385746148e-9,1.74262879681919e-8,4.6611329906909305e-8,9.331947836699328e-8,1.2306078644996887e-7,1.2359445022980253e-8,1.1173199254699284e-8,2.7724862405875683e-8,2.419210147763806e-7,3.451186785241589e-7,2.593766978975509e-8,9.964568192799561e-8,9.797809674694236e-9,1.9085564417764544e-7,3.972706252852731e-8,2.6639204619982593e-8,6.874148805735558e-9,3.146993776681484e-8,2.4086594407890516e-7,1.3126927456141857e-7,2.1254339799270383e-7,2.050203384840188e-8,3.694976058454813e-8,6.563175816154398e-7,2.560050127442537e-8,2.6882981174480847e-8,6.880636078676616e-7,2.0092733166166e-7,2.788039665801989e-8,2.628409134786125e-8,5.1678345158734373e-8,1.8935413947929192e-7,4.61852835087484e-7,1.1086777718105623e-8,1.4542604276357451e-7,2.8737009216683873e-8,6.105167926762078e-7,
1.2016463379893594e-8,1.3944705301582871e-7,2.093712758721722e-8,4.3801410498645055e-8,1.966320795077081e-8,6.654448991838535e-9,1.1149590584125235e-8,6.424939158478082e-8,6.971554888934861e-9,3.260019587614238e-9,1.4260189473702667e-8,2.7895078247297533e-8,8.11578289017234e-8,2.5995715802196173e-8,2.2855578762914774e-8,1.055962854934478e-7,8.145542551574181e-8,3.7793686402665116e-8,4.881891513264236e-8,2.342062366267328e-8,1.059935517133681e-8,3.604105103249822e-8,5.062430830093945e-8,3.6804440384230475e-8,1.501580193519203e-9,0.0000014475033367489232,0.000001076210423889279,1.304991315009829e-7,3.073601462233455e-8,1.7184021317007137e-8,2.0421090596300928e-8,7.904992216367646e-9,1.6902052379919041e-7,1.2416506933732308e-8,5.4758292122869534e-8,2.6250422280327257e-8,1.3261367115546818e-8,6.29807459517906e-8,1.270998595259698e-8,2.0171681569536304e-7,4.386637186826192e-8,6.962349630157405e-8,2.9565120485131047e-7,7.925131626507209e-7,2.0868920103112032e-7,1.7341794489311724e-7,4.2942417621816276e-8,4.213406956665722e-9,8.824785169281313e-8,1.7341569957807224e-8,7.321587247588468e-8,1.7941774288487977e-8,1.1245148101579616e-7,4.242405395871174e-7,8.259573469615589e-9,1.1336403105133286e-7,8.268798978861014e-8,2.2186977588489754e-8,1.9539720952366224e-8,1.0675703876472653e-8,3.288517547161973e-8,2.4340963022950746e-8,6.639137239972115e-8,5.604687380866835e-9,1.386604697728444e-8,6.675873720496384e-8,1.1355886009312144e-8,3.132159633878473e-7,3.12451788886392e-8,1.502181845580708e-7,1.3461754377885882e-8,1.8882955998833495e-7,4.645742279762999e-8,4.6453880742092224e-8,7.714453964524637e-9,3.5857155467056145e-8,7.60832108426257e-9,4.221501370693659e-8,4.3407251126836854e-9,1.340157496088068e-8,8.565600495558101e-8,1.7045413969185574e-8,5.4221903411644234e-8,3.021912675649219e-8,6.153376119755194e-8,3.938857240370908e-9,4.135628017820636e-8,1.781920389021252e-8,4.3105885083605244e-8,3.903354972578654e-9,7.663085455078544e-8,1.1890405993142394e-8,9.304217840622186e-9,1.0968062014171664e-9,1.0536767902635802e-8,1.1516804221400889e-7,8.134522886393825e-7,5.952623993721318e-8,2.806350174466843e-8,1.2833099027886874e-8,1.0605690192733164e-7,7.872949936427176e-7,2.7501393162765453e-8,3.936289072470345e-9,2.0519442145428002e-8,7.394815870753746e-9,3.598397313453461e-8,2.5378517065632877e-8,4.698972233541099e-8,7.54952989012736e-9,6.322805461422831e-7,5.582006412652163e-9,1.29640980617296e-7,1.5874988434916304e-8,3.3837810775594335e-8,6.474512037613067e-9,9.121148281110436e-8,1.3918511676536127e-8,8.230025549949005e-9,2.7061290097663004e-8,2.6095918315149902e-8,5.722363471960534e-9,6.963475698285038e-7,4.685091781198025e-8,9.590579885809802e-9,2.099205858030473e-7,3.082160660028421e-8,3.563162565001221e-8,7.326312925215461e-7,0.0000021759731225756695,2.407518309155421e-7,2.974515780351794e-7,2.529018416908002e-8,7.667950718825978e-9,2.663289251358947e-7,3.4358880185436647e-8,2.3130198201215535e-8,3.1239693498719134e-8,2.8691621878351725e-7,3.895845068768722e-8,2.4184130253956937e-8,1.1582445225144511e-8,5.1545349322168477e-8,2.034345492063494e-8,8.201963197507212e-8,1.164153573540716e-8,5.496356720868789e-7,1.1682151246361627e-8,4.7576914852243135e-8,1.6349824605299546e-8,4.090862759653646e-8,2.1271189609706198e-7,1.6697286753242224e-7,3.989708119433999e-8,0.000002852450279533514,1.2500372292834072e-7,2.4846613655427063e-7,1.245429093188477e-8,2.9700272463628608e-8,4.250991558762962e-9,1.61443480806156e-7,2.6386018703306036e-7,7.638056409575711e-9,3.4455793773702226e-9,7.273289526210647e-8,1.7631434090503717e-8,7.58661311550668e-9,2.1547013062672704e-8,1.2675349125856883e-7,2.5637149292379036e-8,3.500976220038865e-8,6.472243541111311e-8,8.387915251262257e-9,3.069512288789156e-8,7.520387867998579e-8,1.5724964441687916e-7,1.9634005354873807e-7,1.2290831818972947e-7,1.112118730439704e-9,1.546895944670723e-8,9.91701032404535e-9,6.882473257974198e-7,8.267616635748709e-8,4.469531234008173e-8,2.075201344098332e-8,8.649378457903367e-8,5.202766573120243e-8,4.5564942041664835e-8,2.0319955496006514e-8,8.705182352741758e-9,6.452066969586667e-8,2.1777438519166026e-8,1.030954166481024e-8,3.211904342492744e-8,2.3336936294526822e-7,8.054096056753224e-9,1.9623354319264763e-7,1.2888089884199871e-7,1.5392496166555247e-8,1.401903038100727e-9,5.696818305978013e-8,6.080025372057207e-9,1.0782793324892737e-8,2.4260730313585555e-8,1.9388659566743627e-8,2.2970310453729326e-7,1.9971754028347277e-8,2.8477993296860404e-8,5.2273552597625894e-8,2.7392806600801123e-7,9.857291161097237e-8,3.12910977129377e-8,4.151442212219081e-8,5.251196366629074e-9,0.000001580681100676884,8.547603442821128e-7,1.068913135782168e-8,0.0000010621830597301596,7.737313012512459e-8,6.394216711669287e-8,1.1698345758759388e-7,1.0486609625104393e-7,2.1161000063329993e-7,1.53396815250062e-8,5.094453570109181e-8,1.4005379966874898e-8,2.6282036102998063e-8,8.778433624456738e-8,7.772066545896905e-9,4.228875383205377e-8,3.3243779284930497e-7,7.729244799747903e-8,7.636901111496286e-10,5.989500806435899e-8,1.326090597331131e-7,1.2853634245857393e-7,8.844242671557367e-9,1.0194374766570036e-7,
2.493779334145074e-7,1.6547971881664125e-7,1.1762754326127833e-8,1.1496195639892903e-7,2.9342709240154363e-7,1.326124099421122e-8,8.630262726683213e-8,5.7394842656322e-8,1.1094081031615133e-7,2.2933713239581266e-7,3.4706170026765903e-7,1.4751107357824367e-7,1.502495017291494e-8,6.454319390059027e-8,5.164533689594464e-8,6.23741556182722e-8,1.293601457064142e-7,1.4052071506398534e-8,5.386946000385251e-8,2.0827554791935654e-8,1.3040637902861363e-8,1.0578981601838677e-7,1.5079727688771527e-8,8.92632726845477e-7,4.6374381668101705e-8,7.481006036869076e-7,5.883147302654379e-9,2.8707685117979054e-9,8.381598490814213e-7,7.341958596640552e-9,1.4245998158912698e-8,1.0926417104428765e-7,1.1308178216040687e-7,2.52339901862797e-7,1.1782835684925885e-7,4.6678056975224536e-8,2.7959197179683315e-9,3.4363861090014325e-8,1.4674496640054713e-7,3.5396915620822256e-8,2.0581127557761647e-7,7.18387909159901e-8,2.7693943138729082e-8,4.5493386835460115e-8,1.9559182717898693e-8,1.5359708172013598e-8,1.2336623278486059e-8,2.9570605519779747e-8,2.877552560676122e-7,9.051845495378075e-7,2.3732602016934834e-7,1.6521676471370483e-8,1.5478875070584763e-8,3.526786329643983e-8,3.616410637619083e-8,1.61590953950963e-8,7.65007328595857e-8,1.9661483108279754e-8,4.917534823789538e-8,1.1712612746350715e-7,1.0889253054813253e-8,0.000001494120169809321,1.018585660261806e-8,3.7575969003000864e-8,2.097097784314883e-8,3.368558054717141e-8,4.845588819080149e-9,6.039624622644624e-7,1.037331109898787e-8,2.841650257323636e-7,4.4990630954089283e-7,3.463186004637464e-8,7.720684180867465e-8,1.471122175189521e-7,1.1601575522490748e-7,4.007488030310924e-7,3.025649775167949e-8,6.706784461130155e-8,2.0128741340386114e-8,1.5987744461654074e-9,4.1919822280078733e-8,1.3167154477855547e-8,3.231814815762846e-8,9.247659704669786e-8,1.3075300842047e-7,1.0574301256838226e-7,3.762165334819656e-8,1.0942246575496029e-7,7.001474955359299e-8,2.742706151082075e-8,2.0766625752344225e-8,4.5403403703403455e-8,3.39040298058535e-8,1.0469661759771043e-7,2.8271578855765256e-8,3.406226767310727e-7,5.146206945028098e-7,6.740708613506285e-7,6.382248063374618e-9,3.63878704945364e-8,3.626059807970705e-8,1.6065602892467723e-7,3.639055989879125e-7,6.232691696084203e-9,4.805490050330263e-8,3.372633727849461e-8,6.328880317596486e-7,6.480631498106959e-8,2.1165197949812864e-7,8.38779143919055e-8,1.7589144363228115e-8,2.729027670511641e-9,2.144795097080987e-8,7.861271456022223e-8,2.0118186228046397e-8,2.8407685093156942e-8,2.4922530883486615e-7,2.0156670998972004e-8,2.6551649767725394e-8,2.7848242822869906e-8,6.907123761834555e-9,1.880543720744754e-8,1.3006903998302732e-8,3.685918272822164e-7,3.967941211158177e-7,2.7592133022835696e-8,2.5228947819755376e-8,1.547002881352455e-7,3.689306637966183e-8,1.440177199718562e-9,2.1504929392790473e-8,5.068111263994979e-8,5.081711407228795e-8,1.171875219085905e-8,5.409278358570191e-8,7.138276600926474e-7,2.5237213208129106e-7,7.072044638789521e-8,7.199763984999663e-8,1.2525473103153217e-8,3.4803417747752974e-7,1.9591827538079087e-7,1.2404700555634918e-7,1.234617457157583e-7,1.9201337408958352e-8,1.9895249181445251e-7,3.7876677794201896e-8,1.0629785052174157e-8,1.2437127772102485e-8,2.1861892207653e-7,2.6181456291851646e-7,1.112900775979142e-7,1.0776630432474121e-7,6.380325157095967e-9,3.895085143312826e-9,1.5762756788717525e-7,2.909027019271093e-9,1.0381050685737137e-8,2.8135211493918177e-8,1.0778002490496874e-8,1.3605974125141529e-8,2.9236465692861202e-8,1.9189795352758665e-7,2.199506354827463e-7,1.326399790002597e-8,4.9004846403022384e-8,2.980837132682268e-9,8.926045680368588e-9,1.0996975774446582e-8,7.71560149104289e-9,7.454491246505768e-9,5.086162246925596e-8,1.5129764108223753e-7,1.1960075596562092e-8,1.1323334270230134e-8,9.391332156383214e-9,9.585701832293125e-8,1.905532798218701e-8,1.8105303922766325e-8,6.179227796110354e-8,6.389401363549041e-8,1.1853179771037503e-8,9.37277544466042e-9,1.2332148457971925e-7,1.6522022860954166e-8,1.246116454467483e-7,4.196171854431441e-9,3.996593278543514e-8,1.2554556505506298e-8,1.4302138140465104e-8,6.631793780798034e-9,5.964224669696705e-9,5.556936244488497e-9,1.4192455921602232e-7,1.7613080771639034e-8,3.380189639301534e-7,7.85651934620546e-8,2.966783085867064e-8,0.0000028992105853831163,0.0000013787366697215475,5.313622430946907e-9,2.512852859126724e-8,8.406627216572815e-8,4.492839167369311e-8,5.408793057881667e-8,2.4239175999696272e-8,4.016805235096399e-7,4.1083545454512205e-8,5.4153481698904216e-8,8.640767212853007e-9,5.773256717134245e-8,2.6443152023603034e-7,8.953217047746875e-7,2.7994001783326894e-8,5.889480014786841e-9,4.1788819515886644e-8,2.8880645430717777e-8,2.135752907861388e-8,2.3024175277441827e-7,8.786625471657317e-8,2.0697297209437693e-9,2.236410523437371e-8,3.203276310870251e-9,1.176874686592555e-8,6.963571053120177e-8,2.271932153519174e-8,7.360382525689602e-9,6.922528772435044e-9,3.213871480056696e-8,1.370577820125618e-7,1.9815049157045905e-8,1.0578956377571558e-8,2.7049420481262132e-8,2.9755937713815683e-9,2.1773699288019088e-8,1.09755387001087e-8,1.991872444762066e-8,2.3882098076910552e-8,2.1357365653784655e-8,
6.109098560358461e-9,1.1890497475519624e-8,1.1459891702259029e-8,3.73173456580389e-8,1.572620256240498e-8,3.404023374287135e-8,3.6921580459647885e-8,9.281765045443535e-8,1.2323201303843234e-7,4.2347593876002065e-8,1.7423728237986325e-8,5.8113389656000436e-8,3.931436154402945e-8,2.3690461148362374e-8,1.792850135018398e-8,1.440664210150544e-8,7.019830494670032e-9,6.041522482291839e-8,4.867479930226182e-8,1.0685319296044327e-8,1.0051243393149889e-8,4.2426261614991745e-8,2.607815297039906e-8,5.136670200300841e-9,1.69729952315123e-9,1.9131586981302462e-8,2.111743526711507e-7,1.337269672774255e-8,2.0002481448955223e-8,1.0454256482717028e-7,2.8144228281234973e-8,2.1344791889532644e-7,2.1046110632028103e-8,1.9114453664315079e-7,3.957693550660224e-8,2.931631826186276e-8,1.105203111251285e-7,4.84007678380749e-8,5.583606110803885e-8,1.2130111315400427e-7,1.77621615193857e-8,2.5610853882085394e-8,1.203865309662433e-7,4.674859610531712e-9,1.5916098661250544e-8,3.147594185293201e-8,6.147686093527227e-8,2.204641802450169e-8,3.257763410147163e-7,1.198914532096751e-7,2.3818989802748547e-7,1.4909986134625797e-8,5.10168831624469e-8,5.5142201915714395e-8,2.288550327023131e-8,5.714110073995471e-8,5.185095801607531e-7,4.977285783525076e-8,1.1049896109227575e-8,1.264099296349741e-7,8.174881571676451e-8]}]}

Deploy the Model with GRPC endpoint through InferenceService

Create the inference service resource and expose the gRPC port using the below yaml.

Note

Currently, KServe only supports exposing either HTTP or gRPC port. By default, HTTP port is exposed.

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "paddle-v2-resnet50-grpc"
spec:
  predictor:
    model:
      modelFormat:
        name: paddle
      protocolVersion: v2
      runtime: kserve-paddleserver
      storageUri: "gs://kfserving-examples/models/paddle/resnet"
      ports:
        - name: h2c     # knative expects grpc port name to be 'h2c'
          protocol: TCP
          containerPort: 8081

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "paddle-v2-resnet50-grpc"
spec:
  predictor:
    model:
      modelFormat:
        name: paddle
      protocolVersion: v2
      runtime: kserve-paddleserver
      storageUri: "gs://kfserving-examples/models/paddle/resnet"
      ports:
        - name: grpc-port  # Istio requires the port name to be in the format <protocol>[-<suffix>]
          protocol: TCP
          containerPort: 8081

Apply the InferenceService yaml to get the gRPC endpoint

kubectl apply -f paddle-v2-grpc.yaml

Test the deployed model with grpcurl

After the gRPC InferenceService becomes ready, grpcurl, can be used to send gRPC requests to the InferenceService.

# download the proto file
curl -O https://raw.githubusercontent.com/kserve/open-inference-protocol/main/specification/protocol/open_inference_grpc.proto

INPUT_PATH=jay-v2-grpc.json
PROTO_FILE=open_inference_grpc.proto
SERVICE_HOSTNAME=$(kubectl get inferenceservice paddle-v2-resnet50-grpc -o jsonpath='{.status.url}' | cut -d "/" -f 3)

Determine the ingress IP and port and set INGRESS_HOST and INGRESS_PORT. Now, you can use curl to send the inference requests. The gRPC APIs follows the KServe prediction V2 protocol / Open Inference Protocol. For example, ServerReady API can be used to check if the server is ready:

grpcurl \
  -plaintext \
  -proto ${PROTO_FILE} \
  -authority ${SERVICE_HOSTNAME} \
  ${INGRESS_HOST}:${INGRESS_PORT} \
  inference.GRPCInferenceService.ServerReady

Expected Output

{
  "ready": true
}

You can use the example payload jay-v2-grpc.json as the sample input to test the model. Notice that the input format differs from the in the previous REST endpoint example.

ModelInfer API takes input following the ModelInferRequest schema defined in the grpc_predict_v2.proto file. 

grpcurl \
  -vv \
  -plaintext \
  -proto ${PROTO_FILE} \
  -authority ${SERVICE_HOSTNAME} \
  -d @ \
  ${INGRESS_HOST}:${INGRESS_PORT} \
  inference.GRPCInferenceService.ModelInfer \
  <<< $(cat "$INPUT_PATH")

Expected Output

Resolved method descriptor:
// The ModelInfer API performs inference using the specified model. Errors are
// indicated by the google.rpc.Status returned for the request. The OK code
// indicates success and other codes indicate failure.
rpc ModelInfer ( .inference.ModelInferRequest ) returns ( .inference.ModelInferResponse );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc
date: Tue, 10 Oct 2023 14:55:27 GMT
grpc-accept-encoding: identity, deflate, gzip
server: istio-envoy
x-envoy-upstream-service-time: 190

Estimated response size: 4093 bytes

Response contents:
{
    "modelName": "paddle-v2-resnet50-grpc",
    "id": "97db0c56-95d2-4171-afd5-f7609a87032d",
    "outputs": [
      {
        "name": "output-0",
        "datatype": "FP32",
        "shape": [
            "1",
            "1000"
        ],
        "contents": {
           "fp32Contents":[6.7366788e-9,1.1535991e-8,5.1422507e-8,6.6471706e-8,4.0944926e-8,1.3402452e-7,9.355561e-8,2.8935892e-8,6.845367e-8,7.680616e-8,0.000002033469,0.0000011085679,2.3477592e-7,6.582037e-7,0.00012373104,4.2878804e-7,0.00000641996,0.99934965,0.000073720024,0.000031011357,0.0000056028093,0.0000021862509,1.9544045e-8,3.728894e-7,4.2903633e-7,1.825118e-7,7.159926e-8,9.231618e-9,6.4692415e-7,7.0316903e-9,4.4512316e-8,1.2455972e-7,9.4463275e-8,4.3477044e-8,4.65822e-7,6.797721e-8,2.1060276e-7,2.2605123e-8,1.431149e-7,7.9512986e-8,1.2341783e-7,0.0000010921714,0.000015243892,3.1173343e-7,2.4152058e-7,6.8637625e-8,8.467682e-8,9.424677e-8,1.021921e-8,3.3770753e-8,3.6928835e-8,1.3694032e-7,1.06742846e-7,2.5994837e-7,3.4866406e-7,3.1320535e-8,3.5748732e-7,6.648439e-8,3.1638956e-7,0.0000012095878,8.66409e-8,4.0144172e-8,1.2544761e-7,3.320118e-8,1.9731445e-7,3.8064056e-7,1.3827865e-7,2.300226e-8,7.144225e-8,2.8511145e-8,2.9825674e-8,8.936033e-8,6.2238837e-7,6.478839e-8,1.3663023e-7,9.9731814e-8,2.5761555e-8,4.13022e-8,3.9384464e-8,1.215808e-7,0.0000043028217,0.0000018179063,0.0000018520155,0.0000016246107,0.000016448314,0.000010544916,0.000003993062,2.6464798e-7,0.000019193476,4.803243e-7,1.696285e-7,0.0000045505058,0.000042359294,0.0000044433386,0.0000051040097,0.0000135063965,4.1758724e-7,4.4944915e-7,3.1566984e-7,0.00000105576,1.3364639e-8,1.389366e-8,6.7703795e-8,1.4129697e-7,7.170519e-8,7.9344666e-8,2.6391543e-8,2.6134321e-8,7.196726e-9,2.1752363e-8,6.68464e-8,3.417796e-8,1.6228276e-7,4.1071146e-7,6.4721354e-7,2.9513794e-7,5.653474e-9,4.8301445e-8,8.887482e-9,3.730617e-8,1.7784265e-8,4.641905e-9,3.4131187e-8,1.9373938e-7,0.0000012980177,3.5641005e-8,2.1493324e-8,3.0552937e-7,1.5532517e-7,0.0000014520979,3.4884646e-8,0.000038254384,4.5088433e-7,4.176697e-7,6.7706225e-7,1.4142249e-7,0.000014235998,6.2938204e-7,0.000004762867,9.0249006e-7,9.058988e-7,0.0000015713684,1.5720647e-7,1.8185365e-7,7.1931886e-8,0.0000011952825,8.8748374e-7,2.0870831e-7,9.906239e-8,7.793622e-9,1.00584984e-7,4.205944e-7,1.8436246e-7,1.6437947e-7,7.025353e-8,2.5704486e-7,7.5868776e-8,7.841314e-7,2.4953093e-7,5.157682e-8,4.0674127e-8,7.5317965e-9,4.7974854e-8,1.7419973e-8,1.7958679e-7,1.2566392e-8,8.97544e-8,3.2696548e-8,1.120836e-7,3.9067462e-8,4.6769046e-8,1.8523554e-7,1.4833053e-7,1.227935e-7,0.0000010729105,3.653849e-9,1.6198403e-7,1.619072e-8,1.2004934e-7,1.48002774e-8,4.0229484e-8,2.150609e-7,1.1925697e-7,4.8982514e-8,7.60892e-8,2.313748e-8,8.52105e-8,9.5862134e-8,1.3351651e-7,3.021699e-8,4.4238764e-8,2.610667e-8,2.397709e-7,1.3192565e-7,1.6734932e-8,1.588337e-7,4.0643516e-7,8.7534545e-8,8.3669994e-7,3.4375987e-8,7.847893e-8,8.526395e-9,9.601383e-8,5.258924e-7,1.3557448e-7,1.0307227e-7,1.04298135e-8,5.1877144e-8,2.1870013e-8,1.179144e-8,2.9806564e-8,4.3383935e-8,2.9991046e-8,2.850761e-8,3.058665e-8,6.441099e-8,1.5364101e-8,1.5973884e-8,2.573685e-8,1.0903766e-7,3.2118738e-8,6.819743e-9,1.9251311e-7,5.825811e-8,1.8765762e-7,4.007079e-7,1.5791578e-8,1.9501584e-7,1.0142063e-8,2.744815e-8,1.2843532e-8,3.7297493e-8,7.407497e-8,4.2060783e-8,1.6924805e-8,1.4592035e-7,4.344977e-8,1.7191404e-7,3.5817443e-8,8.440249e-9,4.1948297e-8,2.5140324e-8,2.83402e-8,8.747196e-8,8.277126e-9,1.1676294e-8,1.4548515e-7,7.200282e-9,0.000002623601,5.675737e-7,0.0000019483527,6.752595e-8,8.168475e-8,1.09330465e-7,1.6709137e-7,3.1387277e-8,2.9735245e-8,5.752164e-8,5.8508775e-8,3.2544622e-7,3.3302214e-8,4.1867867e-7,1.5085907e-7,2.334682e-7,2.8640278e-7,2.294032e-7,1.8537604e-7,3.1517982e-7,0.0000011075967,1.5369783e-7,1.923751e-7,
            1.6404466e-7,2.9008353e-7,1.2466549e-7,5.802622e-8,5.1862205e-8,6.0094205e-9,1.2333241e-7,1.3798474e-7,1.7370232e-7,5.617761e-7,5.160447e-8,4.8132776e-8,8.0326984e-8,0.0000020645264,5.6385977e-7,8.7942e-7,0.000003478598,2.9723893e-7,3.3904533e-7,9.469074e-8,3.7548457e-8,1.5679038e-7,8.203105e-8,6.8479626e-9,1.8251624e-8,6.050241e-8,3.9563428e-8,1.0699948e-7,3.2566635e-7,3.536943e-7,7.326295e-8,4.857656e-7,7.7177134e-7,3.456778e-8,3.2462046e-7,0.0000031608602,5.3309947e-8,3.6456873e-7,5.4815894e-7,4.6230696e-8,1.3466178e-7,4.3529482e-8,1.6404105e-7,2.4636954e-8,5.9587126e-8,9.493651e-8,5.5234626e-8,5.7412358e-8,0.00001185035,0.0000058263945,0.0000074208674,9.127966e-7,0.0000020019581,0.000001033499,3.514685e-8,0.0000020589953,3.565551e-7,6.873234e-8,2.1935298e-9,5.5603635e-8,3.3266997e-7,1.3073692e-7,2.718763e-8,1.0462929e-8,7.4666804e-7,6.923166e-8,1.6145664e-8,8.568521e-9,4.76221e-9,1.2339771e-7,8.340629e-9,3.2649041e-9,5.063249e-9,4.0704995e-9,1.2043539e-8,5.1056084e-9,7.267143e-9,1.1845163e-7,7.535579e-8,6.386964e-8,1.6212937e-8,2.6104294e-7,6.9794254e-7,6.647513e-8,7.717493e-7,6.651207e-7,3.3244953e-7,3.707282e-7,3.9956424e-7,6.411632e-8,7.107352e-8,1.6380017e-7,6.876801e-8,3.4624745e-7,2.0256503e-7,6.1961015e-7,2.6841073e-8,6.7203354e-7,0.0000011348341,0.0000018397932,6.3972516e-7,7.257533e-8,4.221391e-7,3.9657925e-7,1.403744e-7,3.2498568e-7,1.5857655e-7,1.1122217e-7,7.391421e-8,3.4232224e-7,5.3979615e-8,8.5172964e-8,0.00000406101,0.0000144787555,7.3170328e-9,6.948496e-9,4.4689173e-8,9.231412e-8,5.4119823e-8,2.2242811e-7,1.7609555e-8,2.090628e-8,3.6797683e-9,6.17792e-8,1.7920289e-7,2.627918e-8,2.69882e-8,1.6432807e-7,1.2827613e-7,4.4689088e-8,6.316553e-8,1.946176e-8,2.0871258e-8,2.241458e-8,2.4765244e-8,6.7853985e-9,2.4248795e-8,4.5549795e-9,2.8977038e-8,2.0402325e-8,1.6009503e-7,2.019971e-7,1.6111885e-8,5.964114e-8,4.0983186e-9,3.9080128e-8,7.511338e-9,5.965624e-7,1.6478224e-7,1.4106989e-8,3.2855585e-8,3.3387166e-9,1.2200434e-8,4.624639e-8,6.8423094e-9,1.7426288e-8,4.661133e-8,9.331948e-8,1.2306079e-7,1.2359445e-8,1.1173199e-8,2.7724862e-8,2.4192101e-7,3.4511868e-7,2.593767e-8,9.964568e-8,9.79781e-9,1.9085564e-7,3.9727063e-8,2.6639205e-8,6.874149e-9,3.1469938e-8,2.4086594e-7,1.3126927e-7,2.125434e-7,2.0502034e-8,3.694976e-8,6.563176e-7,2.5600501e-8,2.6882981e-8,6.880636e-7,2.0092733e-7,2.7880397e-8,2.6284091e-8,5.1678345e-8,1.8935414e-7,4.6185284e-7,1.1086778e-8,1.4542604e-7,2.873701e-8,6.105168e-7,1.2016463e-8,1.3944705e-7,2.0937128e-8,4.380141e-8,1.9663208e-8,6.654449e-9,1.1149591e-8,6.424939e-8,6.971555e-9,3.2600196e-9,1.42601895e-8,2.7895078e-8,8.115783e-8,2.5995716e-8,2.2855579e-8,1.05596285e-7,8.1455426e-8,3.7793686e-8,4.8818915e-8,2.3420624e-8,1.0599355e-8,3.604105e-8,5.062431e-8,3.680444e-8,1.5015802e-9,0.0000014475033,0.0000010762104,1.3049913e-7,3.0736015e-8,1.7184021e-8,2.042109e-8,7.904992e-9,1.6902052e-7,1.2416507e-8,5.4758292e-8,2.6250422e-8,1.3261367e-8,6.2980746e-8,1.2709986e-8,2.0171682e-7,4.3866372e-8,6.9623496e-8,2.956512e-7,7.9251316e-7,2.086892e-7,1.7341794e-7,4.2942418e-8,4.213407e-9,8.824785e-8,1.734157e-8,7.321587e-8,1.7941774e-8,1.1245148e-7,4.2424054e-7,8.2595735e-9,1.1336403e-7,8.268799e-8,2.2186978e-8,1.9539721e-8,1.0675704e-8,3.2885175e-8,2.4340963e-8,6.639137e-8,5.6046874e-9,1.3866047e-8,6.675874e-8,1.1355886e-8,3.1321596e-7,3.124518e-8,1.5021818e-7,1.3461754e-8,1.8882956e-7,4.6457423e-8,4.645388e-8,7.714454e-9,3.5857155e-8,7.608321e-9,4.2215014e-8,4.340725e-9,1.3401575e-8,8.5656005e-8,1.7045414e-8,5.4221903e-8,3.0219127e-8,6.153376e-8,3.9388572e-9,
            4.135628e-8,1.7819204e-8,4.3105885e-8,3.903355e-9,7.6630855e-8,1.1890406e-8,9.304218e-9,1.0968062e-9,1.0536768e-8,1.1516804e-7,8.134523e-7,5.952624e-8,2.8063502e-8,1.2833099e-8,1.060569e-7,7.87295e-7,2.7501393e-8,3.936289e-9,2.0519442e-8,7.394816e-9,3.5983973e-8,2.5378517e-8,4.6989722e-8,7.54953e-9,6.3228055e-7,5.5820064e-9,1.2964098e-7,1.5874988e-8,3.383781e-8,6.474512e-9,9.121148e-8,1.3918512e-8,8.2300255e-9,2.706129e-8,2.6095918e-8,5.7223635e-9,6.9634757e-7,4.6850918e-8,9.59058e-9,2.0992059e-7,3.0821607e-8,3.5631626e-8,7.326313e-7,0.0000021759731,2.4075183e-7,2.9745158e-7,2.5290184e-8,7.667951e-9,2.6632893e-7,3.435888e-8,2.3130198e-8,3.1239693e-8,2.8691622e-7,3.895845e-8,2.418413e-8,1.1582445e-8,5.154535e-8,2.0343455e-8,8.201963e-8,1.1641536e-8,5.496357e-7,1.1682151e-8,4.7576915e-8,1.6349825e-8,4.0908628e-8,2.127119e-7,1.6697287e-7,3.989708e-8,0.0000028524503,1.2500372e-7,2.4846614e-7,1.2454291e-8,2.9700272e-8,4.2509916e-9,1.6144348e-7,2.638602e-7,7.638056e-9,3.4455794e-9,7.2732895e-8,1.7631434e-8,7.586613e-9,2.1547013e-8,1.2675349e-7,2.563715e-8,3.5009762e-8,6.4722435e-8,8.387915e-9,3.0695123e-8,7.520388e-8,1.5724964e-7,1.9634005e-7,1.2290832e-7,1.1121187e-9,1.546896e-8,9.91701e-9,6.882473e-7,8.2676166e-8,4.4695312e-8,2.0752013e-8,8.6493785e-8,5.2027666e-8,4.5564942e-8,2.0319955e-8,8.705182e-9,6.452067e-8,2.1777439e-8,1.0309542e-8,3.2119043e-8,2.3336936e-7,8.054096e-9,1.9623354e-7,1.288809e-7,1.5392496e-8,1.401903e-9,5.6968183e-8,6.0800254e-9,1.0782793e-8,2.426073e-8,1.938866e-8,2.297031e-7,1.9971754e-8,2.8477993e-8,5.2273553e-8,2.7392807e-7,9.857291e-8,3.1291098e-8,4.1514422e-8,5.2511964e-9,0.0000015806811,8.5476034e-7,1.0689131e-8,0.0000010621831,7.737313e-8,6.394217e-8,1.1698346e-7,1.04866096e-7,2.1161e-7,1.5339682e-8,5.0944536e-8,1.400538e-8,2.6282036e-8,8.7784336e-8,7.7720665e-9,4.2288754e-8,3.324378e-7,7.729245e-8,7.636901e-10,5.989501e-8,1.3260906e-7,1.2853634e-7,8.844243e-9,1.0194375e-7,2.4937793e-7,1.6547972e-7,1.1762754e-8,1.14961956e-7,2.934271e-7,1.3261241e-8,8.630263e-8,5.7394843e-8,1.1094081e-7,2.2933713e-7,3.470617e-7,1.4751107e-7,1.502495e-8,6.4543194e-8,5.1645337e-8,6.2374156e-8,1.2936015e-7,1.40520715e-8,5.386946e-8,2.0827555e-8,1.3040638e-8,1.05789816e-7,1.5079728e-8,8.926327e-7,4.637438e-8,7.481006e-7,5.8831473e-9,2.8707685e-9,8.3815985e-7,7.3419586e-9,1.4245998e-8,1.0926417e-7,1.1308178e-7,2.523399e-7,1.1782836e-7,4.6678057e-8,2.7959197e-9,3.436386e-8,1.4674497e-7,3.5396916e-8,2.0581128e-7,7.183879e-8,2.7693943e-8,4.5493387e-8,1.9559183e-8,1.5359708e-8,1.2336623e-8,2.9570606e-8,2.8775526e-7,9.0518455e-7,2.3732602e-7,1.6521676e-8,1.5478875e-8,3.5267863e-8,3.6164106e-8,1.6159095e-8,7.650073e-8,1.9661483e-8,4.917535e-8,1.1712613e-7,1.0889253e-8,0.0000014941202,1.0185857e-8,3.757597e-8,2.0970978e-8,3.368558e-8,4.845589e-9,6.0396246e-7,1.0373311e-8,2.8416503e-7,4.499063e-7,3.463186e-8,7.720684e-8,1.4711222e-7,1.16015755e-7,4.007488e-7,3.0256498e-8,6.7067845e-8,2.0128741e-8,1.5987744e-9,4.1919822e-8,1.31671545e-8,3.2318148e-8,9.24766e-8,1.3075301e-7,1.0574301e-7,3.7621653e-8,1.09422466e-7,7.001475e-8,2.7427062e-8,2.0766626e-8,4.5403404e-8,3.390403e-8,1.0469662e-7,2.8271579e-8,3.4062268e-7,5.146207e-7,6.7407086e-7,6.382248e-9,3.638787e-8,3.6260598e-8,1.6065603e-7,3.639056e-7,6.2326917e-9,4.80549e-8,3.3726337e-8,6.3288803e-7,6.4806315e-8,2.1165198e-7,8.3877914e-8,1.7589144e-8,2.7290277e-9,2.1447951e-8,7.8612715e-8,2.0118186e-8,2.8407685e-8,2.492253e-7,2.0156671e-8,2.655165e-8,2.7848243e-8,6.9071238e-9,1.8805437e-8,1.3006904e-8,3.6859183e-7,3.9679412e-7,2.7592133e-8,2.5228948e-8,1.5470029e-7,3.6893066e-8,1.4401772e-9,2.150493e-8,5.0681113e-8,5.0817114e-8,1.1718752e-8,5.4092784e-8,7.1382766e-7,2.5237213e-7,7.0720446e-8,7.199764e-8,1.2525473e-8,3.4803418e-7,1.9591828e-7,1.24047e-7,1.2346175e-7,1.9201337e-8,1.9895249e-7,3.7876678e-8,1.0629785e-8,1.2437128e-8,2.1861892e-7,2.6181456e-7,1.1129008e-7,1.07766304e-7,6.380325e-9,3.895085e-9,1.5762757e-7,2.909027e-9,1.0381051e-8,2.8135211e-8,1.07780025e-8,1.3605974e-8,2.9236466e-8,1.9189795e-7,2.1995064e-7,1.3263998e-8,4.9004846e-8,2.9808371e-9,8.926046e-9,1.0996976e-8,7.7156015e-9,7.454491e-9,5.0861622e-8,1.5129764e-7,1.1960076e-8,1.1323334e-8,9.391332e-9,9.585702e-8,1.9055328e-8,1.8105304e-8,6.179228e-8,6.3894014e-8,1.185318e-8,9.3727754e-9,1.2332148e-7,1.6522023e-8,1.2461165e-7,4.196172e-9,3.9965933e-8,1.25545565e-8,1.4302138e-8,6.631794e-9,5.9642247e-9,5.5569362e-9,1.4192456e-7,1.761308e-8,3.3801896e-7,7.856519e-8,2.966783e-8,0.0000028992106,0.0000013787367,5.3136224e-9,2.5128529e-8,8.406627e-8,4.492839e-8,5.408793e-8,2.4239176e-8,4.0168052e-7,4.1083545e-8,5.415348e-8,8.640767e-9,5.7732567e-8,2.6443152e-7,8.953217e-7,2.7994002e-8,5.88948e-9,4.178882e-8,2.8880645e-8,2.1357529e-8,2.3024175e-7,8.7866255e-8,2.0697297e-9,2.2364105e-8,3.2032763e-9,1.1768747e-8,6.963571e-8,2.2719322e-8,7.3603825e-9,6.9225288e-9,3.2138715e-8,1.3705778e-7,1.981505e-8,1.0578956e-8,2.704942e-8,2.9755938e-9,2.17737e-8,1.0975539e-8,1.9918724e-8,2.3882098e-8,2.1357366e-8,6.1090986e-9,1.18904975e-8,1.1459892e-8,3.7317346e-8,1.5726203e-8,3.4040234e-8,3.692158e-8,9.281765e-8,1.2323201e-7,4.2347594e-8,1.7423728e-8,5.811339e-8,3.931436e-8,2.3690461e-8,1.7928501e-8,1.4406642e-8,7.0198305e-9,6.0415225e-8,4.86748e-8,1.0685319e-8,1.0051243e-8,4.242626e-8,2.6078153e-8,5.13667e-9,1.6972995e-9,1.9131587e-8,2.1117435e-7,1.3372697e-8,2.0002481e-8,1.04542565e-7,2.8144228e-8,2.1344792e-7,2.104611e-8,1.9114454e-7,3.9576936e-8,2.9316318e-8,1.1052031e-7,4.8400768e-8,5.583606e-8,1.2130111e-7,1.7762162e-8,2.5610854e-8,1.2038653e-7,4.6748596e-9,1.5916099e-8,3.1475942e-8,6.147686e-8,2.2046418e-8,3.2577634e-7,1.1989145e-7,2.381899e-7,1.4909986e-8,5.1016883e-8,5.5142202e-8,2.2885503e-8,5.71411e-8,5.185096e-7,4.9772858e-8,1.1049896e-8,1.2640993e-7,8.1748816e-8]} 
      }
    ]
}

Response trailers received:
(empty)
Sent 1 request and received 1 response
Back to top