Bugra Akyildiz: Putting Tensorflow Models into Production | PyData Austin 2019
"Increased success of various neural networks and deep learning models for various tasks such as image classification, text classification and object detection; more and more companies want to utilize deep learning models into their product offerings and workflows.
However, building a machine learning system is a challenging task as machine learning systems tend to be complex because components of the system are distinct and have different requirements. Training and evaluation of a model require different resources than serving the model in the prediction layer.
Training and validating a model can be done offline, and most of the serving predictions can also be done offline. However, certain use cases like search where input space can be very dynamic, the model serving needs to be online. This also allows building certain product features that cannot be done through offline predictions such as personalization where you want to modify predictions or re-rank the results per customer.
Serving a model to customer requests in the production is a difficult task as the machine learning model needs to respond to customer requests under certain service level agreements(SLAs) and in a high throughput request volume.
Serving the model in production with an API should support the following requirements:
Model life cycle management Experimentation with multiple algorithms Low latency in serving predictions Easily scalable Should support multiple different TensorFlow models in different product offerings TensorFlow Serving allows to productionize various TensorFlow models is a perfect match for these use cases. It also allows software engineers to safely deploy new models and run experiments on different models while keeping the same server architecture and APIs.
In this talk, I am going to show how a machine learning model can be put into production environment through TensorFlow Serving in a scalable manner with low latency in mind."
"TensorFlow Model Serving supports:
Model life cycle management Experimentation with multiple algorithms Low latency in serving predictions Easily scalable In this tutorial, I am going to show how a machine learning model can be put into production environment through TensorFlow Serving in a scalable manner with low latency in mind."
www.pydata.org
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.
Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVi...