Description
Speaker:: Jan-Benedikt Jagusch Christian Bourjau
Track: General: Production Taking trained machine learning models from inside a Jupyter notebook and deploying them into a production microservice is painful for two reasons:
- Models are not fully self-contained and need to be packaged together with their environment
- Models are optimized for batch processing but slow down for single-row predictions, which could lead to timeouts in a fast-paced online microservice.
In this talk, you will learn how to use the Open Neural Network Exchange (ONNX) framework to compile models into fully-self contained computational graphs, which can reduce single-row inference time by up to 99%, while also drastically simplifying model management.
You will be introduced to the ONNX ecosystem, such as the sklearn-onnx and onnxmltools libraries for converting models into ONNX graphs, and will learn how to write converters for custom estimators and transformers.
Recorded at the PyConDE & PyData Berlin 2022 conference, April 11-13 2022. https://2022.pycon.de More details at the conference page: https://2022.pycon.de/program/BLVRVL Twitter: https://twitter.com/pydataberlin Twitter: https://twitter.com/pyconde