Productionizing your Machine Learning model

by Gilbert Tanner on Mar 12, 2019

Productionizing your Machine Learning model

Even though pushing your Machine Learning model to production is one of the most important steps of building a Machine Learning application there aren’t many tutorials out there showing how to do so. Especially not for small machine or deep learning libraries like Uber Ludwig.

Therefore in this article, I will go over how to productionize a Ludwig model by building a Rest-API as well as a normal website using the Flask web micro-framework. The same process can be applied to almost any other machine learning framework with only a few little changes.

Both the website and RESTful-API we will build in this article are very simple and won’t scale very well. Therefore in the next article, we will take a look at how to scale our Flask website to handle multiple recurrent requests, which is also shown in this excellent article.

If you prefer a visual tutorial you can check out my video on the topic:

Deploying your Machine Learning model using Flask video

What is Flask?

Flask is a micro web framework written in Python. It is called a microframework because it doesn’t make use of particular tools or libraries.

Flask makes it easy to create a simple web application that can be deployed on a lot of different platforms and also allows for easy productionizing your machine learning models written in Python.

Building a basic Flask website.

To start with we will install Flask and build a basic hello word website using it. To install Flask you can either use pip or conda

pip install Flask
conda install -c anaconda flask

To create a basic Flask website we need to import it, create an application and a basic route. For more information check out the official Flask documentation.

from flask import Flask
app = Flask(__name__) # create a Flask app

def hello():
    return "Hello World!"
if __name__=='__main__':, debug=True)

This now can be run like any other python script and after it started you will be able to access the website on localhost port 3000.

Website Example
Figure 1: Website Example

Building a RESTful-API for our model

A RESTful API is an application program interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data.

This kind of API/service is perfect if you want to be able to use your model from a lot of different other services like GUIs or websites. It is also really good because it can be accessed by pretty much all programming language.

Flask can be used to create simple RESTful-APIs out-of-the-box but for bigger projects, I would recommend using the Flask-RESTful extension which adds support for building large RESTful-APIs by introducing OOP.

from flask import Flask, request, jsonify # loading in Flask
from ludwig import LudwigModel # loading in Ludwig
import pandas as pd # loading pandas for reading csv

# creating a Flask application
app = Flask(__name__)

# Load the model
model = LudwigModel.load('model')

# creating predict url and only allowing post requests.
@app.route('/predict', methods=['POST'])
def predict():
    # Get data from Post request
    data = request.get_json()
    # Make prediction
    df = pd.DataFrame([str(data['text'])], columns=['content'])
    # making predictions
    pred = model.predict(data_df=df)
    # returning the predictions as json
    return jsonify(pred['airline_sentiment_predictions'][0])

if __name__ == '__main__':, debug=True)

Creating a RESTful-API for using a Machine Learning model requires the following steps:

  • Importing the needed libraries: Flask, Ludwig and Pandas in our example
  • Creating the Flask app
  • Defining a route
  • Getting and transforming the data (we need to transform our data to a pandas   dataframe or python dictionary for making predictions with Ludwig)
  • Making predictions
  • Returning the predictions as json

To try out our RESTful-API we can create a simple script using the request library.

import requests # import request library

url = 'http://localhost:3000/predict'

# make post request and print response
r =,json={'text': 'very good'})

After executing this script we will get the following result:

Output: positive

Building a website for our model

Only very few changes need to be made in order to change our RESTful-API into a simple website. The main ones are that we now need to allow both get and post requests and that we will now render HTML using the render_template method instead of just returning json.

This article isn’t focused on the HTML and CSS and therefore I will not explain it but if you have any questions feel free to ask them in the comments.

<!DOCTYPE html>
<html lang="en">
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Twitter US-Airline Sentiment Analysis</title>
    <link rel="stylesheet" href="" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
    <script src="" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
    <script src="" integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1" crossorigin="anonymous"></script>
    <script src="" integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM" crossorigin="anonymous"></script>
<body style="width:60%; margin-left:20%;">
    <h1 class="text-center">US Airline Sentiment Analysis</h1>
    <form action="/" method="post">
        <input class="form-control" type="text" name="text" placeholder="Text">
        <input class="btn btn-primary" type="submit" value="Make prediction">

With our HTML ready we can now write the logic of the website.

from flask import Flask, request, render_template
from ludwig import LudwigModel
import pandas as pd

# Creating a flask app and specifying a folder for our templates (HTML files)
app = Flask(__name__, template_folder="templates")

# Load the model
model = LudwigModel.load('model')

# Creating a route and allowing for both get and post requests
@app.route('/', methods=['GET','POST'])
def home():
    # Checking if the request is a post request
    if request.method == 'POST':
        # getting data from the form
        data = request.form.get('text')
        # Making prediction
        df = pd.DataFrame([str(data)], columns=['content'])
        pred = model.predict(data_df=df)
        # Returning the html and passing it our sentiment
        return render_template('index.html', sentiment=pred['airline_sentiment_predictions'][0])
    # If we have a get request we won't return a sentiment
    return render_template('index.html', sentiment='')
if __name__ == '__main__':, debug=True)

The code above only has a few changes from the RESTful-API:

  • When creating the Flask app we will now also pass a path which specifies where we have our templates (HTML files).
  • Instead of using the /predict route, we will use the default route specified by a /.
  • In the method we check if we have a get or post request and react accordingly.
  • If we have a post request we will get the data from the form, make a prediction using our model and pass the prediction to the render_template function which renders our index.html file.
  • If we have a get request we will call the render_template function without passing it a sentiment.

The last thing we need to do is to display the passed sentiment so the user can see the result of their inputted text. This can be done using Jinja which is the template language Flask uses.

Jinja allows us to load in variables, create if-statements and for-loops as well as many other things inside our templates. It uses the {% syntax for statements and the {{ for variables.

We will add the following lines to the end of our body tag to display the sentiment.

{% if sentiment %}
  <p>The sentiment of the text is: {{sentiment}}</p>
{% endif %}

Now that this is done we can use our sentiment analysis model by typing in some text into the form and pressing the submit button. This will then display the prediction underneath the form.

Figure 2: Website


Productionizing  your  Machine Learning model is a important part of a machine learning project and Flask can be used to create both a website and a RESTful-API with only a few lines of code.

In this article, we learned how to build a simple website and RESTful-API. In the next article, we will take a look at how to scale our Flask applications to handle multiple recurrent requests.

If you liked this article consider subscribing to my Youtube Channel and following me on social media.

The code covered in this article is available as a Github Repository.

If you have any questions, recommendations or critiques, I can be reached via Twitter or the comment section.