How I built and deployed my first machine learning project

8 min readJun 21, 2019

While taking up good courses online to learn something, sure does help gain a lot of insight on the topic but as it is said by a famous writer:

Are you building the project or is the project building you?

Projects are obviously the way to go! Whining about your “limited” skill set won’t do you any good, rather stepping up and building a good project and learning on the go is what legends chose to do. And this is totally what I did… not do. After wasting a sufficient amount of time on learning new things and not putting it to work, I decided to change my attitude towards learning and get the answer to my own quote.

Step 0: Idea and Research

This is the first step of making any project. Few points to get started:

Whether you have a unique idea or not, you’ll always add your own flavor to the idea while you are at it.
Start with the implementation of a small feature.
Always search for previous related projects and research papers. This will give you a head start.

Step 1: Building a project

Machine learning has been a buzz word in today’s era. Building a good model requires a lot of research and iterations. Before getting to the design phase you have to analyze and plan for your project. Make a good schedule that you can stick to and get to work.

About my project:

A style transfer app which stylizes your photo inspired by a paper on Artistic Neural Style Transfer by Gatys, et al.

Overhead: The algorithm described in this paper takes around 1–2 hours to stylize a single image.

Solution: To overcome this overhead, Perceptual Losses for Real-Time Style Transfer and Super-Resolution and Instance Normalization were used which train a neural net for a single style.

Training model as cited in Perceptual Losses of Real-Time Style Transfer and Super-Resolution

Model overview:

The model consists of two main sections:

An Image Transform Net: The model to be trained. This model is used to predict the style image and the resulting image is fed to the loss network during the time of training to get the loss and back propagate. For training, I used COCO dataset for sample images.
A Loss Network: A VGG16 network. The mean square error between the output of the image fed directly and the image from the image transform network to the loss network at relu2_2 layer accounts for content loss. The mean square error between the gram matrix of the image from image transform network and the style image at relu1_2, relu2_2, relu3_3, relu4_3 layers accounts for style error.

After training for a single style, the model parameters for Image Transform Net are saved and used to stylize content images.

Using this model a photo of about 1500 px * 1500 px can take about 3–4 seconds to stylize on CPU.

To check out all the features and details of the project at this GitHub repo and the flask extension.

Step 2/3: Building a Flask app

I have used Pytorch for implementing my model, so integration with flask was as smooth as baby’s bottom.

Sorry about that. But speaking formally, Flask does the work like a charm. It is easy to use and you don’t have to learn a lot to build a basic website.

The code for the flask app can be viewed here. Let’s break down the main functions of the code piece by piece.

Upload function:

The upload function serves two HTTP methods (GET and POST). While the user goes to the upload page after selecting a style, a get request serves the request, so we render the template and pass the name of the style to be fetched using the scrape function.

If we get a post request, which is when the user uploads the content image, we perform the following steps:

If the file has not been uploaded: Flash an error message.
If the file has one of the allowed extensions or not.
If the file is valid, get the style strength and then feed the file to the stylize function from the stylize module to get an output image numpy array.
Uploading the Numpy array object to the S3 bucket and saving it as a jpeg file.
Keeping track of the location of the file in the sessions.
Rendering the final page template(uploaded.html).

Download Function:

While the uploaded page is rendered, a download button will redirect you to “/download/” which is handled by the following function:

We will firstly get the location of the stylized image using the session cookie “file”, then retrieve the image from S3 and use flask’s make_response function to download the file.

Step 2/3: Choosing a cloud service

While making a flask app is a step on its own, but choosing the cloud server and storage service will change the flow of your code. These two steps go hand in hand. Things to consider while choosing a cloud service:

Nature of project: Whether you want to make this a live project or not?
Computation used by your project: As we are working with a machine learning project, you’ll have to consider this crucially. Run the project and monitor the amount of RAM being used.
GPU: Whether or not you are using a GPU. We are not using any GPU for our project except for training the model, but for stylizing we don’t need any GPUs.

Available options:

As I am a student and I am thrifty, I have done a lot of research on cheap/free services.

Heroku: Only provides with 512 MB RAM on the free tier, clearly not sufficient for our model. Also, we want to expand our knowledge, we want to use something that is used in the industry.
Google Cloud: Google Cloud provides some good options, with its free first year. It can be considered for our purpose as well but I will tell why we are not going to use it for now.
Amazon Web Services: Did I tell you I am a student? And being a student you get a free AWS account with apparently $365 valid for 365 days. It also has a free first-year offer but I need it for when I am not a student anymore. So here we come AWS. We are going to use AWS EC2 and S3 service. #GoingAllEco

I selected a 4GB RAM variant (t2.medium) Ubuntu server for hosting my app.

Step 4: Setting up the EC2 instance

Making an EC2 instance and S3 bucket is a straightforward task. Let’s discuss working with the EC2 instance and S3 bucket.

Working with S3 in Flask

To integrate AWS S3 with Flask, we use boto3. Boto3 is a python library to create, manage and configure AWS services. For more reference, refer to the code and documentation.

Working with EC2 in Flask

You can do it on your own, just remember to download the key file(.pem) while creating the instance. The real deal is how to use your instance. Here’s how:

Select the instance you want to use.

2. Click on Connect button.

3. Copy the Example command from the pop-up window.

4. Open terminal in the directory which contains the key(.pem) file and run the copied command with superuser privileges. If you are using Windows follow the procedure using PuTTY.

5. Upload your flask project to GitHub as a public or private repo, based on your preferences. Make a requirements.txt file with all the dependencies listed in it along with Gunicorn as we are going to use it for deploying our website.

6. Run the following commands on the terminal while you are connected to your server:

7. Click on security groups in the description of your instance:

8. Click on the inbounds tab and click on edit.

9. Click on Add rule and select HTTP type with source set to anywhere. Hit save.

10. Run the following command on terminal:

gunicorn -b 0.0.0.0:80 -w 4 flaskfile:app -t 300 -D

-b: socket to bind to
-w: number of workers
-t: gunicorn workers are killed or restarted if remain silent for 300 seconds
-D: daemonize this process

Here “flaskfile” is the python script for the flask app, and “app” is the variable that holds the Flask object.

11. Check the website running successfully at the IPv4 Public IP of your instance.

Step 5: Debugging/Maintenance

Are you building the project or is the project building you?

This step is what builds you. You’ve been building the project and the project is building your skills. Debug and remove any errors. Do not give up on it unless you find a solution. Learning new things to overcome the situations you’ve never experienced, that’s how we evolve- Reinforcement!

Results

Live webcam stylization using style transfer cli

Find out more about my project at these two GitHub repos.

Stylize App: Flask app for style transfer
Style Transfer CLI: Command line tool for Style transfer, with added live webcam stylization

If this article helped you, leave a few claps, a star on my GitHub repository, a few kind words below, and a few thousand Rupees at my home. Just kidding. I’ll be happy to receive any appreciation or recommendations. Feel free to contribute to my project at GitHub. :)