Deploy Neural Network with Flask, Docker and AWS Beanstalk

How to deploy a machine learning multi-class classification Keras model for predicting voice sentiment.

Part I:

build the NN model, the Flask application and include Html, CSS, js for the voice recorder.

Part II:

how to choose the right configuration for your Docker image.

Part III:

deploy on AWS Elastic Beanstalk and how to adjust the voice recorder.

All the above contains common troubleshoots as well. Some of them you’ll find on my StackOverflow .

Part I:

Why voice sentiment? Due to exponential growth in technology in the past decade, lifestyle has changed and people demand faster and easier to use services. Many companies do renew contracts and get customer agreement legally valid only through a phone call. Not to mention all the marketing, customer success and sales use. And it became quite necessary to predict what your customer sentiment is from a voice input/phone call.

Now let’s go through the voice sentiment model.

Data sets for training the model: contains different speakers, mostly actors, in a controlled environment and do convey the following sentiments, for both female and male: angry, disgust, fear, happy, neutral, sad, surprised . (14 classes — since it’s always wise to train female separate from male voice).

To extract and transform the data features(Mel frequency cepstral coefficients: MFCC ) from the audio I have used LibROSA . Building the model using Keras: Sequential with Conv1D layers with BatchNormalization , Dropout, and MaxPooling1D . Feel free to have your try on my Kaggle kernel .

Flask application: requests the data from the user and saves it as a WAV file which is processed and predicted later.

A couple of things to keep in mind here:

  • make sure the name of your application file is: and application=FLASK(__name__)
  • when it comes to Keras and Flask: you will find out that Keras is not thread-safe and you will need to use
    Why is this happening and how to solve it, find it on my StackOverflow solution (and here you got so pros and cons of using the above).

Find the Html, CSS, and js for voice recorder at my Github repo . You can modify the recording time in recapp.js, just remember that 1 second=1000 milliseconds: setTimeout(function(){console.log("Recording:time up");stopRecording();},4000);

Part II:

I have first deployed the Flask app with AWS Beanstalk and found that the server error log is not that detailed and clear about errors and so I have found Docker error log to be quite useful.

Of course, Docker comes with many other all in one tool and functions that makes it somehow easier and more compact for building an environment.

This will help you with making the right choices when comes about building Docker images and dealing with troubleshooting:

  • think from the start about the container/image size constraints and pick the right version for your python image. You will set this up in your Dockerfile: FROM python:3.6-slim (note: will avoid Alpine Linux for now due to some misfits: much fewer libraries, a different C library, etc.).
    The only thing about picking the slimmest version is that you might need to install some libs/dependencies. And it can be quite cumbersome … python dependencies as SoundFile for LibROSA, GCC, etc..
  • you might need to install system packages and for that, you will need to install sudo. Side note: don’t forget to use -y for when prompted Y/N, otherwise it aborts the install if no possible input from the keyboard.
  • for audio install libsndfile1-dev (this should solve all the problems for Linux and Ubuntu).
  • make sure you run everything under a single RUN to avoid the creation of intermediate images:
RUN apt-get update && \
      apt-get -y --no-install-recommends install sudo && \
      sudo apt-get -y --no-install-recommends install libsndfile1-dev && \
      pip install --no-cache-dir -r requirements.txt && \
      sudo rm -rf /var/lib/apt/lists/*
  • to make sure packages do not install unsolicited dependencies:
    - sudo --no-install-recommends or pip instal --no-deps #no dependencies though this can be cumbersome when dealing with Tensorflow and Keras (as Scikit-learn, SciPy, etc.)
    - pip instal --no-cache-dir can be helpful with that extra cache,
    - as well for cleaning leftovers: sudo rm -rf /var/lib/apt/lists/*
  • other lines in your Dockerfile should cover the rest of the application as:
WORKDIR /deploy/   # pick your working directory
COPY . .           # useful for copying all folders and subfoldersEXPOSE 5000        # set port 5000 for Flask app
CMD [“python”, “”]  #application to be run

Part III:

Amazon Web Services and Elastic Beanstalk seemed like a good choice.

I have followed all the instructions for creating a single container docker environment here .

If you use the AWS console then things are quite easy. If you AWS/EB CLI then make sure you set the path after installing EB CLI:

echo 'export PATH="/Users/Daniel/.ebcli-virtual-env/executables:$PATH"' >> ~/.bash_profile && source ~/.bash_profile

You create a new application and environment in beanstalk, uploading the archive/zip of your docker app folder contents. From the console you can modify the capacity, the instances you want to use, environment type (load balancing, autoscaling) and so on.

Now let’s have a look at troubleshoots:

  • now that all has been solved and deployed, you access the AWS beanstalk Http link that gets you to your app and might find that your voice application works superbly, less the part where, you can ask for access for your user’s microphone, through the web browser. And the answer is simple: make sure you run it on https .
  • you then OpenSSL to register and get the keys as well as signing the RSA yourself(for development purposes only). And add everything in a YAML or JSON file under docker’s “.ebextensions” folder. (find it on my GitHub )
  • if you are Mac owner and you run into a __MACOSX/ error just do this , to your archive, in the command line: zip -d __MACOSX/\*
  • if you get the errors as The configuration file .ebextensions/filename.config in application version… contains invalid YAML or JSON. YAML exception: Invalid Yaml: while parsing a block mapping in “<reader>”, line …etc. : make sure your RSA Keys lines are aligned and YAML compatible.
  • Conclusions of configuring AWS Beanstalk Single Instance SSL for HTTPS (for Docker env): after time spent with figuring out YAML and tabs vs space(or no space after keys footer, or or..) in my editor( Atom/Packages/Whitespace ), or even convert YAML to JSON( Atom/Packages/YAML_JSON Converter ), I have realized that the initial keys were broken and had to generate a new set!!!

All the above code and instructions can be found via my StackOverflow response.

I hope this helps (and saves you days of hard hard work ) and now your app is running smoothly and you enjoy the predictions. The model provided on my Github gives a little bit less than 50% accuracy — though this can be quite good for a multi-class classification at a first glance.

Take care and wishing you all a superb 2020,

A new exciting decade for exponential growth and innovation!!!