Tenth Week of LITG Program

It is the tenth week of the LITG program and we have reached almost the end of the project. This week is dedicated to pushing the developed API to the Heroku platform with the basic understanding gained during the last week.

I ran into a number of difficulties while pushing the API to Heroku. The tutorial that I was referring at the first place was outdated which resulted in giving me many errors when following it. After some effort and a bit of Google search I could find this official documentation of the Heroku platform available at https://devcenter.heroku.com/articles/getting-started-with-python#introduction. This tutorial gives step by step guidance to successfully deploy a Python application on the Heroku platform in a much intuitive manner. It also gives instructions to deploy an app locally so that we can test it using localhost. Nevertheless I directly deployed the app on Heroku.

According to this tutorial there is a number of steps which should be followed in order to deploy a Python application to the Heroku platform. For these steps to be successful, two more files should be added to the API namely, the requirements.txt file and the Procfile. These will be explained next.

requirements.txt File

As mentioned in the tutorial at https://devcenter.heroku.com/articles/getting-started-with-python#declare-app-dependencies the purpose of this file is to declare the app dependencies. Heroku recognizes an app as a Python app by the existence of this file in the root directory. For example the QUserAPI that I developed contains the following set of Python packages along with their versions in the requirements.txt file as dependencies.

Flask==0.10.1

Jinja2==2.8

Werkzeug==0.11.3

gunicorn==19.4.5

itsdangerous==0.24

MarkupSafe==0.23

newrelic==2.60.0.46

scrape_quora==0.1.3

wsgiref==0.1.2

lxml==3.5.0

When an app is deployed, Heroku reads this file and installs the appropriate Python dependencies using the pip install -r requirements.txt command.

Procfile

As mentioned at https://devcenter.heroku.com/articles/getting-started-with-python#define-a-procfile this file included within the root directory of the app explicitly defines the command that should be executed to start the app. For example QUserAPI contains the following command in the Procfile.

web: newrelic-admin run-program gunicorn -b 0.0.0.0:$PORT server:app

This file declares a single process type, web, and the command needed to run it. The name web declares that this process type will be attached to the HTTP routing stack of Heroku, and receive web traffic once deployed.

There are two other Python packges listed in this command namely, newrelic and gunicorn. newrelic is a package that instruments our applications for performance monitoring and advanced performance analytics with New Relic. It helps to trace performance issues of applications even while monitoring them at production environments. On the other hand gunicorn which is the shortened form of 'Green Unicorn' is a Python WSGI HTTP Server for UNIX and it is broadly compatible with various web frameworks. It basically helps in transforming Python code to run on HTTP.

The $PORT piece of the command instructs Heroku to deploy the app on whatever port that is free at the moment.

Once these files and the code for the API are ready we can move forward to follow the steps to deploy the app on Heroku.

Deploying the Application

The steps to deploy the app fall into a number of sub activities. These steps will be discussed under those sub activities.

Initial Steps

1. Create a free Heroku account

This is required since authentication is needed for heroku and git commands to work in an upcoming step.

2. Install virtualenv locally using the command pip install virtualenv on the terminal. 

(In addition to this, it is required to have a Python version installed on the system. In my case I already had Python installed.)

Set up

3. Install the Heroku Toolbelt which provides access to the Heroku Command Line Interface(CLI). The documentation at https://devcenter.heroku.com/articles/getting-started-with-python#set-up provides the facility to download the version of Toolbelt compatible with the OS used. Once it is installed we can use the heroku command from the terminal.

4. The next step is to login to heroku using heroku login command on the terminal.

The email address and the password of the created free Heroku account can be used for this login.

Prepare the app

5. First go to the project folder(root directory) using the cd command on the terminal.

Deploy the app

6. Create an app on Heroku using the command heroku create which prepares Heroku to receive the source code.

By default Heroku gives some randomly generated name to this app which also comes on the URL to access the application once deployed. I ran into a small issue at this point. At first I did not know that Heroku gives such a default name to the app. Therefore I ran this command with no arguments. Then it gave a name as floating-taiga-50750. I wanted to change the name to quser-api. Therefore I ran the command as heroku create quser-api. Then it created an app with the name quser-api.

As mentioned in the documentation, when a new app is created in this manner, a git remote called heroku is also created and associated with this local git repository.

7. Next deploy the source code using the command git push heroku master.

At this step I got the following error.

remote: Compressing source files... done.

remote: Building source:

remote:

remote:

remote: ! Push rejected, no Cedar-supported app detected

remote: HINT: This occurs when Heroku cannot detect the buildpack

remote: to use for this application automatically.

remote: See https://devcenter.heroku.com/articles/buildpacks

remote:

remote: Verifying deploy...

remote:

remote: ! Push rejected to floating-taiga-50750.

remote:

To https://git.heroku.com/floating-taiga-50750.git

! [remote rejected] master -> master (pre-receive hook declined)

error: failed to push some refs to 'https://git.heroku.com/floating-taiga-50750.git'

I noticed that the name of the app in the error is the random name(floating-taiga-50750) first generated by Heroku and not the name that I created next. After some Google search I could find the fix available at http://stackoverflow.com/questions/31330587/heroku-error-message-no-cedar-supported-app-detected. According to this article, reinitializing the .git files can fix the issue. So I followed the following commands in the given order to resolve the error.

rm -rf .git

git init

git add .

git commit -am "Reinitialize"

heroku create quser-api

I changed the name to quser-api since I need the application to have that name on Heroku. But then I got another error.

Creating ⬢ quser-api... !!!

▸ Name is already taken

To solve this problem I removed all the exiting apps under my user account on the web dashboard of Heroku. Then I ran the heroku create quser-api command again and the issue was resolved. Next I ran the git push heroku master command to deploy the project on Heroku.

8. Visit the app at the URL generated by the app name(http://quser-api.herokuapp.com/). 

We can also use the command heroku open on the terminal as a shortcut to open the website. Likewise I tested all the routes of the API at all the URLs.

The app was successfully deployed on Heroku. With this, I have completed all the work of my project about creating an API to scrape Quora user profiles. The final touch ups and my overall learning throughout the project will be summarized in the next week's blog post.