The main task expected to be completed during the sixth week is to push the developed scrape_quora package into the Python Package Index which is also known as PyPI as a shortened form. Python Package Index is a repository of software for the Python programming language. At the moment of writing this article, there are 78557 packages in the package index. Once you have created some awesome piece of software using Python, you can simply push it to the Python package index and let people install it using pip install. You can also use PyPI's test server to test the developed package. Pushing a package to the Python package index requires a special directory structure. This was explained in detail in the blog post titled 'Fifth Week of LITG Program - Part 2'. Nevertheless I re-post the required directory structure in this article as well.
 |
| Python Package Directory Structure |
Once the package is ready, we need few other things before moving onto pushing the package into the Python Package Index.
Before pushing the package directly to the live server we need to push it to the test server and test the package using the pip install command. Therefore first of all we need to have user accounts on both these servers.
- .pypirc configuration file:
This file basically contains the information to authenticate the user with PyPI test and live servers. On a Linux machine, this configuration file should be in the home directory.
Once we are done with all these steps, all we have to do is to work through a sequence of terminal commands in order to push the package into the Python Package Index.
1. Register the package against PyPI's test server.
1. Register the package against PyPI's test server.
python setup.py register -r test
At this step I got several errors. First I got the following error.
Traceback (most recent call last):
File "setup.py", line 10, in <module>
packages = ['scrape_quora']
File "/usr/lib/python2.7/distutils/
dist.run_commands()
File "/usr/lib/python2.7/distutils/
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/
cmd_obj.run()
File "/usr/lib/python2.7/dist-
_register.run(self)
File "/usr/lib/python2.7/distutils/
self._set_config()
File "/usr/lib/python2.7/distutils/
config = self._read_pypirc()
File "/usr/lib/python2.7/distutils/
current['username'] = config.get(server, 'username')
File "/usr/lib/python2.7/
raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'pypi # the live PyPI'
I had to do some Google search and also play around with the content of the .pypirc configuration file to resolve this error. My .pypirc file looked as follows prior to the error.
[distutils] # this tells distutils what package indexes you can push to
index-servers =
pypi # the live PyPI
test # test PyPI
[test] # authentication details for test PyPI
repository = 'https://testpypi.python.org/pypi
username = <your_user_name>
password = <your_password>
[pypi] # authentication details for live PyPI
repository = https://pypi.python.org/pypi
username = <your_user_name>
password = <your_password>
I had to remove the two comments highlighted in yellow to resolve the aforementioned error. After that I got a different error.
Registering scrape_quora to 'https://testpypi.python.org/pypi
Server response (500): <urlopen error unknown url type: 'https>
For this too, I did some Google search but still could not find a workaround. But later it was found that I have inserted an unwanted single quote(highlighted in red) at the beginning of the URL of the test server. After removing this I could very easily resolve this error. The next error encountered was regarding authentication details. It was basically a 401 authentication failed error. PyPI live and test servers provide the option for the users to login with their gmail accounts. Therefore first I used my gmail address as the username and the gmail password as the password to login to the PyPI live and test servers. These details were included in the .pypirc file. Later my mentor instructed me to create user accounts on both the websites instead of using the default gmail credentials. Once this was done, I changed the credentials of the .pypirc file to these new login details. After that, the 401 error was resolved. I could successfully register the package against PyPI's test server.
2. Upload the package to PyPI's test server to test.
2. Upload the package to PyPI's test server to test.
python setup.py sdist upload -r test
This command did not give any errors. I could successfully upload the package to PyPI's test server. But after uploading there was a small issue with the format of the README file. Also I was previously asked to add more test cases to the package. So I deleted the already uploaded package, made these two modifications and tried to re-upload the package with the same version number(0.1.0). Then I had the following error.
Submitting dist/scrape_quora-0.1.0.tar.gz to https://testpypi.python.org/pypi
Upload failed (400): This filename has previously been used, you should use a different version.
The version number of the package takes the following format.
<major>.<minor>.<patch>
So I had to change the patch number once for every modification. After adding one more test case the patch number was changed to 1. Then after the change of the README file, the patch number was changed to 2. So the final version number was 0.1.2. The sequence of version changes was recorded under the CHANGES.txt file. The setup.py file was also updated accordingly. Once this was done, uploading the package to the test server went fine. The package is accessible at the test server on the URL https://testpypi.python.org/pypi/scrape_quora/0.1.2.
3. Install package from test PyPI server.
pip install -i test scrape_quora
I could successfully install the package from the test server using the above command.
After testing with PyPI's test server I went ahead to upload the package to PyPI's live server. The following three commands were used for the process.
1. Register the package against PyPI's live server.
python setup.py register -r pypi
2. Upload the package to the live server of PyPI.
python setup.py sdist upload -r pypi
3. Install the package to the machine.
pip install scrape_quora
The uploaded python package is available at the URL https://pypi.python.org/pypi/scrape_quora/0.1.2. I also created a small Python file to test some of the features of the package. Those functionalities actually went all fine.
During the next weeks I will be acquiring a basic understanding of how to develop an API using the Flask framework. I will indeed create an API that uses this package to retrieve Quora user account information as and when needed.
No comments:
Post a Comment