When we talk about Python, couple of things always comes to mind. One is that Python is very beginner friendly and two, it has a lot of available libraries. Data scientists use Python extensively. This is because of the sheer number of AI/ ML libraries available. Developers use Python whenever a quick and dirty solution/ POC has to be built.
Python always lacked a good way of managing dependencies. It does not provide a clean way of packaging and distributing applications. This gap is mostly filled by third party tools. One standard way is to maintain a requirements file. PIP or conda can read and build python environment based on that. The other option available is using pipenv. One more such tool is Poetry. I recently found this tool and it looked interesting. So, I used it to create a dummy project. I will put my experience with this tool below.
About Poetry
Poetry helps developers manage module dependencies and also package and deploy applications. I am shamelessly copying text below from their website.
Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.
Poetry works on top on venv, and it manages these environments for you. I normally prefer conda to manage all my environments, but in this case went ahead with venv.
Installation
There are multiple options for installing Poetry. We will go with the preferred option in this blog.
% python3 --version Python 3.9.2 % export POETRY_HOME=/Users/me/apps/poetry --------------- % curl -sSL https://install.python-poetry.org | python3 - Retrieving Poetry metadata # Welcome to Poetry! This will download and install the latest version of Poetry, a dependency and package manager for Python. It will add the `poetry` command to Poetry's bin directory, located at: /Users/me/apps/poetry/bin You can uninstall at any time by executing this script with the --uninstall option, and these changes will be reverted. Installing Poetry (1.1.12): Done Poetry (1.1.12) is installed now. Great! To get started you need Poetry's bin directory (/Users/me/apps/poetry/bin) in your `PATH` environment variable. Add `export PATH="/Users/me/apps/poetry/bin:$PATH"` to your shell configuration file. Alternatively, you can call Poetry explicitly with `/Users/me/apps/poetry/bin/poetry`. You can test that everything is set up by executing: `poetry --version` --------------- % export PATH="/Users/me/apps/poetry/bin:$PATH" % poetry --version Poetry version 1.1.12
The first thing I did was to setup an install location (line 3). Next I downloaded the installer and ran it through Python (line 5). Line 6 through 33 shows the output of this command.
Configuration
You can manage Poetry global settings using the config command. I wanted to have custom directories for caching and virtual environments, so made two changes to the configuration. We can do the same thing using environment variables, but I will change the configuration in global context.
% poetry config --list
cache-dir = "/Users/me/Library/Caches/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs" # /Users/me/Library/Caches/pypoetry/virtualenvs
% poetry config cache-dir /Users/me/codes/python/poetry/_cache
% poetry config virtualenvs.path /Users/me/codes/python/poetry/_venvsHere I have set the cache directory and venv path to custom directory.
Creating a project
% poetry new quote-server % cd quote-server % poetry env info Virtualenv Python: 3.9.2 Implementation: CPython Path: NA System Platform: darwin OS: posix Python: /var/local
We start by creating a project on line 1. This will create an entire project scaffolding as shown in the next screenshot. Poetry uses pyproject.toml file to manage dependencies.
% cat pyproject.toml [tool.poetry] name = "quote-server" version = "0.1.0" description = "" authors = ["Me <me@gmail.com>"] [tool.poetry.dependencies] python = "^3.9" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"
We will now initialize the python environment. Ideally we would initialize with the system python path, but we can use a specific version if needed.
You can also initialize an existing project with: % cd existing-project % poetry init Check the following page for all Poetry commands. https://python-poetry.org/docs/cli/
% poetry env use system % poetry install Creating virtualenv quote-server--h0Kp8Dy-py3.9 in /Users/me/codes/python/poetry/_venvs Updating dependencies Resolving dependencies... (2.7s) Writing lock file Package operations: 8 installs, 0 updates, 0 removals • Installing pyparsing (3.0.7) • Installing attrs (21.4.0) • Installing more-itertools (8.12.0) • Installing packaging (21.3) • Installing pluggy (0.13.1) • Installing py (1.11.0) • Installing wcwidth (0.2.5) • Installing pytest (5.4.3) Installing the current project: quote-server (0.1.0)
One thing to note here is that Poetry created a .lock file that contains all the installed dependencies. This can be checked in to GitHub. If Poetry finds the poetry.lock file, it will be used to install the exact dependencies.
For this project, I will be using the Flask framework and a small library called quote. I will use Flask to expose one endpoint and quote library to generate some random quotes. Let’s add them now.
% poetry add Flask Using version ^2.0.2 for Flask % poetry add quote Using version ^2.0.4 for quote
I have skipped a lot of output here. Flask and quote should now be available in the virtual environment. Poetry also made changes to the toml file. Let’s see what changes was done.
[tool.poetry.dependencies] python = "^3.9" Flask = "^2.0.2" quote = "^2.0.4"
Poetry added all installed file list to poetry.lock file. Also we can see a list of all dependencies as a tree as shown below.
% poetry show --tree flask 2.0.2 A simple framework for building complex web applications. ├── click >=7.1.2 │ └── colorama * ├── itsdangerous >=2.0 ├── jinja2 >=3.0 │ └── markupsafe >=2.0 └── werkzeug >=2.0 pytest 5.4.3 pytest: simple powerful testing with Python ├── atomicwrites >=1.0 ├── attrs >=17.4.0 ├── colorama * ├── more-itertools >=4.0.0 ├── packaging * │ └── pyparsing >=2.0.2,<3.0.5 || >3.0.5 ├── pluggy >=0.12,<1.0 ├── py >=1.5.0 └── wcwidth * quote 2.0.4 A wrapper for the Goodreads Quote API └── gazpacho >=1.0
The Mundane Stuff
Now that we have all scaffolding done, I will add some dummy Python code there and come back.
from flask import Flask
from quote import quote
app = Flask(__name__)
@app.route('/')
def show_version():
return 'PyPoetry Tester'
@app.route('/quote/<search>')
def give_quote(search):
qt = quote(search=search, limit=1)
return str(qt)
def start_app():
app.run()Okay, that was one of the simplest Flask code you can write. To run this I will add the following configuration to pyproject.toml.
[tool.poetry.scripts] run-main = "quote_server.application:start_app"
You can have additional run commands here if needed. The configuration management looks very much like node.js npm. To run this script, we will use the run command as given below.
% poetry run run-main * Serving Flask app 'quote_server.application' (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Now that we have the application running, we can freely generate quotes.
http://127.0.0.1:5000/quote/Saki
--------------------------------
[{'author': 'Saki', 'book': 'The Unbearable Bassington', 'quote': "I'm living so far beyond my income that we may almost be said to be living apart."}]
Packaging and Deployment
Poetry can also build the project and deploy it to a repository. By default it will deploy to PyPi. We can use the build command to build a wheel.
% poetry build Building quote-server (0.1.0) - Building sdist - Built quote-server-0.1.0.tar.gz - Building wheel - Built quote_server-0.1.0-py3-none-any.whl % poetry publish --repository my-repo
As seen above, build will create a tar for all platforms. It should serve for all simple cases. I have not investigated yet how to create distributions that are platform specific.
Uninstalling Poetry
Now there may come a time when you want to go back to conda. To deal with such situations, Poetry has also provided a way to remove it entirely.
curl -sSL https://install.python-poetry.org | python3 - --uninstall
There are quite a few other commands that I have not used here. However, the link given elsewhere in this blog will help get started.
Conclusion
From what I have seen for Poetry, it seems a nice tool for managing and viewing all dependencies. It also helps in building a Python package easily and deploy it. I will potentially use it for one of my projects. Ciao for now!