I recently found myself in a situation, where I was writing Python software so much that I needed to upgrade my development environment. Up to that point I had used:

  • vim as my editor
  • flake8 plugin as linter
  • pyCharm every now and then as debugger

I realized I needed to manage the environment better, when I was replacing my old laptop with a new one. It was tedious and manual work to take care of all the projects and their dependencies and move them to the new laptop. I duckduckgoed for a couple of days and came up with a list of issues I wanted to solve before they become bigger issues:

  • Separate git configs automatically between work and personal projects.
  • Separate the Python projects so that they can use different versions of Python instead of all using system version.
  • Manage the project dependencies so that you don’t have to write them manually to README.md.
  • Measure the test coverage and report it.
  • Format the code according to best practices.

Git config Link to heading

I came across a blog post by Alex Mitelman that helped me a lot along the way. But first, before going to Alex’s post, I had to solve the problem of multiple git configs. I realized I needed to do something, when I was spending too much time rewriting git history, since I had committed with the wrong git config.

Micah Henning has written an excellent guide about how to handle multiple git configs. When that was combined with this Stack Overflow post, it was easy to get a configuration, where I have all the work related repositories under ~/git/work/ and personal projects under ~/git/personal/.

In my main git config (~/.gitconfig), I have the following lines:

[init]
    defaultBranch = main
[user]
    useConfigOnly = true
[includeIf "gitdir:~/git/personal/"]
    path = ~/git/personal/.gitconfig
[includeIf "gitdir:~/git/work/"]
    path = ~/git/work/.gitconfig

In the personal git config (~/git/personal/.gitconfig), I have just name, email and signing key:

[user]
	name = Tha-Fox
	email = alice@example.com
	signingkey = ECCGFC062D3DA137

This means that as long as I place my repositories properly under “work” and “personal” directories, I don’t have to think about git configurations.

To get signed commits working, I needed to install pinentry to my Mac. I had to restart gpg-agent to get everything going:

gpgconf --kill gpg-agent

Python environment management Link to heading

Next step was to manage my Python projects better. There are several tools available, but I chose asdf and poetry. I use asdf to control my Python versions and virtual environments. Poetry, then again, is handling all the rest regarding the dependencies.

For the test coverage, I use pytest and coverage. This gives me to put to the repository frontpage. Configuring Gitlab to show the number is most likely harder than getting the number itself. The joy of reporting!

To show the test coverage, Gitlab needs a couple of things.

  1. CI/CD has to be enabled in repository settings.
  2. Gitlab needs the test coverage figure somewhere in the CI/CD output.
  3. You need give Gitlab the regex, with which to extract the figure from the CI/CD output. Luckily, Gitlab gives you some examples for the most common code coverage tools.

One good guide on how to achieve all of this is Patrick Kennedy’s post.

Since I’m terrible to follow conventions etc., I like to use black to format my code.

Workflow Link to heading

After all this has been set up via homebrew and Ansible, never by installing and configuring everything manually, it is time to take a look how it all looks from the bird’s-eye view.

  1. Create the directory for the project and initialize it as git repository.
  2. Install Python using asdf: asdf local python 3.9.1
  3. Install all the needed dependencies with poetry: poetry add -D black coverage flake8 pytest
  4. While/after coding, use the linting tools via poetry: poetry run black example.py and poetry run flake8 example.py
  5. Test the code with pytest and run coverage against it: poetry run coverage run -m pytest && poetry run coverage report -m
  6. Push the code to Gitlab and be proud of yourself.