Automate Clean Code and Linting in Python
We will be looking at some packages to format, lint, test our code and later create a pre-commit hook to automate the process.
Before we talking about the packages we will use, let's look at the python files we will work with
We have a function called helpers.py
It has simple arithmetic functions. All the functions accept two parameters and perform an arithmetic operation on them.
We have another file called tester_helpers.py
This file simply tests the functions we defined earlier. It uses assert to do a simple equality check.
Now let's look at the packages we will be using.
Pytest 7.7k+ ⭐️
This package helps us run unit tests. One requirement to keep in mind is that your python file containing the unit tests should begin with 'test_'.
Only assert statements are supported. To install the package
pip install pytest
To run the unit tests, type the following commands
If all your tests pass, you should see a similar output
test_helpers.py .... [100%] ========= 4 passed in 0.01s ===========
If you get an error related to multiple relative imports
It is probably an issue with one of pytest's dependencies. You'll have to uninstall astroid and install it again. This ensures the altest astroid version is installed.
pip uninstall astroid pip install astroid
After this, we will have to uninstall pytest and install pytest
pip uninstall pytest pip install pytest
YAPF 12k+ ⭐️
This was developed by google and supports in-place formatting. To install the package
pip install yapf
To format your files, type the following
yapf --in-place *.py
This will format all your top level python files, if you want to include folders as well you can use the following
yapf --in-place **/*.py
However, this will also include our virtual environment folder. To ignore the venv folder, simply create a file .yapfignore and add venv to it.
Note: This command might take some time to run. Instead of '**' you could use the folder's specific names.
isort 4.1k+ ⭐️
This packages sorts your import statements to ensure they follow pep8 rules.
Imports should be grouped in the following order:
- Standard library imports.
- Related third party imports.
- Local application/library specific imports.
isort re-orders import statements to ensure the above rule is followed. To install the package
pip install isort
To run isort
It helps in getting rid of unused imports, variables and object keys.
To install the package
pip install autoflake
To run autoflake
autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py
Some other formatters
Pylint 3.5k+ ⭐️
pylint ensures your code is following pep8 rules and standards. It gives each python file a score out of 10 (It can given you a negative score as well)
To install the package
pip install pylint
To run the linter
pylint --fail-under=7 *.py
--fail-under is the lower bound, if any file has a score below the lower bound, an error will be returned.
Pre-commit Hook 🪝
What Are Git Hooks?
Git hooks are basically scripts fired before an important action occurs, e.g., before a commit is made, before code is pushed to a repo after a commit is made, etc. You can learn more about Git Hooks and the different kinds of hooks over here.
We will be focussing on a pre-commit hook.A pre-commit hook is a hook that is run before you make a commit.
First let's install the package
pip install pre-commit
Now we will generate a sample pre-commit hook YAML file, we will edit this later.
Now let's add our hook
Now before every commit, the pre-commit hook defined in our YAML file will be executed.
Now let's update our YAML file. Remove everything and only keep the following
repos: - repo: local hooks:
We will add our plugins(packages) under
hooks: in the YAML file. Below is the general syntax for the plugin
- id: (unique id of hook) name: (name to be displayed in terminal) entry: (command to excute) language: system (for our case, always system) always_run: true (if true, it will always run) pass_filenames: true (if true, hook will have access to the file name)
Let's define a sample plugin for YAPF
- id: YAPF name: YAPF 🧹 entry: zsh -c 'yapf --in-place *.py' language: system always_run: true pass_filenames: true
If you are using bash or are on windows, replace the zsh in 'entry' with bash.
All the other plugins are pretty similar, below is the entire YAML file with all the plugins
Whenever you update your YAML file, you will have to add the file to the staging area using git add . or git add .pre-commit-config.yaml
Below is a successful commit