Professional-grade mypy configuration
Type hints are an essential part of modern Python. Type hints are the enabler for clever IDEs, they document the code, and even reduce unit testing needs. Most importantly, type hints make the code more robust and maintainable which are the attributes that every serious project should aim for.
At Wolt we have witnessed the benefits of type hints, for example, in a web backend project which has 100k+ LOC and has already had 100+ Woltians contributing to it. In such a massive project the advantages of the type hints are evident. For example, the onboarding experience of the newcomers is smoother thanks to easier navigation of the codebase. Additionally, the static type checker surprisingly often catches bugs which would slip through the tests.
Mypy is the de facto static type checker for Python and it’s also the weapon of choice at Wolt. Alternatives include pyright from Microsoft, pytype from Google, and Pyre from Facebook. Mypy is great! However, its default configuration is too “loose” for serious projects. In this blog post, we’ll list a set of mypy configuration flags which should be flipped from their default value in order to gently enforce professional-grade type safety.
By default, mypy doesn’t require any type hints, i.e. it has
disallow_untyped_defs = False by default. If you want to enforce the usage of the type hints (for all function/method arguments and return values),
disallow_untyped_defs should be flipped to
disallow_untyped_defs = True is arguably the most important configuration option if you truly want to adopt typing in a Python project, whether it’s an existing or a fresh one.
You might think that it’d be too laborious to enforce type hints for all modules of an older (read: legacy) project. That’s of course true, and often not worth the initial investment. However, wouldn’t it be awesome if the mypy configuration would enforce all the new modules to have type hints? This can be achieved by
disallow_untyped_defs = Trueon the global level (see global vs per-module configuration),
disallow_untyped_defs = Falsefor the “legacy” modules.
In other words, strict by default and loose when needed. This allows gradual typing while still enforcing usage of type hints for new code. At Wolt, we have successfully employed this strategy, for example, in the aforementioned 100k+ LOC project. It’s good practice to get rid of the
disallow_untyped_defs = False option for an individual (legacy) module if the module requires some changes, e.g. a bug fix or a new feature. Also, adding types to legacy modules can be great fun during Friday afternoons. This way the type safety also gradually increases in the older parts of the codebase.
disallow_any_unimported = True is to protect developers from falsely trusting that their dependencies are in tip-top shape typing-wise. When something is imported from a dependency, it’s resolved to
Any if mypy can’t resolve the import. This can happen, for example, if the dependency is not PEP 561 (Distributing and Packaging Type Information) compliant.
Way too often developers tend to take a shortcut and use
ignore_missing_imports = True (even in the global level of the mypy configuration 😱) when they face something like
error: Skipping analyzing 'my-dependency': found module but no type hints or library stubs [import]
error: Library stubs not installed for "my-other-dependecy" (or incompatible with Python 3.9) [import] note: Hint: "python3 -m pip install types-my-other-dependency" note: (or run "mypy --install-types" to install all missing stub packages)
In the latter case the solution would be even immediately available and nicely suggested in the mypy output: just add a development dependency which contains the type stubs for the actual dependency. In fact, a number of popular projects have the stubs available. However, the former case is trickier. The developer could either
- try to find an existing stub project from the wild wild web,
- generate the stubs themselves (and hopefully consider open-sourcing them 😉) or
ignore_missing_imports = Truefor the dependency in question.
disallow_any_unimported = True is basically to protect the developers from the consequences of the
ignore_missing_imports = True case. For example, consider a project which depends on
requests and would ignore the imports in the mypy.ini file.
[mypy-requests] ignore_missing_imports = True
that would result in the following
from requests import Request def my_function(request: Request) -> None: reveal_type(request) # Revealed type is "Any"
and mypy would not give any errors. However, when
disallow_any_unimported = True is used, mypy would give
Argument 1 to "my_function" becomes "Any" due to an unfollowed import [no-any-unimported]
Getting the error would be great as it would force the developer to find a better solution for the missing import. The “lazy” solution would be to add a suitable type ignore:
from requests import Request def my_function(request: Request) -> None: # type: ignore[no-any-unimported] ...
This is already better as it nicely documents the
Any for the readers of the code. Less surprises, less bugs.
If the snippet below is ran with the default configuration of mypy, the type of
arg is implicitly interpreted as
Union[str, None]) although it’s annotated as
def foo(arg: str = None) -> None: reveal_type(arg) # Revealed type is "Union[builtins.str, None]"
In order to follow the Zen of Python’s (
python -m this) “explicit is better than implicit“, consider setting
no_implicit_optional = True. Then the above snippet would result in
error: Incompatible default for argument "arg" (default has type "None", argument has type "str") [assignment]
which would gently suggest to us to explicitly annotate the
from typing import Optional def foo(arg: Optional[str] = None) -> None: ...
This version is notably better, especially considering the future readers of the code.
With the mypy defaults, this is legit:
def bar(arg): not_very_wise = "1" + 1
By default mypy doesn’t check what’s going on in the body of a function/method in case there are no type hints in the signature (i.e. arguments or return value). This won’t be an issue if
disallow_untyped_defs = True is used. However, more than often it’s not the case, at least for some modules of the project. Thus, setting
check_untyped_defs = True is encouraged. In the above scenario it would result in
error: Unsupported operand types for + ("str" and "int") [operator]
This doesn’t give errors with the default mypy configuration:
from typing import Any def baz() -> str: return something_that_returns_any() def something_that_returns_any() -> Any: ...
You might think that the example is not realistic. If so, consider that
something_that_returns_any is a function in one of your project’s third-party dependencies or simply an untyped function.
warn_return_any = True, running mypy would result in:
error: Returning Any from function declared to return "str" [no-any-return]
which is the error that developers would probably expect to see in the first place.
Firstly, adding a type ignore should be the last resort. Type ignore is a sign of developer laziness in the majority of cases. However, in some cases, it’s justified to use them.
It’s good practice to ignore only the specific type of an error instead of silencing mypy completely for a line of code. In other words, prefer
# type: ignore[<error-code>] over
# type: ignore. This both makes sure that mypy only ignores what the developer thinks it’s ignoring and also improves the documentation of the ignore. Setting the configuration option
show_error_codes = True will show the error codes in the output of the mypy runs.
When the code evolves, it’s not uncommon that a type ignore becomes useless, i.e. mypy would also be happy without the ignore. Well-maintained packages tend to include type hints sooner or later, for example, Flask added them in 2.0.0). It’s good practice to cleanup stale ignores. Using the
warn_unused_ignores = True configuration option ensures that there are only ignores which are effective.
The kind of global mypy configuration that best suits depends on the project. As a rule of thumb, make the configuration strict by default and loosen it up per-module if needed. This ensures that at least all the new modules follow best practices.
As a final tip: never ever “silence” mypy by using
ignore_errors = True. If you need to silence it for one reason or another, prefer
type: ignore[<error-code>]s or a set of targeted configuration flags instead of silencing everything for the module in question. Otherwise the modules which depend on the erroneous module are also affected and, in the worst scenario, the developers don’t have a clue about it as the time bomb is well-hidden deep inside a configuration file.
Want to join Jerry and build great things with Python at Wolt? 👋 We are looking for multiple Senior Python Engineers!