Python

Everything you have to Know about Type Hinting In Python

Overview of Gradual Type Hinting in Python, the Usable Types in Variable Annotations and the Consequences of Type Hinting.

Nadim Jendoubi

Nadim Jendoubi

— 8 min read
Everything you have to Know about Type Hinting In Python
Variable Annotation in Python

Introduction

Typing in Python works differently! Yes it is very different as right now Python supports Four Type Systems.

  • First we have the Duck Typing approach, this is the oldest and the most known approach for Python.
  • Then the Goose Typing which was introduced in Python 2.6 and with it we started using the isinstance checks.
  • The Static Typing most commonly used in Java and C# languages.
  • Finally the Static Duck Typing introduced in Python 3.8 with typing.Protocol type hints.
The Typing Map. Image from the Book Fluent Python
The Typing Map. Image from the Book Fluent Python

Since Python supported Duck Typing only before, it started with implementing the Gradual Typing System.

We will be covering the following topics today:

  • The Gradual Typing System
  • Usable Type Hints in Functions
  • Pros and Cons of Type Hints in Python

Gradual Typing in Python

A Gradual Type System is a system where:

  • Type Hint checking is Optional, i.e. Static Type Checkers like PyCharm’s Type Checker or MyPy Type Checker won’t emit warnings or errors when checking the code.
  • Errors are not caught at runtime, i.e. Type hints are used by type checkers, linters and IDEs only, in runtime values can be passed even if they are inconsistent with the intended type.
  • The Type Hints do not enhance the Performance of our apps.
  • Type Hints or what Python calls Annotations are optional.

How to Gradually Type a Function?

We will start with a simple example of a function that prints the number of occurrences of a word. This function will take 2 arguments: the number of occurrences and a word.

def number_occ(nb, word):
    if nb == 1:
       return f'1 {word}'
    occur_str = str(nb) if nb else 'no'
    return f'{occur_str} {word}s'

If we check this with a Linter or a Type Checker they will return no issues or warnings.

Let’s gradually Annotate our function:

def number_occ(nb: int, word: str) -> str:
    if nb == 1:
       return f'1 {word}'
    occur_str = str(nb) if nb else 'no'
    return f'{occur_str} {word}s'

Now imagine we want to support plural irregular nouns, like for example mouse in plural is mice, a Default Parameter “plural” might solve this.

from typing import Optional

def number_occ(nb: int, word: str, plural: Optional[str] = None) -> str:
    if nb == 1:
       return f'1 {word}'
    occur_str = str(nb) if nb else 'no'
    if not plural:
       plural = singular + 's'
    return f'{occur_str} {plural}'

print(number_occ(1, “mouse”)) 
# 1 mouse
print(number_occ(10, “mouse”, “mice”))
# 10 mice

How are Types Decided at Runtime?

Types in Python are defined by the supported operations! At runtime, the set of operations conducted with an object is what determines the type of the object itself. For example:

def sum(a, b):
    return a + b

There are multiple types that could support this “+” operation from the primitive types like int and float to the more elaborate ones such as str, list, array and any sequence really.

The Python runtime will accept any objects as the arguments of the sum function, the computation or the operation itself could raise a TypeError but that’s not the point here. Python will determine the type of the object as long as it accepts operations.

In general, in a gradual type system we have an interplay of two typing systems:

  • The Duck Typing: This is the view adopted by Python originally, objects have types but variables, when declared, are untyped, i.e. It doesn’t matter what the type of the object is as long as it supports the operations we want.
  • Nominal Typing: This is the view adopted by statically typed languages such as Java, C# and C++. Objects and Variables are Typed, and the Static Type Checker will enforce certain rules.

Here’s an example that shows both duck typing and nominal typing in action:

class Bird:
    pass

class Duck(Bird):
    def quack(self):
        print('Quack!')

def alert(birdie):
    birdie.quack()

def alert_duck(birdie: Duck) -> None:
    birdie.quack()

def alert_bird(birdie: Bird) -> None:
    birdie.quack()

daffy = Duck()

alert(daffy)
# Quack!
alert_duck(daffy)
# Quack!
alert_bird(daffy)
# Quack!

#They all quack! Python Executes all

Usable Type Hints in Functions

Every type in Python can be used as an Annotation for a variable, however, there are certain recommendations and restrictions that we need to be aware of.

The Any Type

The Any Type also known as the Dynamic Type, this is the cornerstone of any gradually typed language. When we write this:

def sum(a, b):
    return a + b

It is the equivalent of this:

def sum(a: Any, b: Any) -> Any:
    return a + b

So Type checkers will not raise any exceptions or warnings..

Built-in Python Types and Classes

Built-in types like int, float, str, and bytes may be used directly in variable annotations, also any user defined classes.

Optional and Union Types

The Optional type solves the problem of having None as a return type.

from typing import Optional

def number_occ(nb: int, word: str, plural: Optional[str] = None) -> str:
    if nb== 1:
       return f'1 {word}'
    occur_str = str(nb) if nb else 'no'
    if not plural:
       plural = singular + 's'
    return f'{occur_str} {plural}'

And writing Optional[str] is actually a shortcut for Union[str, None], which means the type of plural may be str or None.

Since Python 3.10, we can write str | bytes instead of Union[str, bytes]. It’s less typing, and there’s no need to import Optional or Union from typing.

def parse_token(token: str) -> str | float:
    try:
        return float(token)
    except ValueError:
        return token

It’s better to avoid writing functions that generate Union types, because this adds an extra strain on the user who needs to look into the type of the returned value when the program is running to figure out what to do with it.

Collections

Python collections usually contain different types of items. As an example, any mixture of different types can go in a list. In practice, this isn’t very effective: if you place objects in a collection you will likely want to do something with them later, which means they must have at least one method in common.

Generic Collections can be declared with type parameters to specify the type of the items they can contain.

def tokenize(text: str) -> list[str]:
    return text.upper().split()

Type Hints for Tuples

There are 3 possible use cases for Tuples:

  • Tuples as Records: When using a tuple as a record, we use the tuple built-in and declare the types of the fields within the brackets [] such as:
def get_lat(lat_lon: tuple[float, float]) -> float:
    return lat_lon[0]
  • Named Tuples used as Data classes.
from typing import NamedTuple

class Coordinate(NamedTuple):
    lat: float
    lon: float

def get_lat(lat_lon: Coordinate) -> float:
    return lat_lon.lat
  • Tuples as an Immutable Sequence: To annotate tuples of unspecified length that are used as immutable lists, we specify a single type, followed by a comma and … (that’s Python’s ellipsis token, made of three periods, not Unicode U+2026 — HORIZONTAL ELLIPSIS). For example, tuple[int, …] is a tuple with int items.
def get_lat(lat_lon: tuple[float, ...]) -> float:
    return lat_lon[0]

Mappings and Dictionaries

Dictionaries are annotated simply like this: 

users: dict[int,str] = {
    1: "Mia",
    2: "Neo"
}

The collections.abc Mapping is annotated similarly to a dictionary, only it allows the caller to provide an instance of dict, defaultdict, ChainMap, a UserDict subclass, or any other type that is a subtype of Mapping.

In general it’s better to use abc.Mapping or abc.MutableMapping in parameter type hints, instead of dict (or typing.Dict in legacy code) since the ABCs are the parent classes of all mapping types and they work perfectly well for runtime type checking.

Iterables

An Iterable is annotated like a sequence.

from collections.abc import Iterable

def replace(text: str, changes: Iterable[tuple[str, str]]) -> str:
    for from_, to in changes:
        text = text.replace(from_, to)
    return text

The Type Hints Sequence and Iterable are best used as a parameter type. They are too vague as return types. A function should be more precise about the concrete type it returns.

Generic Types

Generic types, also known as parameterized generics, can be written as T, where T is a type variable that’s bound to a specific type when used.

A Parameterized Generic Type is declared with typing.TypeVar.

TypeVar accepts extra positional arguments to restrict the type parameter. We can improve the signature of a function to accept specific types.

from typing import Sequence, TypeVar

# restricted TypeVar can be float or str only
T = TypeVar('T', float, str)
  
def first_item(l: Sequence[T]) -> T:   # Generic function
    return l[0]

Also TypeVar can accept another argument: bound. The bound keyword parameter sets an upper boundary for the acceptable types.

from typing import TypeVar

# Allow all subtypes of float and float itself
T = TypeVar('T', bound=float)
  
def first_item(l: Sequence[T]) -> T:   # Generic function
    return l[0]

To summarize:

• A restricted type variable will be set to one of the types named in the TypeVar declaration.

• A bound type variable will be set to the inferred type of the expression as long as the inferred type is consistent with the boundary Type.

Static Protocols

Protocols are Informal Interfaces that are an important part of object oriented programming and have been a mainstay of Python since its inception. A Protocol subclass defines an interface that a type checker can verify.

Protocols in Python can be written using typing.Protocol. However, classes that implement a protocol don’t need to inherit, register, or declare any relationship with the class that defines the protocol. It’s up to the type checker to find the available protocol types and enforce their usage.

from typing import Protocol, Any
from collections.abc import Iterable
from typing import TypeVar

# define the protocol greater then
class SupportsGreaterThan(Protocol):
    def __gt__(self, other: Any) -> bool: …

# define the generic type that is bound by the greater then protocol
GT = TypeVar('LT', bound=SupportsGreaterThan)

def top(series: Iterable[GT], length: int) -> list[GT]:
    ordered = sorted(series, reverse=True)
    return ordered[:length]

Callables

Callable objects are returned by higher-order functions or passed as arguments to other functions. For an object to be callable, it must implement the __call__ method.

A Callable type is parameterized like this:

Callable[[ParamType1, ParamType2], ReturnType]

def declared_fn(input_fn: Callable[[Any], str] = input]) -> None:

NoReturn Type

Functions that never return anything can have their return type annotated using this special type, usually, they exist to raise exceptions.

There are a lot of these functions in Python’s Standard Library, for example, sys.exit() raises SystemExit to terminate the Python process. Its signature in typeshed is:

def exit(__status: object = ...) -> NoReturn: ...

Consequences of Type Hinting

Type hinting or annotating as it is called in Python can be a time saver, and sometimes money saver! Static type checkers can find bugs early on and we can get them fixed more cheaply than if they were discovered in production.

Even if static type checkers are beneficial, static typing cannot be trusted fully. It’s not hard to find False positives, these are type errors on code that is correct and False negatives, these are type errors on code that is incorrect.

Also, if we are forced to type check everything, we lose what makes Python so expressive, but we can’t because:

  • Some handy features can’t be statically checked, like sequence unpacking config(**settings).
  • Advanced features are poorly supported by type checkers such as descriptors, meta classes, and metaprogramming.

Conclusion

Type Hinting and Annotations in Python is an interesting topic to dive into and requires a lot of effort to learn, but can eventually save us a lot of time.

I hope this has been informative! I will leave some extra resources that could help your research.

Further Reading

Read more