On October 5, the Python Software Foundation released Python 3.9. Previously, the Python team had been following an 18-month development and release cycle but has now shifted to an annual release cycle. The latest version of this highly popular language has brought forth several new features. Here, we enlist the most prominent ones.

Introduction of PEG-Based Parser

Python 3.9 proposes to replace the current LL(1)-based parser with a high-performing and more stable Parsing Expression Grammar (PEG) parser. With this replacement, the new parser will lift the current LL (1) grammar Python restriction. The PEG parser differs from its predecessor in a way that is not context-free. The fundamental difference is that the choice operator is ordered.

For example, in LL(1):

 rule: A | B | C

The LL(1) parser, in this case, will generate constructions that an input string will deduce which alternative must be expanded, however, a PEG parser will move to a second or a third alternative only if the preceding one fails. Further, the PEG parser does not suffer from ambiguity problems and helps to reduce the maintenance costs in areas related to the compiling pipeline such as grammar, parser, and the AST generation.

Multiprocessing and AsyncIO Improvements

There have been many improvements introduced in multiprocessing and asyncIO. With respect to asyncIO, there have been few additions and a few removals. For example, the reuse_address parameter if the asyncio.loop.create_datagram_endpoint() has been removed due to security concerns; on the other hand, new coroutines, shutdown_default_executor (), and asyncio.to_thread() are some of the additions.

Further, talking of multiprocessing library, a new method close() has been added to the multiprocessing.SimpleQueue class.

Changes to the Dictionary and String Functions

Python introduces two new operators for the dictionary — union (|) and in place union (|=). The union operator can be used to merge two dictionaries and in place union for updating a dictionary in place. The union and in place union in a dictionary is comparable to the concatenate (+) and extend (+=) in lists, respectively. Further, key conflicts will be solved by keeping the rightmost value, which matches with the current behaviour of the similar dict operations where the last seen value is the winner.

Talking about the string, two new functions have been added. The first function str.removeprefix(prefix) can be used to remove the prefix, whereas str.removesuffix(suffix) can be used to remove a suffix. With the introduction of the new functions, the API is expected to get more consistent and descriptive and less fragile.

Support for IANA Timezone

Zoneinfo module has been introduced to support the IANA time zone database which has now been added to the standard library. By default, the zoneinfo module will use the system’s time zone. However, if no system time zone is available, the library will use the tzdata as the time zone

Type Hinting for the Built-In Generic Types

In the new version of Python, the annotation programs have been made simpler by removing the parallel type hierarchy. It has now enabled support for generic syntax in all standard collections. Instead of using the typing.List or typing.Dict in the function signature, we can use a list or dict built-in collection types as generic types. This makes the code much cleaner.

Increased Speed

This has been a substantial performance improvement as compared to the previous version thanks to two new features introduced that boost efficiency without requiring changes to the existing code. 

The first improvement is the increased usage of the vectorcall protocol, which was introduced first in version 3.8. Vectorcall minimises or eliminates temporary objects created for the call, which then makes many common function calls faster. In Python 3.9, vectorcall has been extended to several built-ins such as range, tuple, set, frozenset, list, and dict. The second performance booster is the introduction of PEG parser (as discussed above) which deals with internal inconsistencies and enables faster parsing, especially for large amounts of code.