.title-slide # The Story of Features Coming in Python 3.8 and Beyond .right[Andrey Vlasovskikh] .right[JetBrains] .right[#EuroPython 2019] .right[2019-07-12] --- # About me * I'm [@vlasovskikh](http://twitter.com/vlasovskikh) on Twitter * From St. Petersburg, Russia * I'm the technical lead of PyCharm * I also maintain IdeaVim, intellij-micropython, funcparserlib * I've been working on the Python type system * Contributions to PEPs 484, 561 --- # Python 3.8 * To be released on 2019-10-21 * Feature-complete beta is out $ python3.8 Python 3.8.0b2 (v3.8.0b2:21dd01dad7, Jul 4 2019, 11:45:16) [Clang 6.0 (clang-600.0.57)] on darwin ... >>> --- # Good Docs on What's New * [docs.python.org/3.8/whatsnew/3.8.html](https://docs.python.org/3.8/whatsnew/3.8.html) * Assignment expressions `x := expr` * Positional-only parameters `def f(x, y, /)` * Expression text in f-strings `f'{expr = }'` * New types `Protocol`, `Literal`, `TypedDict`, `Final` * Etc. * I'm **not** going to walk through this document! * It's a good read, take a look at it --- # What This Talk is About * The story of some Python 3.8 features * History * Pros and cons * An overview of some features beyond 3.8 * What might become accepted --- # Wait, "some features"? * The features I've chosen for the talk * I'll do my best to stay objective! * Take it easy, I'm only half-Vulcan 🖖 --- .title-slide # New Syntax --- .title-slide # PEP 572. Assignment Expressions `x := expr` --- # Example >>> if m := re.search('[abc]', 'spam'): >>> print(m.group()) a --- # Initial PEP * Syntax: `(expr as x)` * Sublocal scopes * Names were local to the containing statements * Name mangling for `locals()` --- # Simplify the PEP * `:=` aka the walrus operator * In use since 1958 * Dropped `(...)` * Regular scoping rules (almost) * Deprecate stmt-level `:=` --- # Scoping for Comprehensions * The PEP authors wanted to allow this if any((c := line).startswith('#') for line in lines): print(f"First comment: {c}") --- # Controversy * Too complex, too unreadable * Some Python core developers were against it * The PEP had to start with a section "The importance of real code" * Helpful in many typical cases .python with open('/path/to/file', 'rb') as file: while chunk := file.read(size): process(chunk) positive_data = [y for x in data if (y := f(x)) > 0] --- # PEP 13. Python Language Governance * Guido stepped down from his BDFL role * After he accepted PEP 572 Assignment Expressions * Python Steering Council * Barry Warsaw, Brett Cannon, Carol Willing, Guido van Rossum, Nick Coghlan --- # Summary * Pros * Compact syntax for many typical assignment patterns * Relatively simple semantics * Cons * Difficult to follow in more complex cases * Two ways to do it * Related EuroPython talk * The soul of the beast: Everything about Python's grammar by Pablo Salgado --- .title-slide # PEP 570. Positional-only parameters `def f(x, y, /)` --- # Example * Positional-only parameters for C-API functions >>> pow(x=5, y=3) TypeError: pow() takes no keyword arguments * A workaround def pow(*args): x, y, z = args * New syntax def pow(x, y, z=None, /): ... --- # Why `/` for positional-only? * Already in PEP 436 Argument Clinic DSL * A preprocessor for parsing Python C-API function arguments * Already in the docstrings of stdlib functions --- # Why not `*` and `**`? * In Python 3.8 we have this scheme def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2): .... * But why not `*` and `**`? * Wouldn't it be closer to `def f(*args, **kwargs)`? * It would be backward-incompatible with `*` for keyword-only --- # Why `*` for keyword-only? * Reasoning from PEP 3102 def compare(a, b, *ignore, key=None): # If ignore is not empty if ignore: raise TypeError --- # Summary * Pros * Consistent with keyword-only parameters * Backwards-compatible * Cons * Syntax is not consistent with `*args` and `**kwargs` --- .title-slide # New Types `Protocol`, `Literal`, `Final`, `TypedDict` --- # New Types * Protocol * Strucural subtyping * Literal * Specific str or int values * Final, @final * Cannot re-assign, cannot override * TypedDict * Specific types for values in dicts --- # Where Do New Types Come From? * [github.com/python/typing](https://github.com/python/typing) * Typing summits, sprints * `mypy_extensions` → `typing_extensions` → `typing` --- # Summary * Pros * Cover more real-world cases with types * Cons * Harder to learn Python types * `typing` vs `typing_extensions` * Related EuroPython talk * Static typing: beyond the basics of def foo(x: int) -> str by Vita Smid --- .title-slide # Beyond Python 3.8 --- # Typing * PEP 560 Possible `list[int]` via `__class_getitem__` * PEP 585 Type Hinting Usability Conventions * No need to import Optional, Any, etc. * PEP 563 `from __future__ import annotations` * Replace `=` for aliases/typevars/etc. with `:` type hints * Shorter syntax for Union, Optional * Some ideas: `int | str`, `int?`, etc. --- # Async/await * Structured concurrency for asyncio tasks * Trio-like nurseries for running tasks in scopes * [Notes on structured concurrency, or: Go statement considered harmful](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) by Nathaniel Smith * Related EuroPython talk * Advanced asyncio: Solving Real-world Production Problems by Lynn Root --- # Mypyc * Better single-core performance * A stricter subset of Python * More static, almost no metaprogramming * Unlikely to become a part of CPython * Similar to Cython * But with Python types instead of C types * Already used for optimizing Mypy * ~5x boost --- # PEP 544. Sub-interpreters * Better multi-core performance * Similar to multiprocessing * But with in-process limited data sharing via channels * Try a draft of Python API for sub-interpreters * `import _xxsubinterpreters` * No per-interpreter GIL yet * Related EuroPython talk * Python Performance: Past, Present and Future by Victor Stinner --- # Release Plans * Python 2.7 retirement * No more Python 2 support after 2020-01-01 * Python 3.8 * To be released on 2019-10-21 * Python 3.9 * 9-month release cycle instead of 18 months? --- # Conclusion * Summary * **Install and try Python 3.8b2 now!** * Try PyCharm 2019.2 Beta that supports 3.8 * Participate in Python development * Follow me on Twitter * [@vlasovskikh](http://twitter.com/vlasovskikh) * Q&A --- .title-slide # Extra Slides --- # typing.Protocol class FileLike(Protocol): def read(self, n: int) -> bytes: ... def close(self) -> None: ... def read_file(f: FileLike): ... class MyFile: def read(self, n: int) -> bytes: return self.fd.read(n) def close(self) -> None: self.fd.close() read_file(MyFile()) # OK --- # typing.Literal @overload def open(path: Union[str, bytes], mode: Literal[ "r", "w", "a", "x", "r+", "w+", "a+", "x+"], ) -> IO[Text]: ... @overload def open(path: Union[str, bytes], mode: Literal[ "rb", "wb", "ab", "xb", "r+b", "w+b", "a+b", "x+b"], ) -> IO[bytes]: ... @overload def open(path: Union[str, bytes], mode: str) -> IO[Any]: ... --- # typing.TypedDict class Movie(TypedDict): name: str year: int class ExtendedMovie(Movie, total=False): director: str movie: ExtendedMovie = { # E: Missing field 'name' 'year': 1982.5, # E: Type mismatch 'description': '...', # E: Unexpected field } --- # typing.Final * Declaring that a method should not be overridden * `@final` method decorator * Declaring that a class should not be subclassed * `@final` class decorator * Declaring that a variable or attribute should not be reassigned * `Final[...]` variable type --- .title-slide # bpo-36817. Expression text in f-strings `f'{expr = }'` --- # Example >>> x = 3 >>> print(f'{x*9 + 15 = }') x*9 + 15 = 42 --- # Initial idea * Use `!d` for debug * Similar to `!r`, `!s` in f-strings >>> x = 3 >>> print(f'{x*9 + 15!d}') x*9 + 15=42 --- # Appeared at PyCon US Sprints .center[
] --- # What's bpo-36817? * [bugs.python.org/issue36817](https://bugs.python.org/issue36817) * An issue at bugs.python.org * The change was too small for a separate PEP --- # Summary * Pros * Allows combining `=` with e.g. `!s` for strings * An easy way to manage spaces around `=` in the output * Cons * It's hard to discover a syntax change without a PEP --- # My Sources * PEP history * [github.com/python/peps](https://github.com/python/peps) * Mailing lists, forums, issues * python-ideas, python-dev, bugs.python.org, etc. * The "hallway track" at PyCon US 2019 --- # Quiz * What will it print in Python 3.8? $ python3.8 >>> import asyncio >>> await asyncio.sleep(1) >>> print('hi') 1. None 2. hi 3. SyntaxError --- # Answer * It will raise a `SyntaxError` $ python3.8 >>> import asyncio >>> await asyncio.sleep(1) File "
", line 1 SyntaxError: 'await' outside function * But you can run an asyncio-enabled REPL in Python 3.8 $ python3.8 -m asyncio --- # Quiz * What will it print in Python 3.7? >>> x = {print(k): print(v) for k, v in enumerate('ABC')} 1. A 0 B 1 C 2 2. 0 A 1 B 2 C 3. {None: None} 4. SyntaxError --- # Answer * In Python 3.7 it is >>> x = {print(k): print(v) for k, v in enumerate('ABC')} A 0 B 1 C 2 * In Python 3.8 it will be typical left-to-right