.title-slide # Type Hints for Python 2 + 3 .right[Andrey Vlasovskikh] .right[JetBrains] .right[#PyConRU 2016] --- .center[
] * I'm [@vlasovskikh](http://twitter.com/vlasovskikh) on Twitter * From St.Petersburg, Russia * I'm the PyCharm Community Lead at JetBrains * IDE for Python and Web development * I've been contributing to PEP 484: Type Hints * A part of Python 3.5, available for Python 2.7 --- # Have you seen this syntax? .python .keyword[async] def stream_events(queue): response = .keyword[await] aiohttp_request('GET', URL) .keyword[async] for chunk in readiter(response.content): .keyword[await] queue.put(chunk) * Async-await coroutines --- # Or this syntax? .python new_dict = {**old_dict, 1: 'one', 2: 'two'} first, second, *rest = items * Extra unpacking operations --- # Not another "use Python 3" talk * I'm not going to convince you to use Python 3 * Well, just a little bit * Just a quick reminder that Python 3 has lots to offer --- # What this talk is about * Fresh Python 2 vs 3 numbers * What is better: static or dynamic typing? ;) * How type hints can help you porting to Python 3 and to Python 2 + 3 --- .title-slide # Python 2 vs 3 numbers --- # PyCharm 2 vs 3 stats .center[
] --- # PyCon RU: 2 vs 3 survey * Which Python version(s) do you regularly use? * Python 2 only * Python 2 and 3 * Python 3 only * What keeps you from switching to Python 3? * No stimulus * Dependencies * Large/legacy code base * Management --- # Python 2 end-of-life is 2020 * 2020-04 is 3 years 9 months from now * [https://pythonclock.org](https://pythonclock.org) * Guido announced the Python 2 retirement party at PyCon US 2020 --- # Django 2.0 will drop Python 2 * Django 2.0 in 2017-12 will drop Python 2 support * 1.11 LTS until 2020-04 --- # How to switch to Python 3? * New projects * Just use Python 3! * Existing projects * Port to Python 3... --- .title-slide # Porting to Python 3 is hard --- # Broken compatibility in Python 3 * Lots of backwards-incompatible changes * The idea was to clean up the language * Controversial design decision * Text and binary data handling is the worst * `str` means completely different things * `bytes` is slightly different * `unicode` is not available on Python 3 * No implicit ASCII conversions in Python 3 .python u'foo' + b'bar' # TypeError struct.unpack(u'3B', u'foo') # TypeError --- # Porting Twisted to Python 3 * Required $60 000 of developers' time * Data from the Python Language Summit 2016 * Done 50% code-wise and 80% feature-wise * Old and rarely used protocols are not ported yet --- # Porting requires tests * You need lots of them * Good code coverage is highly desirable * But even 100% line coverage doesn't catch all errors --- # How can we reduce the cost of porting? * Incompatible changes in APIs aren't that rare * Let's look how other communities deal with them * In particular, statically typed languages * C++, Java, Scala, Rust, Go, etc. --- .title-slide # Static vs dynamic typing --- # Endless flame war * Static typing people * Types catch lots of bugs at compile-time! * Dynamic typing people * You still have to write tests, just write more of them! --- # Statically typed languages * Types are known at compile-time * As opposed to Python where types are known at run-time * Automated code analysis and refactoring tools * Safe and precise thanks to static types --- # Bug found by static analysis fun foo(x: String, c1: Boolean, c2: Boolean): String = if (c1 and c2) x.missingAttribute else x fun testFoo() { assertEquals("bar", foo("bar", false, false)) assertEquals("bar", foo("bar", true, false)) assertEquals("bar", foo("bar", false, true)) } * Kotlin: JVM language by JetBrains * Good for Android development * 100% line coverage doesn't help * You need 100% condition coverage in this case --- # Bug found by testing fun fib(x: Int): Int { val fibs = listOf(1, 1, 2, 3, 5, 8, 13) return fibs[x] } fun testFib() { assertEquals(1, fib(0)) assertEquals(1, fib(1)) assertEquals(55, fib(9)) } * Only a highly elaborate type system can catch it * Even Haskell is not enough, you need a language like Idris or Agda --- # Static typing and testing combined * They complement each other * Testing: points in the set of input data anywhere * Static typing: exclude a large subset of invalid input data * You better have both --- # Static automated refactoring * Go: `go tool fix` * Automates updates in incompatible Go versions * Kotlin: your own automated updates * Find and update all outdated API usages .java @Deprecated(replaceWith = "rename(src, dst)") fun move(dst: String, src: String) { ... } --- # Lack of types in Python code * Type inference is not enough * PyCharm is able to infer 50-60% of types * Cannot rely on code inspections and refactorings * Still useful to some extent def f(x): s = 'hello' return x..highlight[foo()], s..error[foo()] --- .title-slide # Type hints for Python --- # Type hints * Optional type system for Python def iterbytes(x: Text) -> Iterator[bytes]: ... * PEP 484 now supports Python 2.7 as well def iterbytes(x): # type: (Text) -> Iterator[bytes] ... --- # Main benefit: readability * Type hints help both tools and humans * See my 2015 talk at EuroPython and PyCon RU * [http://tinyurl.com/type-hints](http://tinyurl.com/type-hints) * Type hints are not for everyone * Mainly for medium-to-large projects --- # State of type hints * The Typeshed repository * Collection of stub files for the stdlib * Guido is not against adding type hints to new stdlib modules * Django will support type hints * JetBrains donates money to Django * Get 30% off PyCharm, all money goes to DSF * [http://jb.gg/i-support-django](http://jb.gg/i-support-django) --- # Tools use type hints * Static code analysis tools * Mypy, PyCharm * Find Python 3 errors in correct Python 2 code def removed_attribute(x: Text) -> Text: return x..error[decode()] def no_implicit_conversions(x: Text) -> Text: return x + .error[b'foo'] --- .title-slide # Python 2 + 3 --- # What is Python 2 + 3 * Python 2 + 3 is a common subset that is not defined formally * It's becoming larger with Python 3.3-3.5 if PY2: from BaseHTTPServer import HTTPServer else: from http.server import HTTPServer class C(with_metaclass(MyMeta)): def dump(self, sep=u'', coding=u'utf-8'): # type: (Text, Text) -> bytes data = sep.join(self.lines).encode(coding) return b'\x00\x00%s\xFF' % data --- # Easier to port to Python 2 + 3 * Dropbox is adding type hints to their Python 2 code * Readability counts * Module-by-module porting * First to Python 2 + 3, then to Python 3 --- # Text / binary type hints for 2 + 3 * Type hints for Python 2 + 3 partially solve text and binary data handling * `typing.Text` for text data since Python 3.5.2 * `bytes` for binary data * `typing.AnyStr` for both def hello(name): # type: (Text) -> Text return u'Привет, %s' % name hello(.error['\xFF']) --- # Problem with implicit ASCII conversions * Overstrict checking in PyCharm def hello(name): # type: (Text) -> Text return u'Привет, %s' % name hello(.false-positive['John']) # ASCII is OK for Python 2 * Promotion of `str` to `unicode` for Python 2 in Mypy .python hello('\xFF') # UnicodeDecodeError not found --- # ASCII types proposal * By people from JetBrains and Dropbox * Coding sprints at PyCon US 2016 * Type checkers should infer special types for ASCII literals * Users can use ASCII types in their functions class ASCIIText(Text): ... class ASCIIBytes(bytes, ASCIIText): ... hello('John') # OK, since it's ASCIIBytes hello('\xFF') # Error, since it's regular bytes --- .title-slide # Wrap-up --- # Conclusion * Python 3 is ready to overtake Python 2 * Static typing and testing complement each other * Type hints and Python 2 + 3 are useful for porting to Python 3 --- # Thank You! * Time for Q&A * Follow me on Twitter * I'm [@vlasovskikh](http://twitter.com/vlasovskikh)