Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.
#318 GIL, How We Will Miss You
January 10, 2023
00:39:38
33.43 MB
Downloads: 0
Watch on YouTube
About the show
Sponsored by Microsoft for Startups Founders Hub.
Connect with us
- Michael: @mkennedy@fosstodon.org
- Brian: @brianokken@fosstodon.org
- Show: @pythonbytes@fosstodon.org
Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.
Brian #1: PEP 703 - Making the GIL Optional in CPython
- Author: Sam Gross
- Sponsor: Łukasz Langa
- Draft status, but on Standards Track, targeting Python 3.12
- Suggested by: Will Shanks
- “The GIL is a major obstacle to concurrency.” Especially for scientific computing.
- PEP 703 proposes adding a
--without-gil
build configuration to CPython to let it run code without the global interpreter lock and with the necessary changes needed to make the interpreter thread-safe. - PEP includes several issues with GIL and sckikit-learn, PyTorch, Numpy, Pillow, and other numerically intensive libraries.
- Python’s GIL makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications.
- There’s also a section on how the GIL makes many types of parallelism difficult to express.
- Changes primarily in internals, and not much exposed to public Python and C APIs:
- Reference counting
- Memory management
- Container thread-safety
- Locking and atomic APIs
- Includes information on all of these challenges.
- Distribution
- C-API extension authors will need access to a
--without-gil
Python to modify their projects and supply--without-gil
versions. - Sam is proposing “To mitigate this, the author will work with Anaconda to distribute a
--without-gil
version of Python together with compatible packages from conda channels. This centralizes the challenges of building extensions, and the author believes this will enable more people to use Python without the GIL sooner than they would otherwise be able to.”
- C-API extension authors will need access to a
Michael #2: FerretDB
- Via Jon Bultmeyer
- A truly Open Source MongoDB alternative
- MongoDB abandoned its Open-Source roots, changing the license to SSPL making it unusable for many Open Source and Commercial Projects.
- The core of our solution is a stateless proxy, which converts MongoDB protocol queries to SQL, and uses PostgreSQL as a database engine.
- FerretDB will be compatible with MongoDB drivers and will strive to serve as a drop-in replacement for MongoDB 6.0+.
- First release back in Nov 2022
- I still love you MongoDB ;)
Brian #3: Four tips for structuring your research group’s Python packages
- David Aaron Nicholson
- Not PyPI packages, but, you know, directories with
__init__.py
in them. - Corrections for mistakes I see frequently
- Give your packages and modules terse, single-word names whenever possible.
- Import modules internally, instead of importing everything from modules.
- Make use of sub-packages.
- Prefer modules with very specific names containing single functions over modules with very general names like
utils
,helpers
, orsupport
that contain many functions.
Michael #4: Quibbler
- Quibbler is a toolset for building highly interactive, yet reproducible, transparent and efficient data analysis pipelines.
- One import statement and matplotlib becomes interactive.
- Check out the video on the repo page.
Extras
Brian:
- And now for something completely different: turtles talk
Michael:
- More RSS recommendations
- FreshRSS a self-hosted RSS and Atom feed aggregator.
- Feedly (for AI)
- Flym for Android
- Readwise is very interesting
- RSS for courses at Talk Python
- New article: Dev on the Road
Joke: Testing the program Joke: Every Cloud Architecture