Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.
#351 A Python Empire (or MPIRE?)
Topics covered in this episode: mpire mopup - the macOS Python.org Updater Immortal Objects for Python Common Docstring Formats in Python Extras Joke Watch on YouTube About the show Sponsored by Sentry: pythonbytes.fm/sentry Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: mpire A Python package for easy multiprocessing, but faster than multiprocessing MPIRE is faster in most scenarios, packs more features, and is generally more user-friendly than the default multiprocessing package. Tons of features. Nice intro article with benchmarks. Brian #2: mopup - the macOS Python.org Updater Glyph Lefkowitz On a mac, install Python with the standard Python.org installer. Then, periodically, update with python3 -m mopup I just did it and went from Python 3.11.4 to 3.11.5 See also Get Your Mac Python From Python.org for reasons to use python.org over other ways, also from Glyph. Michael #3: Immortal Objects for Python Instagram has introduced Immortal Objects – PEP-683 – to Python. Brian #4: Common Docstring Formats in Python Scott Robinson I don’t mean to disrespect Scott, but I’m honestly curious if this is really common. I like docstrings for the “why” of a function. And prefer type hints for types. Let me know what you use, at @brianokken@fosstodon.org Extras Brian: In search for a working retro Lunar Lander in Python Michael: Releases follow up North Korean hackers behind malicious VMConnect PyPI campaign Joke: It’s Bingo Time!
#350 You've Got The Stamina For This Episode
Topics covered in this episode: Make Each Line Count, Keeping Things Simple in Python Parsel A Comprehensive Guide to Python Logging with Structlog Stamina Extras Joke Watch on YouTube About the show Sponsored by Sentry: pythonbytes.fm/sentry Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Make Each Line Count, Keeping Things Simple in Python Bob Belderbos Some great tips to help you simplify your Python code to make it more understandable and maintainable. Michael #2: Parsel Parsel is a BSD-licensed Python library to extract data from HTML, JSON, and XML documents. Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors. It supports: CSS and XPath expressions for HTML and XML documents JMESPath expressions for JSON documents Regular expressions # Want a RSS feed detail from a website standard HTML? selector = parsel.Selector(text=html_text) for link in selector.css('head > link'): rel = link.xpath('.//@rel').get() rel_type = link.xpath('.//@type').get() href = link.xpath('.//@href').get() Brian #3: A Comprehensive Guide to Python Logging with Structlog Stanley Ulili structlog is an awesome logging tool, and already has great documentation. However, this article is a great starting point, highlighting: how easy it is to get started using structlog configuring the default log level changing the formatting customizing the time stamp adding custom fields adding contextual data filtering async … Michael #4: Stamina via Matthias Bach, by Hynek Production-grade Retries Made Easy stamina is an opinionated wrapper around the great-but-unopinionated Tenacity package. Its goal is to be as ergonomic as possible, while doing the right thing by default, while minimizing potential for misuse. General additions on top of Tenacity Retry only on certain exceptions. Exponential backoff with jitter between retries. Limit the number of retries and total time. Automatic async support. Preserve type hints of the decorated callable. Count (Prometheus) and log (structlog) retries with basic metadata, if they’re installed. Easy global deactivation for testing. Extras Brian: The “pytest fixtures” chapter of the pytest course is available now. Also, the PYTHONBYTES 20% discount still active for bundle through the end of August. Michael: Python 3.12.0 release candidate 1 released PyCon UK: The conference takes place from the 22nd to the 25th of September in Cardiff, Wales. The schedule is available at 2023.pyconuk.org/schedule/ and tickets are available at 2023.pyconuk.org/tickets/. PyData Eindhoven 2023, Nov 30 CFP open PyData Seattle Language Creators Charity Fundraiser: Adele Goldberg - Smalltalk, Guido Van Rossum, Anders Hejlsberg, C#, and James Gosling - Java. September 19, 2023: 12:00 - 4:00 PM, in person only. Joke: Librarian chatgpt-failures
#349 Djangonauts: Ready for Takeoff!
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Python People Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Omnivore app Omnivore is the free, open source, read-it-later app for serious readers. Distraction free. Privacy focused. Open source. Designed for knowledge workers and lifelong learners. Save articles, newsletters, and documents and read them later — focused and distraction free. Add notes and highlights. Organize your reading list the way you want and sync it across all your devices. Syncs with popular Personal Knowledge Management systems including Logseq and Obsidian Wait, what’s Logseq? :) A privacy-first, open-source platform for knowledge management and collaboration. Kinda like Notion? Brian #2: Djangonaut.space “Where contributors launch” This is a group mentoring program where individuals will work self-paced in a semi-structured learning environment over the course of three months. Djangonauts are members of the community who wish to level up their current Django code contributions and potentially take on leadership roles in Django in the future. Michael #3: Server-side hot reload Thanks to Alex Riviere for some improvements Bill Mill suggests websockets and Adam Johnson points he built something like this for Django (sorta) with django-browser-reload To make it work just: Include this script in your web projects for dev-time auto reloading of web browser when any change is detected in content. Works across all web technologies, built out on a FastAPI / Tailwind project. General workflow looks like: Edit the source CSS file Tailwind watcher generates a built CSS file Built CSS file is included the Python web HTML template Template appends a hash ID for the state of the CSS file Changes to the source CSS thus trigger a change in the final ID New ID means the page contents change and the script does a reload Even works for static resources if you put a “version” indicator on them: [HTML_REMOVED] [HTML_REMOVED] Brian #4: Python in Excel Anaconda working with Microsoft to have Python built in to Excel. “Python in Excel is currently in preview and is subject to change based on feedback. To use this feature, join the Microsoft 365 Insider Program and choose the Beta Channel Insider level.” from Microsoft Support article: Getting started with Python in Excel Extras Brian: Working on videos for “Ch3 : pytest Fixtures” for the Python Testing with pytest Course Bundle Adding some drawings and some more bonus videos. Thanks to everyone who’s signed up already. I’ve pushed the 20% discount out till the end of August. I also finally listed it on pythontest.com/courses Also lots of new interviews for pythonpeople.fm, and I’m expecting at least one new episode of testandcode.com this week. It’s going to be a busy week. Michael: PyCon Sweden CFP is open Be on Talk Python around Mobile Apps? Joke: The Password Game KennyLog-in.com - secure password generator
#348 JavaScript in Your Python
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Python People Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Differentiating between writing down dependencies to use packages and for packages themselves Brett Cannon Why can’t we just use pyproject.toml and stop using requirements.txt? Nope. At least not yet. They’re currently for different things. pyproject.toml There’s project.dependencies and project.optional-dependencies.tests that kinda would work for listing dependencies for an app. But you can’t say pip install -r pyproject.toml. It doesn’t work. And that’s weird. project is intended for packaged projects. requirements.txt for applications and other non-packaged projects It has specific versions works great with pip What then? Either we stick with requirements.txt Or we invent some other file, maybe requirements.toml? Or maybe (Brian’s comment), add something like [application] and application.dependencies and application.optional-dependencies.tests to pyproject.toml Michael #2: PythonMonkey PythonMonkey is a Mozilla SpiderMonkey JavaScript engine embedded into the Python VM, using the Python engine to provide the JS host environment. This product is in an early stage, approximately 80% to MVP as of July 2023. It is under active development by Distributive. External contributions and feedback are welcome and encouraged. It will enable JavaScript libraries to be used seamlessly in Python code and vice versa — without any significant performance penalties. Call Python packages like NumPy from within a JavaScript library, or use NPM packages like [crypto-js](https://www.npmjs.com/package/crypto-js) directly from Python. Executing WebAssembly modules in Python becomes trivial using the WebAssembly API and engine from SpiderMonkey. More details in Will Pringle’s article. Brian #3: Quirks of Python package versioning Seth Larson Yes, we have SemVer, 1.2.3, and CalVer, 2023.6.1, and suffixes for pre-release, 1.2.3pre1. But it gets way more fun than that, if you get creative Here’s a few v is an optional prefix, like v.1.0 You can include an “Epoch” and separate it from the version with a !, like 20!1.2.3 Local versions with alphanumerics, periods, dashes, underscores, like 1.0.0+ubuntu-1. PyPI rejects those. That’s probably good. Long versions. There’s no max length for a version number. How about 1.2.3.4000000000000000001? Pre, post, dev aren’t mutually exclusive: 1.0.0-pre0-post0-dev0 More craziness in article - Michael #4: bear-type Beartype is an open-source PEP-compliant near-real-time pure-Python runtime type-checker emphasizing efficiency, usability, and thrilling puns. Annotate @beartype-decorated classes and callables with type hints. Call those callables with valid parameters: Transparent Call those callables with invalid parameters: Boom Traceback: raise exception_cls( beartype.roar.BeartypeCallHintParamViolation: @beartyped quote_wiggum() parameter lines=[b'Oh, my God! A horrible plane crash!', b'Hey, everybody! Get a load of thi...'] violates type hint list[str], as list item 0 value b'Oh, my God! A horrible plane crash!' not str. Extras Brian: Python Testing with Pytest Course Bundle: Limited Pre-Release Beta Use code PYTHONBYTES now through Aug 17 for 20% discount What’s a pre-release beta? There’s a video. Check out the link. Error-tolerant pytest discovery in VSCode Finally! But you gotta turn it on. Also, I gotta talk to them about the proper non-capitalization of pytest. We’re at RC1 for Python 3.12.0 Hard to believe it’s that time of year again Michael: PyPI hires a Safety & Security Engineer, welcome Mike Fiedler PackagingCon October 26-28 Cloud Builders: Python Conf (born in Ukraine): September 6, 2023 | online Joke: Learning JavaScript
#347 The One About Context Mangers
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Python People Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: async-timeout An asyncio-compatible timeout context manager. The context manager is useful in cases when you want to apply timeout logic around block of code or in cases when asyncio.wait_for() is not suitable. Not finished yet timeout can be rescheduled by shift_by() or shift_to() methods Brian #2: PyPI Project URLs Cheatsheet Daniel Roy Greenfield There’s some cool icons available under “Project Links” on pypi.org project pages. How do you get those? And which ones are available. Daniel has found out where to look, and built us a cheat sheet. Nice. Michael #3: httpx-sse Consume Server-Sent Event (SSE) messages with HTTPX. SSE are super lightweight, server → client only subscriptions. Like websockets but less overhead (especially for iot and mobile devices) httpx-sse provides the connect_sse and aconnect_sse helpers for connecting to an SSE endpoint. The resulting EventSource object exposes the .iter_sse() and .aiter_sse() methods to iterate over the server-sent events. Brian #4: Creating a context manager in Python Trey Hunner Context managers are those things you use in a with block. There’s a bunch of cool built in ones. Building your own is a handy skill to have to clean up your code, and they’re pretty easy, with Trey’s tutorial. Shown is a great example of temporarily modifying an environmental variable. Then he gets into what you need to know about as, __enter__, and __exit__. Extras Brian: I think I’ll nix the intro music to Python People. I didn’t know what music to use, so I re-used the music from Test & Code. And I got some very honest feedback that it just doesn’t fit and was better without it. So I’ll rip it out soon. BTW, next episode to be released is with Bob Belderbos from PyBites. Should be later this week. Michael: Facebook and Instagram start blocking news in Canada Joke: day 1 and I hate it
#346 Have you lost your GIL?
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Python People Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: A Steering Council notice about PEP 703 (Making the Global Interpreter Lock Optional in CPython) Thomas Wouters Suggested by John Hagen “We intend to accept PEP 703, although we’re still working on the acceptance details.” Moving forward in 3 stages short-term, no-GIL experimental build in 3.13 or 3.14 mid-term, declare support for no-GIL version long-term, no-GIL becomes default and remove any vestiges of the GIL No commitment and timeframe is nebuous long-term means 5+ years Need community support “We want to be able to change our mind if it turns out, any time before we make no-GIL the default, that it’s just going to be too disruptive for too little gain.” Michael #2: Google's post-cookie world could turn into DRM for the internet A new authentication system could let websites block extensions or jailbroken devices. Google has been trying to implement plans to move beyond cookies for years without denying its partners the means to sell targeted ads. One recent proposal to guarantee user privacy and security could come at the cost of freedom of functionality. Comments are somewhat interesting. More info in a second article. Vivaldi has a response here. Brave won’t ship with it. Brian #3: How ruff changed my Python programming habits Matthias Kestenholz “…there’s always a trade off between development speed (waiting on git commit is very boring) and strictness. “ “ruff is so fast that enabling additional rules is practically free in terms of speed...” ruff has way more rules since last I checked. They are just mostly turned off by default. The article suggests a bunch to try turning on. See also ruff config settings turn on flake8-bugbear while leaving on defaults with select = ["E", "F", "B"] lots of rules to choose from ruff-pre-commit to run these with pre-commit Michael #4: pathlib api extended to use fsspec backends via Justin Flannery Expanding on the capabilities of fsspec, the same GitHub organization also supports another powerful library called universal_pathlib. universal_pathlib is a python library that aims to extend Python's built-in pathlib.Path api to use a variety of backend filesystems using fsspec. This seamless replacement allows developers to leverage the familiar and powerful pathlib API on any type of filesystem. upath.Path is a drop-in replacement for pathlib.Path and is an excellent addition to your toolkit. Joke: Understanding pointers
#345 Some Big Time Releases
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Python People Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Cython 3.0 Long in development, the new major release of the Python-to-C compiler sheds legacy Python support and readies Cython developers for big changes in Python. Cython 3 cleans up and modernizes Cython. Pure Python mode allows Python developers to use their existing Python linting and code analysis tools on Cython. Brian #2: Reading code : An important but seldom-discussed skill Eric Matthes A cool walk through of several techniques to read code Strategies Ignore function definitions And in the example, also ignore comments Simplify repetitive blocks Examples shows mentally lumping a bunch of print statements into “print message” Utilize IDE tools, like folding to hide functions your not looking at Also includes a note about writing readable code. Notes: People believe your function and variable names, they should be descriptive, and they should not be deceptive. Michael #3: Major new version of MicroPython: v1.20.0 via Matt Trentini >10 months, >1000 mainline commits from >100 contributors This release of MicroPython introduces a new lightweight package manager called mip. In the MicroPython runtime, core/built-in types have been compressed by only including in the C-level type struct as many slots for C function pointers as is needed for a given type → Any third-party C extensions will need to be updated to work with this change. Massive list of detailed changes. Brian #4: Advanced Python Tips for Development Scofield Idehen There’s 15 in the article, here’s a few 1 & 2. Use List Comprehensions and Generator Expressions. It’s cool to see them side by side enumerate() is fun Embrace zip(). It’s weird, but very useful. Utilize slots to Reduce Memory Usage Extras Brian: Hear the story behind the quote “I came for the language, but I stayed for the community.” and learn about fountain pens, tea, and a Murderbot, on this week‘s Python People. Michael: Search (LLM like) Talk Python: explore-talk-python-to-me.streamlit.app by Aguss Joke: You’re full stack now Seriously, take the HTMX course :)
#344 AMA: Ask Us Anything
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Main topic: AMA questions from the audience. Use the transcript search to find timestamps if you want to locate a particular one. Extras Michael: Deputy CPython Developer in Residence position accepting applications. My Make Your Python Web App Fly Around the World with CDNs talk at PWC 2023 is online. “Joke”: Ode to Python recommended by FelixTheCat
#343 So Much Pydantic!
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Pydantic v2 released Pydantic V2 is compatible with Python 3.7 and above. There is a migration guide. Check out the bump-pydantic tool to auto upgrade your classes Brian #2: Two Ways to Turbo-Charge tox Hynek Not just tox run-parallel or tox -p or tox --``parallel , but you should know about that also. The 2 ways Build one wheel instead of N sdists Run pytest in parallel tox builds source distributions, sdists, for each environment before running tests. that’s not really what we want, especially if we have a test matrix. It’d be better to build a wheel once, and use that for all the environments. Add this to your tox.ini and now we get one wheel build [testenv] package = wheel wheel_build_env = .pkg It will save time. And a lot if you have a lengthy build. Run pytest in parallel, instead of tox in parallel, with pytest -n auto Requires the pytest-xdist plugin. Can slow down tests if your tests are pretty fast anyway. If you’re using hypothesis, you probably want to try this. There are some gotchas and workarounds (like getting coverage to work) in the article. Michael #3: Awesome Pydantic A curated list of awesome things related to Pydantic! 🌪️ Notable items for me: ML: spaCy 🌟(26575) - spaCy is a free open-source library for Natural Language Processing in Python. It features NER, POS tagging, dependency parsing, word vectors and more. ray 🌟(26496) - Ray provides a simple, universal API for building distributed applications. jina 🌟(18734) - Jina is geared towards building search systems for any kind of data, including text, images, audio, video and many more. With the modular design & multi-layer abstraction, you can leverage the efficient patterns to build the system by parts, or chaining them into a Flow for an end-to-end experience. Data Beanie 🌟(1287) - Beanie - is an Asynchronous Python object-document mapper (ODM) for MongoDB, based on Motor and Pydantic. Utilities datamodel-code-generator 🌟(1694) - Pydantic model generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources. Goodconf 🌟(99) - A thin wrapper over Pydantic's settings management. Allows you to define configuration variables and load them from environment or JSON/YAML file. Also generates initial configuration files and documentation for your defined configuration. Brian #4: CLI tools hidden in the Python standard library Simon Willison (and hat tip to Seth Larson) Simon looked for all of the command line goodies in the standard library. I knew about python -m http.server to run a server at port 8000 from the local directory, but there’s so much more. Here are a few python -m gzip --decompress pypi.db.gz as a gzip utility. Especially handy on Windows as it doesn’t come with gzip by default python -m base64 with -d decode, -e encode, and -t encode and decode python -m asyncio for an asyncio REPL Tokenize a Python file with python -m tokenize somefile.py View the AST with python -m ast somefile.py Pretty print JSON with python -m json.tool Extras Brian: Congrats to Seth Larson, PSFs first Security Developer-in-Residence Announcing Our New Security Developer in Residence! - PSF announcement I am the first PSF Security Developer-in-Residence - Seth’s announcement PythonPeople.fm is live "The NEW podcast about the people who make the Python community awesome.” I’m focusing more on the people, and less on the tech. First episode is with Michael Kennedy Upcoming episodes in the works with Paul Everitt, Paul McGuire, and Steve Holden. More people scheduled, many asked, and many more to be asked. Michael: MongoDB with Async Python course is out! (talkpython.fm/async-mongodb) Meta commits to dedicate three engineer-years to implement the removal of the GIL from Python PyPI has a blog Joke: Containers, that’ll fix it Bonus dad joke: 5 ants rent an apartment. Invite 5 other ants to share the rent. Now there are tenants.
#342 Don't Believe Those Old Blogging Myths
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Plumbum: Shell Combinators and More Suggested by Henry Schreiner last week. (Also, thanks Michael for the awesome search tool on PythonBytes.fm that includes transcripts, so I can find stuff discussed and not just stuff listed in the show notes.) Plumbum is “ a small yet feature-rich library for shell script-like programs in Python. The motto of the library is “Never write shell scripts again”, and thus it attempts to mimic the shell syntax (shell combinators) where it makes sense, while keeping it all Pythonic and cross-platform.” Supports local commands piping redirection working directory changes in a with block. So cool. lots more fun features Michael #2: Our plan for Python 3.13 The big difference is that we have now finished the foundational work that we need: Low impact monitoring (PEP 669) is implemented. The bytecode compiler is a much better state. The interpreter generator is working. Experiments on the register machine are complete. We have a viable approach to create a low-overhead maintainable machine code generator, based on copy-and-patch. We plan three parallelizable pieces of work for 3.13: The tier 2 optimizer Enabling subinterpreters from Python code (PEP 554). Memory management Details on superblocks Brian #3: Some blogging myths Julia Evans <from Brian: I’m not sure if I’m including this to convince all of you to blog more, or to convince myself. Hopefully both happens.> myths (more info of each in the blog post): you need to be original you need to be an expert posts need to be 100% correct writing boring posts is bad you need to explain every concept page views matter more material is always better everyone should blog I’d add Write posts to help yourself remember something. Write posts to help future prospective employers know what topics you care about. You know when you find a post that is outdated and now wrong, and the code doesn’t work, but the topic is interesting to you. Go ahead and try to write a better post with code that works. Michael #4: Jupyter AI A generative AI extension for JupyterLab An %%ai magic that turns the Jupyter notebook into a reproducible generative AI playground. This works anywhere the IPython kernel runs (JupyterLab, Jupyter Notebook, Google Colab, VSCode, etc.). A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant. Support for a wide range of generative model providers and models (AI21, Anthropic, Cohere, Hugging Face, OpenAI, SageMaker, etc.). Official project from Jupyter Provides code insights Debug failing code Provides a general interface for interaction and experimentation with currently available LLMs Lets you collaborate with peers and an Al in JupyterLab Lets you ask questions about local files Video presentation: David Qiu - Jupyter AI — Bringing Generative AI to Jupyter | PyData Seattle 2023 Extras Brian: Textual has some fun releases recently Textualize youtube channel with 3 tutorials so far trogon to turn Click based command line apps into TUIs video example of it working with sqlite-utils. Python in VSCode June Release includes revamped test discovery and execution. You have to turn it on though, as the changes are experimental: "python.experiments.optInto": [ "pythonTestAdapter", ] I just turned it on, so I haven’t formed an opinion yet. Michael: Michael’s take on the MacBook Air 15” (black one) Joke: Phishing
#341 Shhh - For Secrets and Shells
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Pydantic roadmap via Mario Munoz Back in February [Samuel] [announced](https://pydantic.dev/announcement/) Pydantic Inc., but I didn't explain what services we were building. The problem is that even with Pydantic in your corner, working with data when it leaves Python often still sucks. We want to build a data platform to make working with data quick, easy, and enjoyable — where developer experience is our north star. There are five key components to the Pydantic Data Platform that we're thinking of building. Python Analytics/Observability — a logging and metrics platform with tight Python and Pydantic integration, designed to make the data flowing through your application more readily usable for both engineering and business analytics. More info... Data Gateway for object stores — Add validation, transformation and cataloguing in front of object stores like S3, with a schema defined in Pydantic models then validated by our Rust service. More info... Data Gateway for data warehouses — the same service as above, but integrated with your existing data warehouse. More info... Schema Catalog — for many, Pydantic already holds the highest fidelity representation of their data schemas. Our Schema Catalog will take this to the next level, serving as an organization-wide single source of truth for those schemas, tracking their changes, and integrating with our other tools and your wider platform. More info... Dashboards and UI powered by Pydantic models — a managed platform to deploy and control dashboards, auxiliary apps and internal tools where everything from UI components (like forms and tables) to database schema would be defined in Python using Pydantic models. More info... Tell them what you think with their survey Brian #2: The Right Way to Run Shell Commands From Python Martin Heinz Should have a tagline of “especially if you’re on Mac or Linux”. Includes discussion of Python native tools - recommended a few os module functions - but otherwise avoid relying too much on os. subprocess.run() - stick with run() if you can. sh third party package - should be second choice if on Linux or Mac Michael #3: US: Yep, We're Buying Your Data, Including Your Embarrassing Secrets Digital information can be purchased from commercial data brokers and 'deanonymized' to ID the person it's tied to, including US citizens, the Office of the Director of National Intelligence says. The Office of the Director of National Intelligence (ODNI) on Friday declassified(Opens in a new window) a report from January 2022 that outlines the US government’s approach to using Commercially Available Information (CAI), which can come from data brokers working in the internet ad and analytics industries. The purchased information includes details from users' smartphones and social media accounts. To all the “please disable your ad blockers” companies out there 1) It’s not just about supporting your website 2) Ad blockers are not just avoiding ads. 3) It’s not even necessary (our ads are not blocked on the podcast or the website) Consider browser != Chrome and/or nextdns.io for your whole network Brian #4: Pro-Tip – pytest fixtures are magic! Frank Wiles “The magic of pytest fixtures is how they are injected into your tests for you to use and their composability. When done well, writing tests is considerably easier and actually fun.” Setup code is often the biggest headache of test writing. pytest fixtures help solve the setup problem. Fixtures are used by just including them in a tests parameter list can build on top of each other can be used in any test in the project, if you put it in a central conftest.py can return all kinds of things: static data, instantiated objects, callables to make other things, Frank includes an interesting way to organize fixtures such that they are defined in local test directories but usable across a project, under “Organizing Your Fixtures”. Plugins with fixtures: A shoutout to pytest-django and a Revsys plugin called django-test-plus. Built-in fixtures. See also tmp_path. autouse One bit of incorrect info: autouse doesn’t work like that. The value of global_thing cannot be grabbed unless you list it in the parameter list. It will run before every function (since it’s scope=``"``function``" by default), but you gotta list it to get the value. To be fair, it’s really hard to come up with good autouse examples. Partly because there are so few good reasons to use autouse. Extras Brian: Porting Python projects to Rust International Obfuscated Python Code Competition Michael: Remember the AMA (submit your question). “Scheduled” on July 11th at 11am. Joke: Marked as duplicate
#340 Snorkel not included
Watch on YouTube About the show Sponsored by InfluxDB from Influxdata. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Ask me anything episode: Submit your question(s) for our upcoming AMA episode: form here. Thank you! Brian #1: PythonGUIS Martin Fitzpatrick A site with a collection of resources, guides, books, comparisons, etc, around GUIs in Python. Martin recommends starting with PyQT6 However, there are tutorials covering PyQT6 PySide6 PyQT5 TkInter PySide even Kivy Michael #2: JupyterLab 4.0 is Here The next major release of our full-featured development environment You can upgrade by running pip install --upgrade jupyterlab or conda install -c conda-forge jupyterlab. JupyterLab is now faster, thanks to improvements such as CSS rules optimization, CodeMirror 6, MathJax 3, and notebook windowing. JupyterLab 3 was when working with large notebooks. There are additional performance improvements available via opt-in settings: Faster tab-switching on Chromium browsers: “Settings” → “JupyterLab Shell” → switch “Hidden mode” to “contentVisibility” Better performance with long notebooks: “Settings” → “Notebook” → switch “Windowing mode” to “full” An upgraded text editor. Better real time collaboration. Bug fixes. More than 100 bugs have been addressed and resolved, enhancing JupyterLab’s stability and performance. Brian #3: Proposing a struct syntax for Python Brett Cannon This would be a cool syntax for a data only type: struct Point(x: int, y: int) No positional only parameters No inheritance No methods Instances would be immutable, so p = Point(1, 2) would create an object that could be used as a key. A data only focused set of types. Michael #4: Python 3.13 Removes 20 Stdlib Modules via PyCoders From PEP 594 – Removing dead batteries from the standard library we’re saying goodbye to aifc, audioop, cgi, cgitb, chunk, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib As well as the 2to3 program and lib2to3 module in Python. Python 3.12 final release is scheduled in 4 months (October 2023) and Python 3.13 final release is scheduled in 1 year and 4 months (October 2024). Extras Brian: Affirming your PSF Membership voting status You have until June 15 to affirm your voting rights in the upcoming Board Election, if you care about such things. Michael: 5 Career Tips for Budding Python Developers video PyCon US 2023 videos are up Python 3.11.4, 3.10.12, 3.9.17, 3.8.17, 3.7.17, and 3.12.0 beta 2 are now available Joke: Snorkel not included
#339 Actual Technical People
Watch on YouTube About the show Sponsored by InfluxDB from Influxdata. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: pystack PyStack is a tool that uses forbidden magic to let you inspect the stack frames of a running Python process or a Python core dump, helping you quickly and easily learn what it's doing. PyStack has the following amazing features: 💻 Works with both running processes and core dump files. 🧵 Shows if each thread currently holds the Python GIL, is waiting to acquire it, or is currently dropping it. 🗑️ Shows if a thread is running a garbage collection cycle. 🐍 Optionally shows native function calls, as well as Python ones. In this mode, PyStack prints the native stack trace (C/C++/Rust function calls), except that the calls to Python callables are replaced with frames showing the Python code being executed, instead of showing the internal C code the interpreter used to make the call. 🔍 Automatically demangles symbols shown in the native stack. 📈 Includes calls to inlined functions in the native stack whenever enough debug information is available. 🔍 Optionally shows the values of local variables and function arguments in Python stack frames. 🔒 Safe to use on running processes. PyStack does not modify any memory or execute any code in a process that is running. It simply attaches just long enough to read some of the process's memory. ⚡ Optionally, it can perform a Python stack analysis without pausing the process at all. This minimizes impact to the debugged process, at the cost of potentially failing due to data races. 🚀 Super fast! It can analyze core files 10x faster than general-purpose tools like GDB. 🎯 Even works with aggressively optimized Python interpreter binaries. 🔍 Even works with Python interpreters' binaries that do not have symbols or debug information (Python stack only). 💥 Tolerates memory corruption well. Even if the process crashed due to memory corruption, PyStack can usually reconstruct the stack. 💼 Self-contained: it does not depend on external tools or programs other than the Python interpreter used to run PyStack itself. Brian #2: Securing PyPI accounts via Two-Factor Authentication Donald Stufft “… every account that maintains any project or organization on PyPI will be required to enable 2FA on their account by the end of 2023.” “One of the key security promises that PyPI makes is that when you're downloading something, that only the people associated with that project are going to be able to upload, delete, or otherwise modify a project. That when you look at that project and see that it is owned by someone that you trust, that you can be assured that nobody else is making changes to that package on PyPI.” If you maintain a package on PyPI to a point where you are uploading to PyPI or plan to soon, enable 2FA on you account. May as well do it sooner than later. But officially, you have the rest of the year. This has already been a requirement for “critical projects” since last summer. (top 1% of downloads, about 3,500 projects.) See episode 293. Now it’s going to be for everyone. Michael #3: Propan - a declarative Python MQ framework Propan is a powerful and easy-to-use Python framework for building asynchronous web services that interact with any MQ Broker. It's following by fastapi, simplify Message Brokers around code writing and provides a helpful development toolkit, which existed only in HTTP-frameworks world until now. It is a modern, high-level framework on top of popular specific Python brokers libraries, based on pydantic and fastapi, pytest concepts. The key features are Simple: Designed to be easy to use and learn. Intuitive: Great editor support. Autocompletion everywhere. Dependencies management: Minimization of code duplication. Access to dependencies at any level of the call stack. Integrations: Propan is fully compatible with any HTTP framework you want MQ independent: Single interface to popular MQ: Redis (based on redis-py) RabbitMQ (based on aio-pika) Kafka (based on aiokafka) SQS (based on aiobotocore) Nats (based on nats-py) RPC: The framework supports RPC requests over MQ, which will allow performing long operations on remote services asynchronously. Great to develop: CLI tool provides great development experience: framework-independent way to manage the project environment application code hot reload robust application templates Testability: Propan allows you to test your app without external dependencies: you do not have to set up a Message Broker, you can use a virtual one! Brian #4: Makefile tricks for Python projects Ricardo Ander-Egg A pretty short basic starter template Makefile for Python projects with some cool snippets. Some default settings to have make behave sanely exit on error, warn about undefined variables, disable built-in rules set up working directory correctly if called from a different dir A $(py) definition that picks up the virtual environment if it’s there. Also $(pip) from the virtual env. Default goal and help message cool trick so that make with no arguments just prints the help And also picks up target comments as help text for the target. Neat. Injecting paths into PYTHONPATH, and an example on how that works if you need it. A .venv target create a virtual environment, update setuptools, wheel, build, and install requirments Extras Michael: PyCon Portugal CFP DjangoCon 2024 Apple Vision Pro Duet App Joke: Actual technical people
#338 Scripting iOS with Python
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Special guest: GUEST_PROFILE Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: The Basics of Python Packaging in Early 2023 Jay Qi Good description of a minimal-ish pyproject.toml file, which includes a build backend and project metadata. That’s all you need for a Python-only project. Discussion of how to choose a build backend. Mostly it’s baed on extra features you might want, like hatchling’s include/exclude features for source distributions. Some discussion of frontend choices. Nice discussion of non-Python-only builds. Specifically, if you need to compile C or C++ extensions, you can use scikit-build-core, or meson-python, or setuptools. Related: "Sharing is Caring - Sharing pytest Fixtures" by Brian Okken (PyCascades 2023) My PyCascades 2023 on packaging pytest plugins is up on YouTube Michael #2: vecs via Oli Python collection-like interface to storing and searching vectors in postgres. Vector search is a key component in building AI chatbots, and semantic document search. If you're familiar with the space, it's effectively Pinecone built on free OSS It's under the Supabase github org but it's fully open source, and compatible with any pgvector vendor, e.g. RDS, or locally in docker If you’re on macOS and need Postgres, Postgres App is a good option. Brian #3: Introducing Grasshopper - An Open Source Python Library for Load Testing Jacob Fiola “Grasshopper is a library for automated load testing, written in Python.” Open source project from Alteryx, On GitHub and PyPI under the name locust-grasshopper Built on Locust. Adds Tag-based suites for trend analysis and evaluating changes. Custom trends. Useful for actions that span multiple http calls, and you want to see timing trends for the whole action. Checks. Checks validate boolean conditions in the test. Custom tagging for all metrics Send data to time series db & dashboards. Thresholds. Reporting results to other locations. Some reusable base classes that take care of the majority of the boilerplate that tests often contain Readme has a very thorough introduction including configuration and samples. Michael #4: memocast by Daniel Engvall A small iOS app for e.g. iPhone that allow you to add links heard in podcasts into reminders. Good example of how to use Pythonista to build python scripts for iOS Pythonista just made an update (2 weeks ago) so that'd use Python 3.10 on the iOS which makes it even more interesting. Extras Brian: Help test Python 3.12 beta! Python Language Summit write-ups available PyPI was subpoenaed Michael: You Can Ignore This Post Joke: Careful or you might end up summoning a demon.
#337 Backtracking For a Package
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Special guest: GUEST_PROFILE Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Ruff PyCharm plugin via John Hagen Ruff PyCharm plugin has great integration, it will highlight Ruff lint errors in the IDE as you type and you can even apply Alt+Enter (⌥⏎ on Mac) quick fixes through the IDE. Ruff will automatically fix the fixable issues. John also added additional PyCharm-specific instructions for black/Ruff Brian #2: Writing Python like it's Rust Kobzol Rust lessons guiding use of types and type hints in Python Add type hints tun function signatures Use dataclasses instead of tuples or dictionaries to increase clarity and type safety Union types to clarify | typing Michael #3: Pip 23.1 Released - Massive improvement to backtracking Pip 23.1 was released last month Highlight the significant improvement in backtracking that is part of the requirement resolver process in Pip. This process involves Pip finding a set of packages that meet your requirements and whose requirements themselves don't conflict. Prior to Pip 20.3, the default process for Pip would allow conflicting requirements to install if they were transitive dependencies where the last one specified would be the one installed. Once the new resolver was turned on by default it immediately hit problems where backtracking would get stuck for a long time. Pip separates out the resolution logic into a library called resolvelib. It had been discovered that there was a logical error under certain circumstances, and also there was a known better backtracking technique it could employ called backjumping. Both of these were recently fixed and implemented in resolvelib, which were then vendored in to Pip 23.1. Brian #4: Markdown Code Runner markdown-code-runner is a Python package that automatically executes code blocks within a Markdown file, including hidden code blocks, and updates the output in-place. Works with Python & Bash see also cog Extras Michael: Python 3.12.0a7 is out python3 -m venv --upgrade-deps venv (via John Hagen) Talk submissions are now open for both remote and in-person talks at the 2023 PyConZA? The conference will be held on 5 and 6 October 2023 in Durban, South Africa. South Africa is GMT+2, so the times are convenient for Africa, Europe and much of Asia, although probably less so for the rest of the world. All details are on za.pycon.org - via Kim van Wik Trail discovery/riding apps onX offroad Gaia Picture of Michael on one of these trails Joke: User Inyerface