Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.
Similar Podcasts
The Infinite Monkey Cage
Brian Cox and Robin Ince host a witty, irreverent look at the world through scientists' eyes.
The Top Shelf
ThePrimeagen and teej_dv are on a quest to find the best possible technical speakers and ask the best possible questions we can find. You all know ThePrimeagen can't read, so this is a great format for him to really shine. Teej is here to make sure that Prime knows who the guest is and also to interrupt Prime wherever possible
24H24L
Evento en línea, de 24 horas de duración que consiste en la emisión de 24 audios de diversas temáticas sobre GNU/Linux. Estos son los audios del evento en formato podcast.
#320 The Bug Is In The JavaScript
Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: markdown-it-py Yes. another markdown parser. Rich recently switched markdown parsers, from commonmark to markdown-it-py. Let’s look at those a bit. Michael #2: Sketch via Jake Firman Sketch is an AI code-writing assistant for pandas users that understands the context of your data A Natural Language interface that successfully navigates many tasks in the data stack landscape. Data Cataloging: General tagging (eg. PII identification) Metadata generation (names and descriptions) Data Engineering: Data cleaning and masking (compliance) Derived feature creation and extraction Data Analysis: Data questions Data visualization Watch the video on the GitHub page for a quick intro Brian #3: Fixing Circular Imports in Python with Protocol Built on Subclassing in Python Redux from Hynek We covered this in the summer of 2021, episode 240 However, I re-read it recently due to a typing problem Problem is when an object passes itself to another module to be called later. This is common in many design patterns, including just normal callback functions. Normally not a problem with Python, due to duck typing. But with type hints, suddenly it seems like both modules need types from the other. So how do you have two modules use types from each other without a circular import. Hynek produces two options Abstract Data Types, aka Interfaces, using the abc module Requires a third interface class Structural subtyping with Protocol This is what I think I’ll use more often and I’m kinda in love with it now that I understand it. Still has a third type, but one of the modules doesn’t have to know about it. "Structural Subtyping : Structural subtyping is duck typing for types: if your class fulfills the constraints of a [Protocol](https://docs.python.org/3/library/typing.html#typing.Protocol), it’s automatically considered a subtype of it. Therefore, a class can implement many Protocols from all kinds of packages without knowing about them!” The Fixing Circular Imports in Python with Protocol article walks through one example of two classes talking with each other, typing, circular imports, and fixing them with Protocol Michael #4: unrepl via/by Ruud van der Ham We’ve seen the code samples: >>> board = [] >>> for i in range(3): ... row = ['_'] * 3 ... board.append(row) ...>>> board [['_', '_', '_'], ['_', '_', '_'], ['_', '_', '_']] >>> board\[2\][0] = 'X'>>> board [['_', '_', '_'], ['_', '_', '_'], ['X', '_', '_']] But you cannot really run this code. You can’t paste it into a REPL yourself nor can you put it into a .py file. So you unrepl it: Copying the above code to the clipboard and run unrepl. Paste the result and now you can. Unrepl can be used as a command line tool but also as a module. The REPL functionality of underscore (_) to get access to the last value is also supported. Extras Michael: You'll want to update your git ASAP. Get course releases at Talk Python via RSS Gist for using Turnstile with Python + Pydantic Joke: there's a bug in the js You’ve checked all your database indexes, You’ve tuned all your API hooks, You’re starting to think That you might need a drink, Because there’s only one place left to look: … There must be a bug in the javascript Because everything else was built properly But the frontend’s a pile of crap ;)
#319 CSS-Style Queries for... JSON?
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Secure maintainer workflow by Ned Batchelder We are the magicians, but also the gatekeepers for our users Terminal sessions with implicit access to credentials first is unlikely: a bad guy gets onto my computer and uses the credentials to cause havoc second way is a more serious concern: I could unknowingly run evil or buggy code that uses my credentials in bad ways. Mitigations 1Password: where possible, I store credentials in 1Password, and use tooling to get them into environment variables. Side bar: Do not use lastpass, see end segment I can have the credentials in the environment for just long enough to use them. This works well for things like PyPI credentials, which are used rarely and could cause significant damage. Docker: To really isolate unknown code, I use a Docker container. Brian #2: Tools for parsing HTML and JSON Learned these from A Year of Writing about Web Scraping in Review Parsel - extract and remove data from HTML using XPath and CSS selectors jmespath - “James Path” - declaratively specify how to extract elements from a JSON document Michael #3: git-sizer Compute various size metrics for a Git repository, flagging those that might cause problems. Tip, partial clone: git clone --filter=blob:none URL # Stats for training.talkpython.fm # Full: git clone repo Receiving objects: 100% (118820/118820), 514.31 MiB | 28.83 MiB/s, done. Resolving deltas: 100% (71763/71763), done. Updating files: 100% (10792/10792), done. 1.01 GB on disk # Partial: git clone --filter=blob:none repo Receiving objects: 100% (10120/10120), 220.25 MiB | 24.92 MiB/s, done. Resolving deltas: 100% (1454/1454), done. Updating files: 100% (10792/10792), done. 694.4 MB on disk Partial clone is a performance optimization that “allows Git to function without having a complete copy of the repository. The goal of this work is to allow Git better handle extremely large repositories.” When changing branches, Git may download more missing files. Not the same as shallow clones or sparse checkouts Consider shallow clones for CI/CD/deployment Sparse checkouts for a slice of a monorepo Brian #4: Dataclasses without type annotations Probably file this under “don’t try this at home”. Or maybe “try this at home, but not at work”. Or just “that Brian fella is a bad influence”. What! It’s not me. It’s Adrian, the dude that wrote the article. Unless you’re using a type checker, for dataclasses, “… use any type you want. If you're not using a static type checker, no one is going to care what type you use.” @dataclass class Literally: anything: ("can go", "in here") as_long_as: lambda: "it can be evaluated" # Now, I've noticed a tendency for this program to get rather silly. hell: with_("from __future__ import annotations") it_s: not even.evaluated it: just.has(to=be) * syntactically[valid] # Right! Stop that! It's SILLY! Extras Michael: LastPass story just keeps getting worse We will see problems in supply chains because of this too A whole 2 hour discussion diving into what I touched on: twit.tv/shows/security-now Got your new mac mini yet? Or MacBook Pro? Joke: Developer/maker, what’s my purpose?
#318 GIL, How We Will Miss You
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with us Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: PEP 703 - Making the GIL Optional in CPython Author: Sam Gross Sponsor: Łukasz Langa Draft status, but on Standards Track, targeting Python 3.12 Suggested by: Will Shanks “The GIL is a major obstacle to concurrency.” Especially for scientific computing. PEP 703 proposes adding a --without-gil build configuration to CPython to let it run code without the global interpreter lock and with the necessary changes needed to make the interpreter thread-safe. PEP includes several issues with GIL and sckikit-learn, PyTorch, Numpy, Pillow, and other numerically intensive libraries. Python’s GIL makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications. There’s also a section on how the GIL makes many types of parallelism difficult to express. Changes primarily in internals, and not much exposed to public Python and C APIs: Reference counting Memory management Container thread-safety Locking and atomic APIs Includes information on all of these challenges. Distribution C-API extension authors will need access to a --without-gil Python to modify their projects and supply --without-gil versions. Sam is proposing “To mitigate this, the author will work with Anaconda to distribute a --without-gil version of Python together with compatible packages from conda channels. This centralizes the challenges of building extensions, and the author believes this will enable more people to use Python without the GIL sooner than they would otherwise be able to.” Michael #2: FerretDB Via Jon Bultmeyer A truly Open Source MongoDB alternative MongoDB abandoned its Open-Source roots, changing the license to SSPL making it unusable for many Open Source and Commercial Projects. The core of our solution is a stateless proxy, which converts MongoDB protocol queries to SQL, and uses PostgreSQL as a database engine. FerretDB will be compatible with MongoDB drivers and will strive to serve as a drop-in replacement for MongoDB 6.0+. First release back in Nov 2022 I still love you MongoDB ;) Brian #3: Four tips for structuring your research group’s Python packages David Aaron Nicholson Not PyPI packages, but, you know, directories with __init__.py in them. Corrections for mistakes I see frequently Give your packages and modules terse, single-word names whenever possible. Import modules internally, instead of importing everything from modules. Make use of sub-packages. Prefer modules with very specific names containing single functions over modules with very general names like utils, helpers, or support that contain many functions. Michael #4: Quibbler Quibbler is a toolset for building highly interactive, yet reproducible, transparent and efficient data analysis pipelines. One import statement and matplotlib becomes interactive. Check out the video on the repo page. Extras Brian: And now for something completely different: turtles talk Michael: More RSS recommendations FreshRSS a self-hosted RSS and Atom feed aggregator. Feedly (for AI) Flym for Android Readwise is very interesting RSS for courses at Talk Python New article: Dev on the Road Joke: Testing the program Joke: Every Cloud Architecture
#317 Most loved and most dreaded dev tools of 2022
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Michael #1: StackOverflow 2022 Developer Survey Last year we saw Git as a fundamental tool to being a developer. This year it appears that Docker is becoming a similar fundamental tool for Professional Developers, increasing from 55% to 69%. Language: Rust is […] the most loved language with 87% of developers saying they want to continue using it. JS Frameworks: Angular.js is in its third year as the most dreaded. Let me Google that for you: 62% of all respondents spend more than 30 minutes a day searching for answers or solutions to problems. 25% spending more than an hour each day. The demise of the full-stack developer is overrated. I do wish there were more women in the field. Databases: Postgres is #1 and MongoDB is still going strong. The “which web framework do you use?” question is a full on train wreck. Why is this so hard for people to write the question? Node.js or Express (built on Node) vs. FastAPI or Flask (but no Python?) Most wanted / loved language is Rust (wanted) and Python/Rust tied for most wanted. Worked with vs. want to work with has some interesting graphics. Brian #2: PePy.tech - PyPI download stats with package version breakdown Petru Rares Sincraian We’ve discussed pypistats.org before, which highlights daily downloads downloads per major/minor Python version downloads per OS PyPy is a bit more useful for me default shows last few versions and total for this major version “select versions” box is editable. clicking in it shows dropdown with downloads per version already there you can add * for graph of total or other major versions if you want to compare daily/weekly/monthly is nice, to round out some noise and see larger trends Oddity I noticed - daily graph isn’t the same dates as the table. off by a day on both sides not a big deal, but I notice these kinds of things. Michael #3: Codon Python Compiler via Jeff Hutchins and Abdulaziz Alqasem A high-performance, zero-overhead, extensible Python compiler using LLVM You can scale performance and produce executables, even when using third party libraries such as matplotlib. It also supports writing and executing GPU kernels, which is an interesting feature. See how it works at exaloop.io BTW, really terrible licensing. Free for non-commercial (great) “Contact us” for commercial use (it’s fine to charge, but give us a price) Brian #4: 8 Levels of Using Type Hints in Python Yang Zhou (yahng cho) A progression of using type hints that seems to track how I’ve picked them up Type Hints for Basic Data Types. x: int Define a Constant Using Final Type DB: Final = '``PostgreSQL' (ok. I haven’t used this one at all yet) Adding multipe type hints to one variable. int | None Using general type hints. def func(nums: Iterable) Also using Optional Type hints for functions def func(name: str) → str: (I probably would put this at #2) Alias of type hints (not used this yet, but looks cool) PostsType = dict[int, str] new_posts: PostsType = {1: 'Python Type Hints', 2: 'Python Tricks'} Type hints for a class itself, i.e. Self type from typing import Self class ListNode: def __init__(self, prev_node: Self) -> None: pass Provide literals for a variable. (not used this yet, but looks cool) from typing import Literal weekend_day: Literal['Saturday', 'Sunday'] weekend_day = 'Saturday' weekend_day = 'Monday' # will by a type error Extras Brian: I hear a heartbeat for Test & Code, so it must not be dead yet. Michael: New article: Welcome Back RSS From this I learned about Readwise, Kustosz, and Python’s reader. Year progress == 100% PyTorch discloses malicious dependency chain compromise over holidays (of course found over RSS and reeder — see article above) Joke: vim switch
#316 Python 3.11 is here and it's fast (crossover)
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show announcements: @pythonbytes@fosstodon.org Hi folks. For our final episode of 2022 here on Python Bytes, we're crossing the streams with my other show, Talk Python To Me. I present to you one of the more important episodes of the year, the release of Python 3.11 with it's new features and 40% performance improvements. Thank you for listening to Python Bytes in 2022, have a great holiday break, and Brian and I will see you next week. Python 3.11 is here! Keeping with the annual release cycle, the Python core devs have released the latest version of Python. And this one is a big one. It has more friendly error messages and is massively faster than 3.10 (between 10 to 60% faster) which is a big deal for a year over year release of a 30 year old platform. On this episode, we have Irit Katriel, Pablo Galindo Salgado, Mark Shannon, and Brandt Bucher all of whom participated in releasing Python this week on the show to tell us about that process and some of the highlight features. Guests Irit Katriel @iritkatriel Mark Shannon linkedin.com Pablo Galindo Salgado @pyblogsal Brandt Bucher github.com Resources from the show Michael's Python 3.11 Course talkpython.fm/py311 Python 3.11.0 is now available blog.python.org PEP 101 - Releasing Python peps.python.org PEP 678 – Enriching Exceptions with Notes peps.python.org PEP 654 – Exception Groups and except* peps.python.org PEP 657 – Include Fine Grained Error Locations in Tracebacks peps.python.org Python Buildbot python.org Making Python Faster Talk Python Episode talkpython.fm Specializing, Adaptive Interpreter on Talk Python talkpython.fm Specialist Visualizer github.com "Zero cost" exception handling github.com Pyodide pyodide.org pyscript pyscript.net
#315 Some Stickers!
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Michael #1: Jupyter Server 2.0 is released! Jupyter Server provides the core web server that powers JupyterLab and Jupyter Notebook. New Identity API: As Jupyter continues to innovate its real-time collaboration experience, identity is an important component. New Authorization API: Enabling collaboration on a notebook shouldn’t mean “allow everyone with access to my Jupyter Server to edit my notebooks”. What if I want to share my notebook with e.g. a subset of my teammates? New Event System API: jupyter_events—a package that provides a JSON-schema-based event-driven system to Jupyter Server and server extensions. Terminals Service is now a Server Extension: Jupyter Server now ships the “Terminals Service” as an extension (installed and enabled by default) rather than a core Jupyter Service. pytest-jupyter: A pytest plugin for Jupyter Brian #2: Converting to pyproject.toml Last week, episode 314, we talked about “Tools for rewriting Python code” and I mentioned a desire to convert setup.py/setup.cfg to pyproject.toml Several of you, including Christian Clauss and Brian Skinn, let me know about a few tools to help in that area. Thank you. ini2toml - Automatically translates .ini/.cfg files into TOML “… can also be used to convert any compatible .ini/.cfg file to TOML.” “ini2toml comes in two flavours: “lite” and “full”. The “lite” flavour will create a TOML document that does not contain any of the comments from the original .ini/.cfg file. On the other hand, the “full” flavour will make an extra effort to translate these comments into a TOML-equivalent (please notice sometimes this translation is not perfect, so it is always good to check the TOML document afterwards).” pyproject-fmt - Apply a consistent format to pyproject.toml files Having a consistent ordering and such is actually quite nice. I agreed with most changes when I tried it, except one change. The faulty change: it modified the name of my project. Not cool. pytest plugins are traditionally named pytest-something. the tool replaced the - with _. Wrong. So, be careful with adding this to your tool chain if you have intentional dashes in the name. Otherwise, and still, cool project. validate-pyproject - Automated checks on pyproject.toml powered by JSON Schema definitions It’s a bit terse with errors, but still useful. $ validate-pyproject pyproject.toml Invalid file: pyproject.toml [ERROR] `project.authors[{data__authors_x}]` must be object $ validate-pyproject pyproject.toml Invalid file: pyproject.toml [ERROR] Invalid value (at line 3, column 12) I’d probably add tox Don’t forget to build and test your project after making changes to pyproject.toml You’ll catch things like missing dependencies that the other tools will miss. Michael #3: aws-lambda-powertools-python Via Mark Pender A suite of utilities for AWS Lambda Functions that makes distributed tracing, structured logging, custom metrics, idempotency, and many leading practices easier Looks kinda cool if you prefer to work almost entirely in python and avoid using any 3rd party tools like Terraform etc to manage the support functions of deploying, monitoring, debugging lambda functions - Tracing: Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions Logging - Structured logging made easier, and decorator to enrich structured logging with key Lambda context details Metrics - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) Event handler: AppSync - AWS AppSync event handler for Lambda Direct Resolver and Amplify GraphQL Transformer function Event handler: API Gateway and ALB - Amazon API Gateway REST/HTTP API and ALB event handler for Lambda functions invoked using Proxy integration Bring your own middleware - Decorator factory to create your own middleware to run logic before, and after each Lambda invocation Parameters utility - Retrieve and cache parameter values from Parameter Store, Secrets Manager, or DynamoDB Batch processing - Handle partial failures for AWS SQS batch processing Typing - Static typing classes to speedup development in your IDE Validation - JSON Schema validator for inbound events and responses Event source data classes - Data classes describing the schema of common Lambda event triggers Parser - Data parsing and deep validation using Pydantic Idempotency - Convert your Lambda functions into idempotent operations which are safe to retry Feature Flags - A simple rule engine to evaluate when one or multiple features should be enabled depending on the input Streaming - Streams datasets larger than the available memory as streaming data. Brian #4: How to create a self updating GitHub Readme Bob Belderbos Bob’s GitHub profile is nice Includes latest Pybites articles, latest Python tips, and even latest Fosstodon toots And he includes a link to an article on how he did this. A Python script that pulls together all of the content, build-readme.py and fills in a TEMPLATE.md markdown file. It gets called through a GitHub action workflow, update.yml and automatically commits changes, currently daily at 8:45 This happens every day, and it looks like there are only commits if Note: We covered Simon Willison’s notes on self updating README on episode 192 in 2020 Extras Brian: GitHub can check your repos for leaked secrets. Julia Evans has released a new zine, The Pocket Guide to Debugging Python Easter Eggs Includes this fun one from 2009 from Barry Warsaw and Brett Cannon >>> from __future__ import barry_as_FLUFL >>> 1 <> 2 True >>> 1 != 2 File "[HTML_REMOVED]", line 1 1 != 2 ^ SyntaxError: invalid syntax Crontab Guru Michael: Canary Email AI 3.11 delivers First chance to try “iPad as the sole travel device.” Here’s a report. Follow up from 306 and 309 discussions. Maps be free New laptop design Joke: What are clouds made of?
#314 What are you, a wise guy? Sort it out!
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Brian #1: FAQtory Will McGugan “FAQtory is a tool to auto-generate a FAQ.md (Frequently Asked Questions) document for your project. FAQtory has a FAQ.md written by itself, so you can see an example of the project in the project. Builds a markdown FAQ.md that includes questions at the top that link to answers below. “Additionally, a ‘suggest’ feature uses fuzzy matching to reply to GitHub issues with suggestions from your FAQ.” I haven’t tried this part, but looking forward to it. May help to answer GH issues that are really questions. Michael #2: Kagi search "live with it” report Still enjoying it a lot Very fast LOVE blocking SEO-heavy, content-light sites Maps are rough around the edges Not obvious how to set as a private/incognito search engine (but can be done in settings) They have browser extensions - but I don't want to install extensions I only use 1password & zoom It could use some documentation however (e.g. supports !’s, but what are they?) Being tempted by Orion too, but sticking with Vivaldi. Brian #3: Tools for rewriting Python code Luke Plant A collection of tools change your code (hopefully for the better) Several categories formatting and coding style - black, isort, … upgrades - pyupgrade, flynt, … we need one to convert from setup.py/setup.cfg to pyproject.toml type hints - auto type hints? cool. maybe. I haven’t tried any of these, but they look interesting refactoring, editors, rope, jedi other - autoflake, shed, … write your own, with LibCST Michael #4: Socketify Bringing WebSockets, Http/Https High Performance servers for PyPy3 and Python3 A new record for Python no other Web Framework as able to reach 6.2 mi requests per second before in @TFBenchmarks 🥇 🏆 This puts Python in the same ballpark than #golang, #Rust and #C++. Extras Brian: watching mousebender from Brett Cannon BTW, releases watching is cool. Probably a decent reason to use GH releases feature. Python Developer’s Guide has a visual of the Python Versions and release cycle. Shows the stages of releases: end-of-life, security, bugfix, feature Next end-of-life is Python 3.7 on 27-June-2023 Great descriptions of what all these terms mean at the bottom Michael: Michael’s latest post: Sometimes, You Should Build It Yourself Trying all in on proton for personal stuff Bunny fonts AI Stand up Comedy Joke: Wise guy, eh?
#313 Programming Robots With a Marker
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Special guest: Kelly Schuster-Paredes Special guest: Sean Tibor Michael #1: How do you say that number? Inflect: fosstodon.org/@linuxgal@techhub.social/109430499504962727 Num2Words: pypi.org/project/num2words/ # Inflect: import inflect inflector=inflect.engine() print(inflector.number_to_words(8675309)) # eight million, six hundred and seventy-five thousand, three hundred and nine # Num2Words from num2words import num2words print(num2words(8675309)) # eight million, six hundred and seventy-five thousand, three hundred and nine Num2Words also has a CLI: pipx install num2words $ num2words 2948475 two million, nine hundred and forty-eight thousand, four hundred and seventy-five Brian #2: The Origins of Python Lambert Meertens A wonderful tale starting with TELCOMP, traveling through ABC, and finally reaching Python. This is a long article, but a wonderful story. It includes a nice emphasis at all times to keep a language simple enough for the absolute beginner but powerful enough to not be annoying for experienced developers. A few quotes from the article: “Ease of learning and ease of use are both desirable attributes in any programming language. Nonetheless, I have often felt that this aspect of language design does not always receive the attention it deserves. And what may seem easy to a designer may not necessarily be easy for a language learner.” Regarding ABC: “To serve our intended users, absolute beginners, we sought to hide low-level implementation details and instead to provide powerful high-level, task-oriented features. Then Python: “The growth in popularity of Python, from its inception thirty years ago as a one-person effort flying under the radar, has been phenomenal, but not meteoric. Instead it has been a long, slow, and steady rise. Python’s ease of learning gave it a competitive advantage in a period when there was a perpetual need for more programmers. Its clean syntax and semantics make it easier to maintain a software base written in Python than other languages—an important consideration given that the cost of maintaining software dwarfs the cost of creating new software.” Kelly #3: Ozobot Evo Introduces a Python Beta Version. (August 17, 2022) The original Ozobot model – the Ozobot Bit – is no longer available for purchase . The New Evo Ozobit- has three Kit options. The Entry Kit (single robot), the Ozobot Evo 12-Pack, and the Ozobot Evo 18-Pack. https://beta.python.ozobot.com/doc-python-api/ozobot.html#module-0 Still has the updated OzoBlockly platform for Block Programming. This tiny bot comes with: Line following Color detection Sound proximity sensor bluetooth Crash detection Students can even code functions Ozobot simulator(block) https://games.ozoblockly.com/shapetracer-freeform Web beta app: beta.python.ozobot.com Michael #4: setproctitle A Python module to customize the process title Awesome for servers and anytime “python” is not enough Easy to use directly: from setproctitle import setproctitle setproctitle("tp-search daemon") Used automatically by servers like uwsgi and gunicorn I believe. ### # uWSGI server configuration ### [uwsgi] # uWSGI provides some functionality which can help identify the workers procname-prefix = training- auto-procname = true Some nice results, example from Talk Python Training Brian #5: Looking forward to Python 3.12 New features in 3.12a2 Improved Error Messages lots of other goodies, like pathlib.walk(). Release scheduled for Oct 2023 But why wait? Start testing your projects with it now: Testing with Python 3.12 Note that “During the alpha phase, features may be added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is not recommended for production environments.” Actually, with that note, you might want to wait. I don’t. I’ll deal with it when/if I get a failure. Sean #6: Re:Invent 2022 EF Education Breakout Presentation at AWS Re:Invent 2022 Complete redesign of online learning platform by one of the largest education companies in the world We’ve all seen Zoom classrooms with rows on rows of students A more immersive experience for learning with green screens, digital sets, and props Massive amount of analytics around student engagement and learning, including full transcription of every student, engagement tracking, and computer vision Extras Michael: You can support the PSF if you’re selling things on EBay. Check “Donate a portion to charity” and choose “Python Software Foundation” via Joe Riedley Textinator for Windows (the Windows version of TextSniper) Paperlike for iPad Kelly: A new Special Interest Group for the PSF launched 6 days ago. “Edu-sig, through its mailing list, provides an informal venue for comparing notes and discussing future possibilities for Python in education.” Led by Timothy Wilson. Sean: Dr. Werner Vogel’s keynote - everything is coming up async EventBridge Pipes Jokes: fosstodon.org/@kimvanwyk/109389398652030679 And a new mastodon user: fosstodon.org/@vruz@mastodon.social/109394538570819699
#312 AI Goes on Trial For Writing Code
Watch on YouTube About the show Sponsored by Complier Podcast from RedHat Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Brian #1: Coping strategies for the serial project hoarder Simon Willison Also a talk from DjangoCon2022 Massively increase your productivity on personal projects with comprehensive documentation and automated tests. I’m actually not sure what title would be best, but this is an incredible video that I’m encouraging every developer to watch, whether or not you work with open source projects. Covers The perfect commit Implementation, Tests, Documentation, and a link to an issue thread Tests Prove the implementation works, pass if it works, fails otherwise A discussion of how adding tests is way easier than starting testing a project, so get the framework in place early, and devs won’t be afraid to add to it. Cookiecutter repo templates for projects you will likely start super cool idea to have your own that you keep up to date with your preferred best practices A trick for using GitHub actions to use those templates to populate new repos Trying this out is on my todo list Documentation must live in the same repo as the code and be included in PRs for the PR to be accepted by code review maybe even test this using documentation unit tests Everything links to an issue thread Keep all of your thoughts in an issue thread Doesn’t have to be a dialog with anyone but yourself This allows you to NOT HAVE TO REMEMBER ANYTHING Tell people what you did This is just as important in work projects as it is in open source Blog about it Post on Twitter (or Mastodon, etc.) Avoid side projects with user accounts “If you build something that people can sign into, that’s not a side-project, it’s an unpaid job. It’s a very big responsibility, avoid at all costs!” - this is hilarious and something I’m probably not going to follow Michael #2: GitHub copilot lawsuit First, we aren’t lawyers Lawsuit filed on November 3, 2022 We’ve filed a lawsuit challenging GitHub Copilot, an AI product that relies on unprecedented open-source software piracy. GitHub copilot is trained on projects on GitHub, including GPL and other restrictive licenses This is the first class-action case in the US challenging the training and output of AI systems. Brian #3: Use Windows Dialog Boxes from Python with no extra libraries Actual title: Display a message box with Python without using a non-standard library or other dependency (Windows) By Matt Callahan / learned about from from PyCoders weekly When I need a simple pop up dialog box that’s cross platform, PySimpleGUI is awesome and so easy. But when I KNOW it’s only going to run on Windows, why not just use native dialog boxes? Matt’s article shows you how, using ctypes to call into a Windows dll. Small example from article: import ctypes def main(): WS_EX_TOPMOST = 0x40000 windowTitle = "Python Windows Message Box Test" message = "Hello, world!" # display a message box; execution will stop here until user acknowledges ctypes.windll.user32.MessageBoxExW(None, message, windowTitle, WS_EX_TOPMOST) print("User clicked OK.") if __name__ == "__main__": main() Notes: The uType (fourth) parameter is a multi-use value that can be or-ed for things like: Type of dialog box: Help, OK, OK/Cancel, Retry/Cancel, Yes/No, etc. The icon to use: Exclamation, Info, Question, etc. Modality, … Return value is used to understand how user reacted: 1 - OK, 2 - Cancel (or x), …, 6 - Yes, 7 - No, … Michael #4: Extra Extra Extra Python browser extensions takahe - Mastodon on Python - the right way Michael’s article in Black Friday perf We could scale down our server after what I’ve learned. But we’d pay 10x more in bandwidth overages ironically: Last month Talk Python broadly transferred 20.2 TB of data from our servers Moved our static traffic to Bunny CDN, highly recommended service RSS revival My blog: mkennedy.codes Reeder 5 app on iOS and macOS Rivers Cuomo (from Weezer) and Guido sit down for a talk together Also check out the Talk Python episode with Rivers: talkpython.fm/327 Kite is saying farewell
#311 Catching Memory Leaks with ... pytest?
Watch on YouTube About the show Python Bytes 311 Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Special guest: Murilo Cunha Michael #1: Latexify We are used to turning beautiful math into programming symbols. For example: amitness.com/2019/08/math-for-programmers/#sigma Take this code: def do_math(a, b, c): return (-b + math.sqrt(b ** 2 - 4 * a * c)) / (2 * a) Add @latexify.function decorator display do_math in a notebook Get this latex: \mathrm{do_math}(a, b, c) = \frac{-b + \sqrt{b^{{2}} - {4} a c}}{{2} a} Which renders as I could only get it to install with: pip install git+https://github.com/google/latexify_py Brian #2: prefixed From Avram Lubkin “Prefixed provides an alternative implementation of the built-in float which supports formatted output with SI (decimal) and IEC (binary) prefixes.” >>> from prefixed import Float >>> f'{Float(3250):.2h}' '3.25k' >>> '{:.2h}s'.format(Float(.00001534)) '15.34μs' >>> '{:.2k}B'.format(Float(42467328)) '40.50MiB' >>> f'{Float(2048):.2m}B' '2.00KB' Because prefixed.Float inherits from the built-in float, it behaves exactly the same in most cases. When a math operation is performed with another real number type (float, int), the result will be a prefixed.Float instance. also interesting First new SI prefixes for over 30 years new prefixes also show up here - Murilo #3: dbt Open source tool CLI tool Built with Python 🐍 Applies “best practices” to SQL projects Combines git + .sql files + jinja Support many data platforms Let’s you Template SQL queries Including loops Execute DAGs Data validation Easily build docs (data lineage, visualize DAGs, etc.) Now you can also run Python models Useful if there’s a convenient python function for your data transformation or some more complex logic (i.e.:fuzzy string matching, machine learning models, etc.) Available for Snowflake, Databricks, BigQuery dbt’s coalesce’s announcement https://www.youtube.com/watch?v=rVprdyxcGUo Michael #4: Memray pytest plugin pytest-memray is the pytest plugin for, well, memray. :) You can ensure that not too much memory is used with @pytest``**.**``mark``**.**``limit_memory``**(**``"24 MB"``**)** And you get an allocation report with pytest --memray file.py But coming soon, we’ll have memory leak checking too. @pytest.mark.check_leaks() def test_foobar(): # Do some stuff and ensure # it does not leak memory pass Brian #5: Stealing Open Source code from Textual Will McGugan Will reminds us of one of the great benefits of open source code, stealing code (when allowed by the license, of course) Goes as far as to point out some bits of textual that you might want to lift looping with indication of when you’ve hit the first or last item a LRUCache with more flexibility than lru_cache a Color class with conversions for css, hex, monochrome, hsl 2d geometry Murilo #6: Shed Superset of black "shed is the maximally opinionated autoformatting tool. It's all about convention over configuration, and designed to be a single opinionated tool that fully canonicalises my code - formatting, imports, updates, and every other fix I can possibly automate.” Also format code snippets in docstrings, markdown, restructured text No configuration options pre-commit hooks available Bundles together: black isort autoflake pyupgrade blacken-docs Extras Brian: pytest-check (version 1.1.3) changes now live New README, hopefully makes it clear how to use. Use check from from pytest_check import check or from the check fixture: def test_foo(check): … Either form returns the same object. From that check object, you can use helper functions like check.equal(a, b), etc. use it as a context manager, with check: assert a == b even grab the raises context manager: with check.raises(Exception): … Intended to be backwards compatible although some old use cases might be deprecated/removed in the future. Michael: New YouTube Video: Best Native App for Mastodon is ... Nearly 50% of macOS malware comes from one app — do you have it on your MacBook? PyCascades CfP A fresh take on blogging (for Michael): mkennedy.codes Based on Hugo - which is so good. Hosted on netlify.com Murilo: mastodon.py - a Python wrapper around Mastodon’s API Nice notebook diffs in Github PRs 🚀 flake8 is not on Gitlab anymore Who’s gonna win the world cup? lancer Joke: Messing with the algorithm Let’s start this one with some history Recusion joke
#310 Calling All Tools for Readmes
Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Adam Hopkins Python Web Development with Sanic Brian #1: Tips for clean code in Python Bob Belderbos Generally some great tips to think about to keep code maintainable: Smaller units. Break things up into single responsibility. SRP: Single Responsibility Principle Move magic numbers into constants or parameters. Avoid global scope. (even though it’s not really global) Use linters and auto-formatters. Use very narrow try/except blocks. Idiomatic Python. (Although I agree, this is a weird one as it’s hard for new people to follow). Pay attention to data structure choice and learn to utilize standard structures and those in collections. Use the standard libary. Use mappings Flat is better than nested. But I’m gonna focus on the “smaller units” because it applies to modules as well. Try to keep modules organized such that you can keep relevant and related code concepts in your head. Michael #2: Mastodon is picking up speed @pythonbytes@fosstodon.org @mkennedy@fosstodon.org @brianokken@fosstodon.org @admhpkns@fosstodon.org I’m calling this a “Mastodon First” strategy rather than “Let’s burn down Twitter and scatter” Just did a Talk Python about it Money in mouth: I became a patreon of Fosstodon and Mastodon’s company Mastodon is open source, find it here Twitter’s potential collapse could wipe out vast records of recent human history Python’s API for Mastodon: toot Download a proper Twitter archive with this Python script Integrated the API into stream deck You can install it as a PWA: Adam #3: Correction to Sanic Worker Manager in v22.9 Episode #308 covered a recent article published on a new feature in Sanic v22.9 Blog article: Pushing work to the background of your Sanic app The segment focused on the celery-like job queue in Sanic Clarification: Goal of the feature is to bring a consistent development experience from dev thru prod Enables usage of multiprocessing-safe shared objects Simple pattern for managing multiple long-running processes Release notes for Sanic v22.9 Sanic documentation on the Worker Manager Brian #4: Some FastAPI news, and some great READMEs. FastAPI 0.87.0 has some interesting notes Upgraded Starlette, which includes TestClient based on HTTPX instead of Requests Since that might break some peoples use of TestClient, someone named Kludex built bump-testclient to help automatically convert test code to the new interface. That’s so cool! Use Ruff for linting Add a Help Maintain FastAPI section to the docs that emphasizes that it’s super helpful to: Help others with issues Review PRs Both of those sections have other expanded sections to describe what that means. The FastAPI commitment to great documentation is amazing and worth emulating. It also has a really good README. Interesting sponsors section. Cool way for a popular project to get maintenance funding. Testimonials. It’s like a sales landing page, which really, a README kinda is. Other common good practices and cool items Images Some use of collapsable sections. Other notable READMEs pytest short example right away to show how simple it can be to use. textual and rich great use of images and short examples highlighting often missed features, such as pretty and inspect Utilizing expandable/collapsable sections for longer examples httpx like pytest, shows a small example quickly, redirects many other sections to more thorough docs. Michael #5: Closevember An annual event focused on sustainable open source development practices and maintainer well-being. Let's support open source maintainers by helping them close issues and pull requests throughout November. Over at closember.org Contributing to a project carries a number of responsibilities, in order to make it as easy as possible for a project to receive that contribution. For Maintainers: How to Get Ready (see site) If you only want assistance with closing some issues and PRs, then tag your repo with closember and you’re all set. One thing that we often find helpful is to declutter our physical and digital environment: tidying our desks a bit, decluttering our computers’ desktops, unsubscribing from some email lists ... that sort of thing. I did this this month actually. Spent 6 hours completely rebuilding my desk to have zero wires and look tidy and clean (hint: 3m of industrial velcro and things stuck upside down) and formatted my computer to a fresh OS after two years. For the Community: How to Participate (see site) If you’ve never used GitHub before, your first step is going to be signing up for a free account. Also, if you’re super new to git: talkpython.fm/git If you’ve opened issues or PRs on projects in the past, you can start by taking a look at your own GitHub issues and your own PRs to see if any of them are outdated or have already been fixed—if so, close them! After that, start browsing projects: take a look at your favorite projects and see if they’ve been tagged with closember, or browse the list of closember projects. Check out the close boards (on the site) Adam #6: Super simple “Cache with async power” using Cashews Recently popped up in my GitHub Explore Cashews: Async cache framework with simple API to build fast and reliable applications Super simple out-of-the-box API supports in memory, Redis, DiskCache (local sqlite) one-line setup then implemented as a decorator Human-friendly TTL values: example “3h” Client-side caching - For example, if you are using Redis backend you do not need to make a network call on every cache request Strategies for common cache issues cache hits, early recalculation, soft TTL, resource locking, rate limiting!, circuit breaker Has its own interface for middleware Extras Michael: Take the PSF survey Adam: Voting season is upon us: Python Steering Council nominations are open Joke: JavaScript has been Banned from Twitter
#309 When Malware PoC's are Themselves Malware
Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Michael: #0: Python Bytes is 6 years old this week. Thank you! 🎉 Michael #1: Malicious proof-of-concepts are exposing GitHub users to malware and more The paper They found that of the 47,313 GitHub repositories they had downloaded and checked, 4,893 (10.3%) were malicious. In some the attackers were trying to plant malware on users’ machines, while in others, they tried to open backdoors using CobaltStrike, for example Ignoring this problem can cause damage that ranges from infecting yourself as [a] user, to infecting your company and likely your customers as well if it’s a more sophisticated attack,” El Yadmani warned. Languages Ruby 379 Go 400 JavaScript 548 Shell 652 C++ 962 Java 1071 C 1686 Python 8305 Undetected 31858 Example Python exfile script included in the paper Brian #2: The great Mastodon experiment Context should be obvious re Twitter news. A lot of Python people have kept in touch via Twitter. A lot are now experimenting with Mastadon, What I did asked Twitter people which server to use, then just picked fosstodon.org, but there are many servers This is me: @brianokken@fosstodon.org Michael got in too: https://fosstodon.org/@mkennedy just started using it, following people, trying iOS clients, etc. Now I’m ready for some tutorials, and here’s a list that looks decent: An Increasingly Less-Brief Guide to Mastodon Everything I know about Mastodon A hastily written guide for data science folks trying to navigate the fediverse. Mastodon is just blogs - Simon Willison is running his own server. Eight Mastodon apps for iPhone - I’m currently trying like 4, but you can also just log into your sever and do everything there. Fedi.Tips and their Beginners Start Here page Michael #3: Gitpod and the traveling dev Gitpod is an open-source Kubernetes application for ready-to-code developer environments that spins up fresh, automated dev environments for each task, in the cloud, in seconds. Gitpod is paid, but there are decent free tiers Features Run a desktop or browser based version of VS Code or any JetBrains IDE and customise it to your individual needs - from themes to extensions, you have full control. Brian #4: Color in the terminal pytest-check currently doesn’t use color but a little red for failures would be good (and was requested via an issue) I could use rich, but maybe that’s a slightly larger hammer than I need for this job Maybe raw escape sequences like print('\033[31m' + 'some red text') kinda gross won’t work out of the box on Windows. But colorama can fix Windows. It just recently added just_fix_windows_console(), which apparently works better than init() in that it can be called multiple times without blowing up. Includes easier to read codes for some basic colors, so this works: from colorama import just_fix_windows_console from colorama import Fore, Style just_fix_windows_console() print(Fore.RED + 'some red text') print(Style.RESET_ALL) print('back to normal now') Extras Brian: Simon Willison wrote What to blog about, which includes TIL (today I learned) posts that don’t need to be full tutorials Projects you’ve built I’d like to include Projects in progress Bug fixes or feature additions where you needed to learn a bit of something beforehand Example: I should write up “Adding red to pytest-check” Michael: Beanie reorg: There is no sync version here more. Please use Bunnet instead https://twitter.com/nicholdav/status/1589643652598759424 ? PyCon Days Breakdown Been playing with GeForce now, really impressive. Meanwhile, why is google still selling stadia? New video: A Walrus Meets a Python - What is the := Walrus Operator? New video: Python GC Settings - Change This and Go 20% Faster! Joke: Relaxation Relax to it on YouTube: youtube.com
#308 Conference season is heating up
Watch the live stream: Watch on YouTube About the show Sponsored by Complier Podcast from RedHat Michael #0: New livestream time - 11am PT on Tuesdays. Also, subscribe to the youtube channel and “hit the bell” to get notified of all the live streams. Brian #1: It’s PyCon US 2023 CFP time Will be held in Salt Lake City, Salt Palace Convention Center Talks are Friday - Sunday, April 19-23 PyCon US 2023 launch announcement PyCon 2023 site features images taken from past PyCon artwork Call for proposals open until Dec 9, but please don’t wait that long. Michael #2: Any.io AnyIO is an asynchronous networking and concurrency library that works on top of either asyncio or trio. It implements trio-like structured concurrency (SC) on top of asyncio. Cool interpretability between native threads and asyncio Using subprocesses: AnyIO allows you to run arbitrary executables in subprocesses, either as a one-shot call or by opening a process handle for you that gives you more control over the subprocess. Async file I/O: AnyIO provides asynchronous wrappers for blocking file operations. These wrappers run blocking operations in worker threads. Cool synchronization primitives too. Catch the Talk Python episode with Alex: talkpython.fm/385 Brian #3: How to propose a winning conference talk Reuven Lerner Some nice tips and advice Build a list of topics If you train, teach, mentor, lead, or coach already: what questions to people always ask you? what knowledge would help people to have? where do people seem to just “not get it”? If you don’t train or teach, then maybe hit up Stack Overflow… From Brian: I think you can imagine yourself a year or two ago and think about stuff you know now you wish you knew then and could learn faster. Build an outline with times This part often seems scary, but Reuven’s example is 10 bullets with (x min) notes. Write up a summary. One short, one longer. Indicate who will benefit, what they will come out knowing, and how it will help them. Propose to multiple conferences. Why not? Practice (from Brian: Even if you get rejected, you’ve gained. Turn it into a youTube video or blog post or both.) - Michael #4: Sanic release adds background workers via Felix In v22.9 (go cal-ver!), the main new feature is the worker process management - the main Sanic process handles a pool of workers. They are normally used for handling requests but you can also use them to handle background jobs and similar things. You could probably use it for a lot of the reasons people turn to something like Celery. The lead developer (Adam Hopkins) has written a blog post about this feature. MK: Sanic has been flying a bit under my radar. Maybe time to dive into it a bit more. Extras Brian: Create Presentation from Jupyter Notebook Cool walkthrough of how to use the built in slideshow features of Jupyter Notebooks. pytest 7.2.0 is out No longer depends on the py library. So if you do, you need to add it to your dependencies. nose officially deprecated, which includes setup() and teardown(). Really glad I dropped the “x unit” section on the 2nd edition of the pytest book. testpaths now supports shell-style wildcards Lots of other improvements. check out the change log Michael: Rich on pyscript (via Matt Kramer) Python 3.11 in 100 seconds video from Michael Joke: Deep questions & Relationship advice from geeks
#307 Your Python just got faster (3.11 is out!)
Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Michael #1: Python 3.11 is released Live stream of the actual release procedure Talk Python episode coming next week (live stream on Friday) Major new features of the 3.11 series, compared to 3.10 General changes PEP 657 -- Include Fine-Grained Error Locations in Tracebacks PEP 654 -- Exception Groups and except* PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library gh-90908 -- Introduce task groups to asyncio gh-34627 -- Atomic grouping ((?>...)) and possessive quantifiers (*+, ++, ?+, {m,n}+) are now supported in regular expressions. The Faster CPython Project is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython for details. Typing and typing language changes PEP 673 -- Self Type PEP 646 -- Variadic Generics PEP 675 -- Arbitrary Literal String Type PEP 655 -- Marking individual TypedDict items as required or potentially-missing PEP 681 -- Data Class Transforms Brian #2: Installing Python 3.11 on Mac or Windows pythontest.com I wrote this up because there are lots tutorials with weird instructions. For most people, I think the right answer is to use the python.org installer. It just works. One change, for Windows: Use “Advanced Options” and select “Add Python to environment variables”. The default process: allows multiple versions, like 3.7, 3.10, 3.11, to all live side by side. allows tox to use all of these allows you to specify which one if you want python3.10, for example, on mac py -3.10 on windows allows you to update versions in place. Say 3.10.7 to 3.10.8, or 3.11.0 to 3.11.1 when it comes out. Please, blog writers, stop recommending pyenv to novices. It’s a cool project, but it is not a project for casual users. And homebrew lovers, go for it, you are not going to read my article anyway. Michael #4: Bossie 2022 Awards Notable winners Wasmtime: Similar to what Node.js does for the JavaScript runtime, Wasmtime allows developers to leverage all of the advantages that WebAssembly provides inside the browser-including safe sandboxed execution, near-native performance, and support across multiple programming languages and platforms -outside the browser. (Python’s integration) PyScript: One of the long-gestating promises of WebAssembly is enabling the use of languages other than JavaScript in the web browser. PyScript delivers a full Python runtime in the browser, allowing you to use Python in webpages as a full-blown scripting language. Sentry: Alongside security, error and performance tracing are among the most frustratingly inevitable requirements for many apps. Cue a sigh of relief. Sentry offers an entire ecosystem of open source tools for monitoring the health of applications, services, and APIs, from the server-side API for collecting data, to a dashboard for making it manageable, to a comprehensive slew of application-side integrations. nbdev: One of the dirty secrets of notebook programming, using environments like Jupyter or Google Colab, is that it produces some of the worst spaghetti code you've ever seen, with data scientists hopping from cell to cell and creating an unmaintainable mess. Some even go so far as to say that notebook programming might be as harmful as GOTO was back in the day. Brian #5: Textual 0.2.0 All the cool things Will has been showing off on Twitter recently are part of the css branch. This has been merged, and released as 0.2.0 They also held off this release until they were happy with the documentation, which includes: A new tutorial that walks through a stopwatch application and everything that goes into it. An in depth reference guide with fully working examples, all of which are also in github, so you can play with it directly without retyping everything. Extras Michael: Video I created: 17x Faster Page Load in 30 minutes using Python, PyCharm, and MongoDB Pandas Markets Calendar (by Ryan Sheftel) Beanie adds a sync API DuckDuckGo browser, exciting and disappointing int() isn’t done yet via Will Shanks Ubuntu has decided to patch out the int limit and preserve the previous behavior on the basis that the risk factor is low and not worth breaking compatibility for. Joke: i heard you like getters
#306 Some Fun pytesting Tools
Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Brian #1: Awesome pytest speedup Neyts Zupan A checklist of best practices to speed up your pytest suite. as a talk at Plone NAMUR 2022 Measure first Then make sure (all items have explanations) Hardware is fast use a faster computer also try a self-hosted runner seriously, a dedicated computer (or a few) for making test runs faster might be worth it. CI resources are usually slower in cloud than local, and even expensive VM farms are often slower. Try local Collection is fast utilize norecursedirs and specifying the location of the tests, either on the command line or with testpaths PYTHONDONTWRITEBYTECODE=1 is set might help Built-in pytest plugins are disabled try -p no:pastebin -p no:nose -p no:doctest Only a subset of tests are executed Especially when developing or debugging, run a subset and skip the slow tests. Network access is disabled pytest-socket can make sure of that Disk access is disabled interesting idea Database access is optimized great discussion here, including using truncate and rollback. Tests run in parallel pytest-xdist or similar Then keep them fast monitor test speed Michael #2: Strive to travel without a laptop Prompt from Panic for SSH on iThings github.dev for an editor on iPad Push to branch for continuous deployment BTW, Apple could just make M1 iPads boot to macOS rather than chase silly multi windowing systems (stage manager, etc, etc) Brian #3: Some fun tools from the previous testing article hyperfine for timing the whole suite pytest --``durations 10 for finding test times of slowest 10 tests leave the 10 off to find times of everything, sorted pyinstrument for profiling with nice tree structures and how to use it with pytest pytest-socket disables network calls with --disable-socket, helping to find tests that use network calls. pyfakefs, a fake file system that mocks the Python file system modules. “Using pyfakefs, your tests operate on a fake file system in memory without touching the real disk.” BlueRacer.io Michael #4: Refurb A tool for refurbishing and modernizing Python codebases Think of it as suggesting the pythonic line of code. A little sampling of what I got on Talk Python Training file.py:186:25 [FURB106]: Replace x.replace("\t", " ") with x.expandtabs(1) file.py:128:17 [FURB131]: Replace del x[y] with x.pop(y) file.py:103:17 [FURB131]: Replace del x[y] with x.pop(y) file.py:112:39 [FURB109]: Replace not in [x, y, z] with not in (x, y, z) file.py:45:5 [FURB131]: Replace del x[y] with x.pop(y) file.py:81:21 [FURB131]: Replace del x[y] with x.pop(y) file.py:143:9 [FURB131]: Replace del x[y] with x.pop(y) file.py:8:50 [FURB123]: Replace list(x) with x.copy() You don’t always want the change, can suppress the recommendation with either a CLI flag or comment. Extras Michael: Back on episode 54 in 2017 we discussed python apps in systemd daemons. Multiprocessing allows for a cool way to save on server memory Do the scheduled work a multiprocessing.Process Here’s an example from Talk Python Training Completely rewrote search UI for Talk Python courses Google analytics is now illegal? Fleet is finally in public preview I’ll be on a JetBrains/PyCharm webcast Thursday. Joke: Tests pass