Python Bytes
Python Bytes
#technology

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

#311 Catching Memory Leaks with ... pytest?

November 24, 2022 00:49:50 42.0 MB Downloads: 0
Watch on YouTube

About the show

Python Bytes 311

Sponsored by Microsoft for Startups Founders Hub.

Connect with the hosts

Michael #1: Latexify

  • We are used to turning beautiful math into programming symbols.
  • For example: amitness.com/2019/08/math-for-programmers/#sigma
  • Take this code: def do_math(a, b, c): return (-b + math.sqrt(b ** 2 - 4 * a * c)) / (2 * a)
  • Add @latexify.function decorator
  • display do_math in a notebook
  • Get this latex: \mathrm{do_math}(a, b, c) = \frac{-b + \sqrt{b^{{2}} - {4} a c}}{{2} a}

  • Which renders as

  • I could only get it to install with: pip install git+https://github.com/google/latexify_py

Brian #2: prefixed

Murilo #3: dbt

  • Open source tool
  • CLI tool
  • Built with Python 🐍
  • Applies “best practices” to SQL projects
  • Combines git + .sql files + jinja
  • Support many data platforms
  • Let’s you
    • Template SQL queries
      • Including loops
    • Execute DAGs
    • Data validation
    • Easily build docs (data lineage, visualize DAGs, etc.)
  • Now you can also run Python models
    • Useful if there’s a convenient python function for your data transformation or some more complex logic (i.e.:fuzzy string matching, machine learning models, etc.)
    • Available for Snowflake, Databricks, BigQuery
    • dbt’s coalesce’s announcement https://www.youtube.com/watch?v=rVprdyxcGUo

Michael #4: Memray pytest plugin

  • pytest-memray is the pytest plugin for, well, memray. :)
  • You can ensure that not too much memory is used with @pytest``**.**``mark``**.**``limit_memory``**(**``"24 MB"``**)**
  • And you get an allocation report with pytest --memray file.py
  • But coming soon, we’ll have memory leak checking too. 
    @pytest.mark.check_leaks()
def test_foobar():
  # Do some stuff and ensure
  # it does not leak memory
  pass

Brian #5: Stealing Open Source code from Textual

  • Will McGugan
  • Will reminds us of one of the great benefits of open source code, stealing code
    • (when allowed by the license, of course)
  • Goes as far as to point out some bits of textual that you might want to lift
    • looping with indication of when you’ve hit the first or last item
    • a LRUCache with more flexibility than lru_cache
    • a Color class with conversions for css, hex, monochrome, hsl
    • 2d geometry

Murilo #6: Shed

  • Superset of black
  • "shed is the maximally opinionated autoformatting tool. It's all about convention over configuration, and designed to be a single opinionated tool that fully canonicalises my code - formatting, imports, updates, and every other fix I can possibly automate.”
  • Also format code snippets in docstrings, markdown, restructured text
  • No configuration options
  • pre-commit hooks available
  • Bundles together:
    • black
    • isort
    • autoflake
    • pyupgrade
    • blacken-docs

Extras

Brian:

  • pytest-check (version 1.1.3) changes now live
    • New README, hopefully makes it clear how to use.
    • Use check from
      • from pytest_check import check
      • or from the check fixture: def test_foo(check): …
    • Either form returns the same object.
    • From that check object, you can
      • use helper functions like check.equal(a, b), etc.
      • use it as a context manager, with check: assert a == b
      • even grab the raises context manager: with check.raises(Exception): …
    • Intended to be backwards compatible
      • although some old use cases might be deprecated/removed in the future.

Michael:

Murilo:

Joke: