Conversations about programming. By Andreas Ekeroot and Lars Wikman, funded by Underjord.io.
About Ranting at Ecto
Stories about Ecto quickly redeeming itself, and of what it takes to introduce foreign keys.Some of us are super comfortable referencing the ID. Lars dislikes that Ecto needs to be more complicated because of SQL, but the abstractions do hold.Also: the biggest reason to ever use a ORM! It can be reallynice to come back to one after a tour of plain SQL-land.Some people have just been bitten so hard by cowboys.LinksEctoForeign keysRethinkDBReferential integrityAXALantmännenModelForm in DjangoCowboy and PlugDSLUpsertsFragmentsHaxl - DSL for creating queriesSQLAlchemyets
About Long-Lived Code
Fredrik wants to think about long-lived code. Lars is offended, Andreas only a little bit so.Are there other good software development practices out there? Other than the ones focusing on building something quickly? Practices for building software which lives on and is maintained for much longer than we seem to care to admit? Should we remove dependencies over time? The swamp of dependency management and vendoring is probed, gradually shifting into firmware, the horrors of floating point (proper excuses are made), small language models.Finally, of course, indecent cups of tea.LinksLagomReactFlux architectureReduxChangelog episode with Justin Searls about dependencies as liabilitiesKent Beck talking about managing risks in software developmentKent Beck drawing on a whiteboard and staring at the audienceMithril.jsInteract.jsVendoringWorking effectively with legacy code - the book about legacy systemsDelphi 5FlaskDynamic linkingSAMLPOSIXLibcGlibcMuslH.264MicrocodeOxide and friendsCoral TPU:sTensorflow lite286PentiumCUDAROCmQuantizationLLaMA
About Fat Tuesday Buns
The Saint Valentine's peak passed without issue. Andreas had time for semlor.Lars has opinions on semlor, and can imagine many possible improvements. Like having an apple. Or a pizza.Lars has had a nice influx of work, including hardware work using Nerves. Testing and very hackish hot code reloading are both included.Finally, some thoughts on Linux audio, and musings about the possibility of creating really nice audio tools for the platform.LinksSaint ValentineThe strangler fig patternThe strangler patternPhoenixCowboySemlaMudcakeThe Swedish chef making chocolate mooseFinnish fastlagsbulle with jamOne of Lars' blog posts about NervesFrank Hunleth - also hot code reloads the way Lars has doneLars' Stream deck library for ElixirStream deckElgato key lightPulseAudioPipeWireRogue amoeba's audio tools for MacJACKCustom APT repositoryQuotesThe Nordics go all awkward and weirdIn my heart, it was a catastrophyHad time for semlorAn unimpressive pastryIt's less messy to have an appleProfessional nervesBuilding with nervesA reasonable enough abstractionThe Rogue Amoeba for Linux
About things you built long ago that start doing weird things
Andreas tells the story of a old system which suddenly exhibited a new and frightening bug. Lars shares similar experiences of things going wrong in new and novel ways.When things do go wrong, it is so nice to have supervision trees or other things which allow you to hear about problems, not to mention recover from them.Also covered are some stories about TCP, networks, and timeouts. And a realization that testing the frameworks upon which you build could have saved some bacon, had it just been done a long time ago.LinksDjangoModel-view-controllerDrupalUnicode collationSupervision treesOxide and friends - episode 27TCP_NODELAYQUIC and HTTP/3UDPNyqvist-Shannon sampling theoremHexagonal designQuotesGaming convention management systemWhen I say view, I mean controllerView is a better wordIf I ignore it, it will go awayDestructive favouritesAlternative class hierarchiesFailed in new and novel waysBoth a mistake, and interestingAaah, circumflex!TCP the good parts
About Data Pipelines
Lars dove into data pipelines, and emerged bearing arrows and wishing for a lot fewer copies.What is there to think about regarding data pipelines, what is interesting about them?Which tools are out there, and why might you want to use them?Why all this talk about making fewer copies of data?What does Lars' current ideal pipeline look like, and where does Elixir fit in?Links Matt Topol Apache Arrow Large language models Vector search BigQuery sed AWK jq Replacing Hadoop with bash - "Command-line Tools can be 235x Faster than your Hadoop Cluster" Hadoop MapReduce Unix pipes Directed acyclic graph tee - to "materialize inbetween states" Apache Beam Apache Spark Apache Flink Apache Pulsar Airbyte - shoves data between systems using connectors Cronjob Fivetran - Airbyte competitor Apache Airflow ETL - Extract, transform, load Designing data-intensive applications Stream processing Ephemerality Data lake Data warehouse The people's front of Judea DBT - SQL-SQL batch-work-thingy SQL with Jinja templates Snowflake - data warehouse thing Scala Broadway Oban - "robust job processing for Elixir" Dashbit pandas - Python data library APL Arrow flight GRPC DataFusion - query execution engine Polars - "DataFrames in Rust" Explorer - built on top of Polars Voltron data The Composable Codex Pyarrow - Arrow bindings for Python Quotes I've been reading a lot about data pipelines What's so special about data pipelines? There's a lot of special tooling There's a lot of bad, bad tooling Less than optimal tooling Converging on something biggerlk He got me eventually All of your steps in one bucket What tools do you associate with data? I inherited a data pipeline BashReduce Iterate on the L and the T The modern data stack And then you demand more work No unnecessary copies Barely a copy Reconnecting with my Python roots
About Fun With GenServers
GenServers are fun! Andreas gives all the context. Things were learned, knowledge was aquired. You can do so much with GenServers, but make sure you have a good reason.If you don't watch out, this is where concurrency goes to die.Dynamic supervisors, and their children, are thoroughly considered.Also delved into is the mess other ecosystems make of doing things at the same time, waiting, and so on.The strange worlds of C and other unusual languages are considered.Finally, an interesting bug.Links Alan Turing Turing machine GenServer Cowboy Plug Umbrella ETS - Erlang Term Storage Øredev The actor model Virding's first rule of programming Registry DynamicSupervisor The Goth library - Google auth library for Elixir The GIL - the global interpreter lock Friday afternoon deploy Promises Esbuild Uiua - "A stack-based array programming language" Prefix tree Packmatic library, by Evadne Wu - streaming zip archives Quotes Where the system grows horizontally The kind of thing that starts happening when you hire developers It was missing a hat I have become nothing, the simplifier of things Where all the concurrency goes to die A whole dance party of sad, dark people The children of the dynamic supervisor Homes can be nodes Hundreds of interested parties Turns life into promises Poking some C programmers
About What Every Web App Needs But Your Developer Does Not Want You To Know About
Every web app starts out fine, the tabula rasa of an unwritten BODY. But sooner or later you need users. And a million other things which live in trees.Also: email.And that layer between the controller and the database where things like fine-grained access control goes.I'd like to have an admin, please.Eventually, web apps grows up. And while a larger framework with solutions and conventions for all those grown-up features may not necessarily be fun, it can certainly be useful.Links APM - Application Performance Management Django Teams should be an MVP feature! Bullet Train - a "Ruby on Rails SaaS framework" Flask Express Sinatra Scotty Phoenix Auth0 Okta Postfix Postmark Django Anymail Swoosh Model-view-template ACL:s - access-control lists Ecto Multitenancy Zack Daniel on Beam Radio Zack's Elixirconf talk Ash framework Plug DSL - domain-specific language Bigquery gRPC HIPPA Postgrest Function based views Django REST Laravel Titles Check in on your application Do you want details? The view is the controller Because names I'd like to have an admin, please The admin is kind of rough All the data is introspectable Endgame application Not another user management system A very special can of worms
About Code Nerds
The software development industry is very much built for code nerds. It shouldn’t be.Many of us know many people who are really into coding. Not every working developer can, or even should, be though. Doesn't that create kind of a weird gap between professionals who live and breathe code both on and off work, and those who have a more balanced life?Being passionate about your job shouldn't be an expectation or requirement for anyone or anything.Is there too little space for learning - are we assumed to know too much, and assumed to spend our own time figuring out things we don't?Your path into coding is not, can not, and should not be the only path possible.Links The Python 2 to 3 transition Robert A. Heinlein in 999 Words: What Every Human Should Know Ghost in the shell Harvest moon 4x - Explore, expand, exploit, exterminate TDD - test-driven development BDD - behavior-driven development Charity Majors 2017 blog post about career paths for developers. (Bonus: 2019 follow-up about engineering managers) Late-stage capitalism Quotes I think that's perfectly healthy Surrounded by them Delving into software Surrounded by nerds Much more reasonable answers Where the nerd doesn't go so deep Computers are troublesome Why should you be passionate about your job? Squeeze the passion juice Too passionate to defend themselves Experience or scar tissue? Many developers have lives Popping out for the big picture Doing good work takes all kinds
About Databases
Data has moved to a real database. Next, there may be brave attempts to add actual structure. Working with a real database is nice, as is not losing data, and being able to restore.Not everything is ephemeral, after all.Database service providers and cool stuff they do are discussed. The deal with Elastic is clarified. Finally, it is revealed where you should store your traces.It is actually probably fine.Links MongoDB RethinkDB Ecto Ecto changesets Database schema OpenAPI Ash framework Django Django REST framework X-Request-ID Transactions Write-ahead log pg_restore pg_dump Bigquery Logical replication Fly.io ElectricSQL Flutter React native CRDT:s Phoenix liveview Firefly Webassembly SQLite DuckDB Clickhouse Developer voices podcast Episode about Clickhouse Plausible analytics Fathom analytics podcast Paul Jarvis - the "Company of One " guy Jack Ellis Laravel Fathom Elasticsearch Meilisearch Lucene Cardinality Honeycomb OpenTelemetry New relic HyperDX Datadog Quotes A worse MongoDB Migration complete Everything is ephemeral The idea is to add lots of columns It seems a bit more Django The stakes were high but the budget was almost zero It is actually probably fine
About Mingling
It seems a mingle is a thing, and not just in Swedish! But what do we want to get out of them, how do we go into them, and how do we create good ones?Do you want resonance or hole-poking when you tell people about your plan to arm toddlers with nuclear weapons? Do you want to successfully mingle nerds, or just hit the snacks hard?The foood, the cake, the coffee, and the old classmates. Too hot, too loud, too crowded.Links Mingle (noun) Ben Orenstein Tuple Thougtbot podcasts - The bike shed and Giant robots. Ben was on episode 183 of The bike shed and episode 136 of Giant robots. Øredev The art of gathering The Whova app Quotes Interesting and fun Arm toddlers with nuclear weapons We don't trust solutions Excitement and resonance Intensively and excitedly and indefinitely The active rubberduck strategy Talking to fish in a barrell Successfully mingle nerds Hit the snacks pretty hard
About Performance
Performance: we wish the incentives were there to focus on it more often.Lars would like more opportunities and incentives to focus on making things fast, rather than just making them not slow. Unfortunately, things tend to line up so that fast enough and more features are in focus. Plus, performance and optimization can be very context sensitive and age out without anyone really noticing.Also pondered: IRC, Gentoo, and the eldritch horrors buried within the x86 architecture.Links Grep os.walk() in Python Why GNU grep is fast Ripgrep Ag - the silver searcher Travelling salesman problem Bin packing Mnesia ORM Project Euler Fibonacci numbers D. Richard Hipp - the guy behind SQLite Changelog episodes with Richard XKCD and IRC IRC IRCCloud Matrix and Element Elm SVG Canvas Neovim Lisp Arch Nix Gentoo Funtoo Dart Flutter Skia - the graphics library under Flutter Linked list Pointers CISC - Complex instruction set computer RISC - Reduced instruction set computer io_uring Quotes Given up on old man Elixir Gotta go fast I never really needed it Grep, naively All the problems at the same time Travelling knapsack problem My ORM-infected brain Measuring things and muttering under my breath I have a hobby, I do job interviews Tools by toolmakers for toolmakers I'm the IRC guy Machine-whispering optimization
About Developing Speed
CTOs want the ability to get prototypes built and out into production fast. Others preach the gospel of building things properly. How fast can you be? How much can you perpare before you hit the ice? And one you built and shipped that prototype, how can you get any kind of speed trying to maintain and evolve something where many corners were cut for speed?How do we want things to work then? Having an algebra for things might be nice. A sprinkling of interface, things that break noisily, and nice toolboxes to work with structs are all discussed.Links The Scott - Amundsen race to the South pole Accelerate, by Nicole Forsgren Parse, don't validate Mnesia Deep modules Pure functions Plug Elm Bruce Tate CRC - Create reduce convert Ecto Roc Happy Path Programming. Episode 47 features Richard Feldman and Roc Richard Feldman, creator of Roc Quotes The gospel of building things properly The key to speed on the ice Before you hit the ice Bare maps Every step made sense The original intent very easily gets lost The curse of all software Strive for maintainability It must not sprawl A little sprinkling of interface At dawn, we roadmap Things that break noisily A quantity unitless The simple case of HTTP
About System Design
Did they do design, or did they just do a system?Distributed systems are hard in many ways. Andreas describes a system communicating between backends and mobile phones in exciting ways with many exciting possibilities for errors. Like data format changes, loss of messages, having 1.5 source of truths, and of course ordering.In certain cases, nobody likes an optimist.The discussion then moves to discuss the working well-windows for various networking solutions, before diving into WebRTC and finishing up with the various dangers of auto.Links Recursion Eventual consistency Pubsub RethinkDB Event sourcing React native Android studio Mnesia - a "distributed, soft real-time database management system" written in Erlang Dirty reads and writes Websockets QUIC UDP TCP WebRTC NAT HTTP live streaming Lars' ElixirConf talk Zoom H4 Zoom H4n pro Quotes Working with systems and feeling the pain Coping with system design Eventually consistent, on a good day Eventually sourced A disappointment to work with Your internal representation of the user This is the shape of the data, deal with it 1.5 source of thruths Oh, it's an optimist I don't like optimists at all Optimist databases Within its working well-window Outside of the working well-window A crash of servers Bad connections over long distances I don't do math
About Conferences
Lars went to ElixirConf EU. Going to a conference can be a credibly incredible experience. Elixir has more clarity than Erlang.Lars also gave a talk, a fact he was comfortably uncomfortable with. Giving a talk also comes with benefits such as being able to talk to fish in a barrel. But why did he choose to make the whole talk a demo? What is the goal of it all?Gotta build things! Dive in, make stuff.Links ElixirConf EU Lars' conference report blog post Code BEAM Sverok Pieter Hintjens about giving talks by talking to the audience Windows 98 (not 95) demo fail Lars' presentation code Voice Driven Development: Who needs a keyboard anyway? - presentation by Emily Shea Hugging Face Quotes Born during ElixirConf Less clarity to it Genservers and stuff Mainstream Elixir Comfortable with that discomfort Talking to fish in a barrel A buddy from the internet The first one I bothered to count Your loose coupling to anything What do you hypothetically know?
About Text Editors
Text editors - which ones do we enjoy, which ones have we used, and what do we actually want and need in them?Andreas has read about vim, sed and awk. Lars is quite comfortable in vim, but finds Visual studio code more than acceptable enough. Andreas is excited to show Lars how to use Vim properly. Lars considers advanced setups something of a hellscape.Lars has held a lecture about functional programming and wishes to provide a path for new .Net developers (dotnet dots?) to become free software zealots.They both share their history of editors.There are dreams of ergonomic editing - of code as well as text in general - on mobile devices.Any other editors we should be trying? No, but you could hack together collaborative vim editing. Links Humble bundle Learning the Vi and Vim Editors - book Vim Thunderdome Monad ex - line editor which inspired vi ed sed & awk - book AWK sed Sublime text Zed Neovim Tmux I3 GNOME Pop!_OS KDE Treesitter ElixirLS FZF - fuzzy finder for the command line Ripgrep Functional programming Monads Fakeroot Notepad.exe Borland Delphi Notepad++ Eclipse Intellij Android studio Xcode Write/Wordpad Nano Pico Gedit Kate Netbeans Atom Scratch GNU Screen Live Share for Visual Studio Code Quotes Learning violent vim Like Thunderdome, but nobody leaves, ever I could do that with monads instead C's strange cousin There's a new sed on the block The power of just good enough Two terminals beside each other It's all a mess in here My sword and lots of configuration files The dotnet dots Quitters don't use Vim Real code is done on the server Notepad the way I want it to work A load-bearing note Exciting and fun, and incredibly unsafe