Programming Throwdown educates Computer Scientists and Software Engineers on a cavalcade of programming and tech topics. Every show will cover a new programming language, so listeners will be able to speak intelligently about any programming language.
172: Transformers and Large Language Models
172: Transformers and Large Language ModelsIntro topic: Is WFH actually WFC?News/Links:Falsehoods Junior Developers Believe about Becoming Seniorhttps://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/Pure PursuitTutorial with python code: https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit Video example: https://www.youtube.com/watch?v=qYR7mmcwT2w PID without a PHDhttps://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdfGoogle releases Gemmahttps://blog.google/technology/developers/gemma-open-models/Book of the ShowPatrick: The Eye of the World by Robert Jordan (Wheel of Time)https://amzn.to/3uEhg6vJason: How to Make a Video Game All By Yourselfhttps://amzn.to/3UZtP7bPatreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the ShowPatrick: Stadia Controller Wifi to Bluetooth Unlockhttps://stadia.google.com/controller/index_en_US.htmlJason: FUSE and SSHFShttps://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-sshTopic: Transformers and Large Language ModelsHow neural networks store informationLatent variablesTransformersEncoders & DecodersAttention LayersHistoryRNNVanishing Gradient ProblemLSTMShort term (gradient explodes), Long term (gradient vanishes)Differentiable algebraKey-Query-ValueSelf AttentionSelf-Supervised Learning & Forward ModelsHuman FeedbackReinforcement Learning from Human FeedbackDirect Policy Optimization (Pairwise Ranking) ★ Support this podcast on Patreon ★
171: Compilers and Interpreters
Intro topic: Monitor setupsNews/Links:BlueScuti, Willis, beats Tetrishttps://www.youtube.com/watch?v=GuJ5UuknsHUPalWorld accused of being an AI Producthttps://www.forbes.com/sites/paultassi/2024/01/22/palworld-accused-of-using-genai-with-no-evidence-so-far/?sh=26a9651b42394 Billion if-statements to determine if a number is even or oddhttps://andreasjhkarlsson.github.io/jekyll/update/2023/12/27/4-billion-if-statements.htmlSeamless M4Thttps://ai.meta.com/blog/seamless-m4t/Book of the ShowPatrick:Foundation by Isaac Asimovhttps://amzn.to/3SrmgnPJason: Propaganda by Edward Bernayshttps://amzn.to/47JUCXJPatreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the ShowPatrick: The Room Gamehttps://www.fireproofgames.com/games/the-roomJason:Incredibuildhttps://www.incredibuild.com/Topic: Compilers and Interpreters (Request by Jessica W.)Machine CodeArchitecture SpecificAssemblySingle vs Two Pass CompilerHigh level LanguagesIntermediate RepresentationJVM ByteCode vs Machine Code for portabilityScripting/InterpretersJITProfile Guided OptimizationResourceshttps://www.craftinginterpreters.com/https://nandgame.com/Turing Complete ★ Support this podcast on Patreon ★
170: 2023 Holiday Special Live
Predictions:Jason VR for Work Lowering AI training cost/ improved efficiency RISC-V takeoff Patrick Ai claim of AGI Ai peer reviewer Ai Video Generator More space vehicles reaching orbit Early career, finding role at FAANG, liaising vs shipping code. Startup?3 part. 1. How and when current hype for AI will end? 2. Shape of the show 3. Upcoming in techWhat are essential programmer knowledge items?CS Student, how to organize life and goals? What purpose life should serve?What kind of programmer were you in college?Happy Holidays! ★ Support this podcast on Patreon ★
169: HyperLogLog
Intro topic: Testing your car batteryNews/Links: Tech Layoffs still going on https://www.sfchronicle.com/tech/article/google-layoffs-california-companies-18465600.php Real-time dreamy Cloudscapes with Volumetric Raymarchinghttps://blog.maximeheckel.com/posts/real-time-cloudscapes-with-volumetric-raymarching/ Robot Rascals https://en.wikipedia.org/wiki/Robot_Rascals Meta Quest 3 https://www.theverge.com/23906313/meta-quest-3-review-vr-mixed-reality-headset Book of the Show Patrick:HyperLogLog Paper https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf Jason: Eureka! NVIDIA Research Breakthrough Puts New Spin on Robot Learning https://blogs.nvidia.com/blog/2023/10/20/eureka-robotics-research/ Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the Show Patrick: Techtonica: https://store.steampowered.com/app/1457320/Techtonica/ Jason: ESP32 development board: https://amzn.to/3Qpmb20 WEMOS Topic: HyperLogLog MotivationCardinality Counting LinearCounting Hash + expectation of collision based on how full Bloom Filter LogLog Use first N bits as bucket Use max sequential 0s in each bucket Average HyperLogLog Handle empty buckets Use correction factor like linear counting for low counts (number of empty buckets) and high counts Distributing Transfer bucket counts ★ Support this podcast on Patreon ★
168: Godot
Intro topic: What are expectations on developers that have nothing to do with programmingNews/Links: Pushing for a lower dev estimate is like negotiating weather with a Meteorologisthttps://smartguess.is/blog/your-estimate-is-less-than-that/ Announcing python in excelhttps://techcommunity.microsoft.com/t5/excel-blog/announcing-python-in-excel-combining-the-power-of-python-and-the/ba-p/3893439 7 Habits of Highly Effective Software Engineershttps://makingsmallercircles.com/articles/7-habits-of-highly-effective-software-engineers/ Raspberry pi 5 begins shipping https://www.phoronix.com/news/Raspberry-Pi-5-Shipping Book of the Show Patrick:Harry Potter and the Sorcerer's Stone Illustrated Editionhttps://amzn.to/3St3L35 Jason: The Pete and Sebastian Showhttps://podcasts.apple.com/us/podcast/the-pete-and-sebastian-show/id570256898 Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the Show Patrick: Obsidianobsidian.md Jason:Ink by Inkle https://github.com/inkle/ink Topic: Godot What is a game engine? Graphics, animation, particle effects Tilemaps Entity component systems Physics Sound I/O Input handling (touchscreen, joystick) GDScript Why use a game engine? Portability Allow for testing individual components of the game Libraries for ads, in-app-purchases Godot Open source Focused on 2-D but now many 3-D features Great development environment AI Hero Started in phaser (development rut) Moved to Godot, rapid prototyping ★ Support this podcast on Patreon ★
167: Desktop User Interfaces
Intro topic: Jogging MetricsNews/Links: Unholy LLM https://huggingface.co/Undi95/Unholy-v1-12L-13B The reverse red herring https://www.blameless.com/blog/the-reverse-red-herring The "ens–tification" of TikTok https://www.wired.com/story/tiktok-platforms-cory-doctorow/ Response letter to Godot is not the new Unity https://sampruden.github.io/posts/godot-is-not-the-new-unity/ https://gist.github.com/reduz/cb05fe96079e46785f08a79ec3b0ef21 Book of the Show Patrick: Math Games with Bad Drawings by Ben Orlin https://amzn.to/48qlg9A Jason: The Invisible Hook: The Hidden Economics of Pirates https://amzn.to/3LChBff Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the Show Patrick: Factorio (Desktop Game) Jason:AI Hero (iOS and Android) Topic: Desktop user interfaces What is a user interface? Web and Mobile UI toolkits Desktop options Qt Cross platform, custom UI elements Qt creator, code generator WxWidgets Cross platform, uses native UI elements WxFormDesigner, code generation Can look different on different operating systems Electron Local nodejs webserver Html/JavaScript technology Requires interprocess communication to use other languages Jupyter notebooks Mathematica-like notebook Not for distribution StreamlitPython to web compiler Game EnginesUnity, Godot, Unreal Tips for building desktop UI UI is slow (startup time, interaction time) Separate the UI from the engine & business logic ★ Support this podcast on Patreon ★
166: Speedy Database Queries with Lukas Fittl
Links:https://pganalyze.comhttps://www.linkedin.com/in/lfittl/https://www.linkedin.com/company/pganalyze/ ★ Support this podcast on Patreon ★
165: Differential Equations
Intro topic: Revisiting the power of SpreadsheetsNews/Links: LK-99 Isn’t a Superconductorhttps://www.nature.com/articles/d41586-023-02585-7 Normalizing Flowshttps://pyro.ai/examples/normalizing_flows_i.html How is llama.cpp possible?https://finbarr.ca/how-is-llama-cpp-possible/ Chat with open source large language models https://chat.lmsys.org/ Book of the Show Patrick: Math with Bad Drawings by Ben Orlinhttps://amzn.to/44dsgDz Jason: Overboard! https://play.google.com/store/apps/details?id=com.InkleLtd.Overboard Nhl=en_US&gl=US Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the Show Patrick: ffmprovisr https://amiaopensource.github.io/ffmprovisr/ Jason:Pandas read_ods() read_excel() Topic: Differential Equations Why should programmers learn about DiffEqLaw of Large Numbers What are differential equations? When you know the rate of change EigenVectors & EigenValues What is Jacobian What is Jacobian? | The right way of thinking derivatives and integrals Special cases Partial Differential Equations Ordinary Differential Equations Why solvers are important Numerical Stability at larger step sizes Example: https://medium.com/@pukumarathe/eulers-method-and-runge-kutta-4th-order-method-in-python-b4a0068a8ebe Fun Examples Predator-Prey relationships in scipy https://scientific-python.readthedocs.io/en/latest/notebooks_rst/3_Ordinary_Differential_Equations/02_Examples/Lotka_Volterra_model.html Physics Engines for games https://youtu.be/52n2qKgwW_Q PageRank https://arxiv.org/pdf/2001.08973.pdf ★ Support this podcast on Patreon ★
164: Choosing a Database For Your Project With Kris Zyp
Things to consider when choosing a database Speed & Latency Consistency, ACID Compliance Scalability Language support & Developer Experience Relational vs. Non-relational (SQL vs. NoSQL) Data types Security Database environmentClient vs Server access Info on Kris & Harper: Website: harperdb.io Twitter: @harperdbio, @kriszyp Github: @HarperDB, @kriszyp ★ Support this podcast on Patreon ★
163: Recursion
Episode 163 - RecursionIntro topic: Electric CarsNews/Links: Snake Game in 101 Bytes in a QR Codehttps://www.reddit.com/r/programming/comments/15ab4ct/my_qr_code_snake_game_is_now_only_101_bytes/ Superconductor Rumors aboundhttps://arstechnica.com/science/2023/08/whats-going-on-with-the-reports-of-a-room-temperature-superconductor/ OpenWormhttps://github.com/openworm/OpenWorm Creator of vim passes away https://news.itsfoss.com/vim-creator-passed-away/ Book of the Show Patrick:Little Book of Common Sense Investing by Jack Bogle https://amzn.to/43YqANR Jason: Mistborn Saga: https://amzn.to/3DJkUN8 Patreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the Show Jason:reMarkable https://remarkable.com/ Patrick: Stellarium (iOS and Android) Topic: Recursion What is it Divide-And-Conquer Fibonacci numbers How to (not) teach recursion Practical Applications Graph operations Tree retrieval, balancing Graph Search Spatial partitioning PitfallsStack size How to solve problems with recursion (1) Consider the base cases (2) Build the recursive step (3) Look for ways the recursion will not terminate and fix (4) (rest are optional) Remove global contexts (5) Add memoization (6) Build solutions incrementally ★ Support this podcast on Patreon ★
162: Interactive Fiction
In the latest episode of Programming Throwdown, we delve into the captivating world of interactive fiction. We explore: Wordnet, Inform, and how games in the past have been the forerunners of today’s NLP challenges. 00:00:22 Introductions00:00:39 To hard mode or not to hard mode00:08:58 No moats in Google00:16:37 Stable Diffusion blows Jason’s mind00:21:31 Putting beats together00:23:38 GPT4All00:27:44 White Sand00:35:28 Fortuna00:38:55 Patrick’s ‘dirty’ secret00:47:20 Wordnet00:53:56 Procedural generation00:57:29 On tabletop RPGs01:00:48 Inform01:07:27 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today: https://www.patreon.com/programmingthrowdown?ty=h Subscribe to the podcast on Youtube: https://www.youtube.com/@programmingthrowdown4793 News/Links: Google: We have no moat and neither does OpenAIhttps://www.semianalysis.com/p/google-we-have-no-moat-and-neither Stable Diffusion QR Codes https://stable-diffusion-art.com/qr-code/ Beginning to Make Musichttps://learningmusic.ableton.com/ GPT4Allhttps://gpt4all.io/index.html Wordnet:https://wordnet.princeton.edu/ Inform:https://en.wikipedia.org/wiki/Inform Book of the Show Patrick:White Sand https://amzn.to/43CPMKA Jason: The Fortuna https://www.generativefiction.com/ Tool of the Show Jason:Gatsby.js https://www.gatsbyjs.com/ Patrick: Peglin https://store.steampowered.com/app/1296610/Peglin/ If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: programmingthrowdown@gmail.com You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM | Youtube Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★
161: Leveraging Generative AI Models with Hagay Lupesko
MosaicML’s VP Of Engineering, Hagay Lupesko, joins us today to discuss generative AI! We talk about how to use existing models as well as ways to finetune these models to a particular task or domain. 00:01:28 Introductions00:02:09 Hagay’s circuitous career journey00:08:25 Building software for large factories00:17:30 The reality of new technologies00:28:10 AWS00:29:33 Pytorch’s leapfrog advantage00:37:24 MosaicML’s mission00:39:29 Generative AI00:44:39 Giant data models00:57:00 Data access tips01:10:31 MPT-7B01:27:01 Careers in Mosaic01:31:46 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today: https://www.patreon.com/programmingthrowdown?ty=h Subscribe to the podcast on Youtube: https://www.youtube.com/@programmingthrowdown4793 Links: Hagay Lupesko: Linkedin: https://www.linkedin.com/in/hagaylupesko/ Twitter: https://twitter.com/hagay_lupesko Github: https://github.com/lupesko MosaicML: Website: https://www.mosaicml.com/ Careers: https://www.mosaicml.com/careers Twitter: https://twitter.com/MosaicML Linkedin: https://www.linkedin.com/company/mosaicml/ Others: Amp It Up (Amazon): https://www.amazon.com/Amp-Unlocking-Hypergrowth-Expectations-Intensity/dp/1119836115 Hugging Face Hub: https://huggingface.co/ If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: programmingthrowdown@gmail.com You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM | Youtube Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★
160: Position Localization
Where are you now? It’s a question that may seem easy to answer on the surface, but in truth hides more complexity than people expect. In today’s episode, we tackle the latest that they’ve found online on AI, creative endeavors, and more before diving into the meaty discussion of position localization. 00:01:13 Steam Deck00:11:22 Summoning Salt on Mario00:16:49 100k stars00:24:26 ChatGPT spam call00:25:31 Build Your Own DB (from scratch)00:29:50 DuckDB00:35:07 Jason has an idea00:37:58 Fighting Fantasy Classics00:41:52 Patrick’s bread00:47:52 Support the show00:53:54 Awkward CRM emails00:56:07 Rill01:00:29 Position localization in detail01:17:15 Common filter01:25:22 Simultaneous localization01:28:59 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today: https://www.patreon.com/programmingthrowdown?ty=h Subscribe to the podcast on Youtube: https://www.youtube.com/@programmingthrowdown4793News/Links: The History of Super Mario Bros 3 100% World Records (Summoning Salt)https://www.youtube.com/watch?v=_EsFyogVvkw AutoGPT hits 100k starshttps://twitter.com/AlphaSignalAI/status/1649524105647906819 Build Your Own Database from Scratchhttps://build-your-own.org/database/ Asking generative art AI to render mathematical theoremshttps://twitter.com/TivadarDanka/status/1649721970886594561 DuckDB:https://duckdb.org/ Book of the Show: Jason: Fighting Fantasy Classics https://play.google.com/store/apps/details?id=com.tinmangames.ffhub&hl=en_US&gl=US https://apps.apple.com/us/app/fighting-fantasy-classics/id1261201650 Patrick: Evolutions in Bread: Artisan Pan Breads and Dutch-Oven Loaves at Homehttps://amzn.to/44kW4iE Tool of the Show: Jason: Jinja https://jinja.palletsprojects.com/en/3.1.x/ Patrick: Rill https://www.rilldata.com/ If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: programmingthrowdown@gmail.com You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM | Youtube Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★
159: GraphQL with Tanmai Gopal
GraphQL is one of the biggest API enablers in software development, but just how complicated can things be? Tanmai Gopal – Hasura’s CEO extraordinaire – talks with Jason and Patrick about how the secret sauce gets made. They dive deeply from how APIs function to having them managed in practice – among several other topic, making this a must-listen episode. 00:01:19 Introductions00:01:48 Tanmai’s late start in programming00:05:48 Plinko00:13:06 Coursera00:23:28 The question of API development00:30:30 API layer functionality00:34:58 How Hasura leverages JSON00:39:08 GraphQL00:42:49 Worse than an API call00:49:15 The potential REST minefield00:53:41 JSON Web Tokens01:11:34 Scaling writes01:15:17 Careers with Hasura01:22:35 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today: https://www.patreon.com/programmingthrowdown?ty=h Subscribe to the podcast on Youtube: https://www.youtube.com/@programmingthrowdown4793Links: Tanmai Gopal: Website: https://hasura.io/blog/@tanmaig/ Linkedin: https://www.linkedin.com/in/tanmaig/ Twitter: https://twitter.com/tanmaigo Github: https://github.com/coco98 Hasura: Website: https://hasura.io/ Careers: https://hasura.io/careers/ Twitter: https://twitter.com/HasuraHQ Github: https://github.com/hasura Linkedin: https://www.linkedin.com/company/hasura Others: Good Strategy, Bad Strategy (Amazon): https://www.amazon.com/Good-Strategy-Bad-Strategy-audiobook/dp/B07R6XQ8YP Modern Application Development (IIT Madras, archived): https://archive.nptel.ac.in/courses/106/106/106106156/ If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: programmingthrowdown@gmail.com You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM | Youtube Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★
158: Software Supply Chain with Bill Manning
In today’s episode, Jason and Patrick dive deeply with JFrog’s Senior Solutions Engineer, Bill Manning. With the conversation tackling the depth and complexity of software supply chains, vulnerabilities and more, Bill deftly offers grounded advice to listeners old and new. 00:00:26 Introductions00:00:40 Bill’s plethora of job titles00:09:33 The excitement of learning a language00:15:08 Mechanical keyboards00:21:17 Bill’s advice on adapting00:27:55 What a supply chain is00:34:28 Castle analogies00:40:55 Unpacking legalities00:52:11 Log4J00:54:41 What JFrog does01:01:16 What can go wrong01:08:08 Getting started in this space01:14:15 Careers in JFrog01:20:23 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today: https://www.patreon.com/programmingthrowdown?ty=h Subscribe to the podcast on Youtube: https://www.youtube.com/@programmingthrowdown4793Links: Bill Manning: Website: https://about.me/billmanning Linkedin: https://www.linkedin.com/in/williammanning/ Twitter: https://twitter.com/williammanning JFrog: Website: https://jfrog.com/ Careers: https://join.jfrog.com/ Artifactory: https://jfrog.com/artifactory/ Linkedin: https://www.linkedin.com/company/jfrog-ltd/ Others: Liquid Software: https://liquidsoftware.com/ SolarWinds hack incident: https://www.wired.com/story/the-untold-story-of-solarwinds-the-boldest-supply-chain-hack-ever/ Transitive dependencies: https://en.wikipedia.org/wiki/Transitive_dependency More Throwdown? Check out this prior episode:153: ChatGPT: https://www.programmingthrowdown.com/2023/03/153-chatgpt.htmlIf you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/ Reach out to us via email: programmingthrowdown@gmail.com You can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM | Youtube Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★