Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Similar Podcasts
Elixir Outlaws
Elixir Outlaws is an informal discussion about interesting things happening in Elixir. Our goal is to capture the spirit of a conference hallway discussion in a podcast.
The Cynical Developer
A UK based Technology and Software Developer Podcast that helps you to improve your development knowledge and career,
through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer.
Programming Throwdown
Programming Throwdown educates Computer Scientists and Software Engineers on a cavalcade of programming and tech topics. Every show will cover a new programming language, so listeners will be able to speak intelligently about any programming language.
337: Kubernetes on bhyve
Happinesses and stresses of full-time FOSS work, building a FreeBSD fileserver, Kubernetes on FreeBSD bhyve, NetBSD 9 RC1 available, OPNSense 20.1 is here, HardenedBSD’s idealistic future, and more. Headlines The happinesses and stresses of full-time FOSS work (https://drewdevault.com//2020/01/21/Stress-and-happiness.html) In the past few days, several free software maintainers have come out to discuss the stresses of their work. Though the timing was suggestive, my article last week on the philosophy of project governance was, at best, only tangentially related to this topic - I had been working on that article for a while. I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it. February will mark one year that I’ve been working on self-directed free software projects full-time. I was planning on writing an optimistic retrospective article around this time, but given the current mood of the ecosystem I think it would be better to be realistic. In this stage of my career, I now feel at once happier, busier, more fulfilled, more engaged, more stressed, and more depressed than I have at any other point in my life. The good parts are numerous. I’m able to work on my life’s passions, and my projects are in the best shape they’ve ever been thanks to the attention I’m able to pour into them. I’ve also been able to do more thoughtful, careful work; with the extra time I’ve been able to make my software more robust and reliable than it’s ever been. The variety of projects I can invest my time into has also increased substantially, with what was once relegated to minor curiosities now receiving a similar amount of attention as my larger projects were receiving in my spare time before. I can work from anywhere in the world, at any time, not worrying about when to take time off and when to put my head down and crank out a lot of code. The frustrations are numerous, as well. I often feel like I’ve bit off more than I can chew. This has been the default state of affairs for me for a long time; I’m often neglecting half of my projects in order to obtain progress by leaps and bounds in just a few. Working on FOSS full-time has cast this model’s disadvantages into greater relief, as I focus on a greater breadth of projects and spend more time on them. Building a FreeBSD File Server (https://www.vmwareblog.org/building-freebsd-file-server/) Recently at my job, I was faced with a task to develop a file server explicitly suited for the requirements of the company. Needless to say, any configuration of a kind depends on what the infrastructure needs. So, drawing from my personal experience and numerous materials on the web, I came up with the combination FreeBSD+SAMBA+AD as the most appropriate. It appears to be a perfect choice for this environment, and harmonic addition to the existing network configuration since FreeBSD + SAMBA + AD enables admins with the broad range of possibilities for access control. However, as nothing is perfect, this configuration isn’t the best choice if your priority is data protection because it won’t be able to reach the necessary levels of reliability and fault tolerance without outside improvements. Now, since we’ve established that, let’s move on to the next point. This article’s describing the process of building a test environment while concentrating primarily on the details of the configuration. As the author, though, I must say I’m in no way suggesting that this is the only way! The following configuration will be presented in its initial stage, with the minimum requirements necessary to get the job done, and its purpose in one specific situation only. Here, look at this as a useful strategy to solve similar tasks. Well, let’s get started! Report from the first Hamilton BSD Users Group Meeting (https://twitter.com/hambug_ca/status/1227664949914349569) February 11th was the first meeting of this new user group, founded by John Young and myself 11 people attended, and a lot of good discussions were had One of the attendees already owns a domain that fits well for the group, so we will be getting that setup over the next few weeks, as well as the twitter account, and other organization stuff. Special thanks to the illumos users who drove in from Buffalo to attend, although they may have actually had a shorter drive than a few of the other attendees. The next meeting is scheduled again for the 2nd Tuesday of the month, March 10th. We are still discussing if we should meet at a restaurant again, or try to get a space at the local college or innovation hub where we can have a projector etc. News Roundup Kubernetes on FreeBSD Bhyve (https://www.bsdstore.ru/en/articles/cbsd_k8s_part1.html) There are quite a few solutions for container orchestration, but the most popular (or the most famous and highly advertised, is probably, a Kubernetes) Since I plan to conduct many experiments with installing and configuring k8s, I need a laboratory in which I can quickly and easily deploy a cluster in any quantities for myself. In my work and everyday life I use two OS very tightly - Linux and FreeBSD OS. Kubernetes and docker are Linux-centric projects, and at first glance, you should not expect any useful participation and help from FreeBSD here. As the saying goes, an elephant can be made out of a fly, but it will no longer fly. However, two tempting things come to mind - this is very good integration and work in the FreeBSD ZFS file system, from which it would be nice to use the snapshot mechanism, COW and reliability. And the second is the bhyve hypervisor, because we still need the docker and k8s loader in the form of the Linux kernel. Thus, we need to connect a certain number of actions in various ways, most of which are related to starting and pre-configuring virtual machines. This is typical of both a Linux-based server and FreeBSD. What exactly will work under the hood to run virtual machines does not play a big role. And if so - let's take a FreeBSD here! NetBSD 9 RC1 Available (http://blog.netbsd.org/tnf/entry/first_release_candidate_for_netbsd) We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year). Here are a few highlights of the new release: Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA) Enhanced hardware support for Armv7-A Updated GPU drivers (e.g. support for Intel Kabylake) Enhanced virtualization support Support for hardware-accelerated virtualization (NVMM) Support for Performance Monitoring Counters Support for Kernel ASLR Support several kernel sanitizers (KLEAK, KASAN, KUBSAN) Support for userland sanitizers Audit of the network stack Many improvements in NPF Updated ZFS Reworked error handling and NCQ support in the SATA subsystem Support a common framework for USB Ethernet drivers (usbnet) You can download binaries of NetBSD 9.0RC1 from our Fastly-provided CDN: https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0RC1/ OPNsense 20.1 Keen Kingfisher released (https://opnsense.org/opnsense-20-1-keen-kingfisher-released/) For over 5 years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. 20.1, nicknamed "Keen Kingfisher", is a subtle improvement on sustainable firewall experience. This release adds VXLAN and additional loopback device support, IPsec public key authentication and elliptic curve TLS certificate creation amongst others. Third party software has been updated to their latest versions. The logging frontend was rewritten for MVC with seamless API support. On the far side the documentation increased in quality as well as quantity and now presents itself in a familiar menu layout. Idealistic Future for HardenedBSD (https://hardenedbsd.org/article/shawn-webb/2020-01-26/idealistic-future-hardenedbsd) Over the past month, we purchased and deployed the new 13-CURRENT/amd64 package building server. We published our first 13-CURRENT/amd64 production package build using that server. We then rebuilt the old package building server to act as the 12-STABLE/amd64 package building server. This post signifies a very important milestone: we have now fully recovered from last year's death of our infrastructure. Our 12-STABLE/amd64 repo, previously out-of-date by many months, is now fully up-to-date! HardenedBSD is in a very unique position to provide innovative solutions to at-risk and underprivileged populations. As such, we are making human rights endeavors a defining area of focus. Our infrastructure will integrate various privacy and anonymity enhancing technologies and techniques to protect lives. Our operating system's security posture will increase, especially with our focus on exploit mitigations. Navigating the intersection between human rights and information security directly impacts lives. HardenedBSD's 2020 mission and focus is to deliver an entire hardened ecosystem that is unfriendly towards those who would oppress or censor their people. This includes a subtle shift in priorities to match this new mission and focus. While we implement exploit mitigations and further harden the ecosystem, we will seek out opportunities to contribute a tangible and unique impact on human rights issues. Providing Tor Onion Services for our core infrastructure is the first step in likely many to come towards securely helping those in need. Beastie Bits Warner Losh's FOSDEM talk (https://fosdem.org/2020/interviews/warner-losh/) Relational Pipes v0.15 (https://relational-pipes.globalcode.info/v_0/release-v0.15.xhtml) A reminder for where to find NetBSD ARM images (http://www.armbsd.org/arm/) New Safe Memory Reclamation feature in UMA (https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019866.html) BSD Users Stockholm Meetup (https://twitter.com/niclaszeising/status/1216667359831842817) Feedback/Questions ZFS - Rosetta Stone Document? (http://dpaste.com/13EK8YH#wrap) Pat - Question (http://dpaste.com/2DN5RA4#wrap) Sigflup - Wayland on the BSDs (http://dpaste.com/03Y4FQ7#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
336: Archived Knowledge
Linux couldn’t duplicate OpenBSD, FreeBSD Q4 status report, OPNsense 19.7.9 released, archives retain and pass on knowledge, HardenedBSD Tor Onion Service v3 Nodes, and more. Headlines OpenBSD has to be a BSD Unix and you couldn't duplicate it with Linux (https://utcc.utoronto.ca/~cks/space/blog/unix/OpenBSDMustBeABSD?showcomments) OpenBSD has a well deserved reputation for putting security and a clean system (for code, documentation, and so on) first, and everything else second. OpenBSD is of course based on BSD (it's right there in the name) and descends from FreeBSD NetBSD (you can read the history here). But one of the questions you could ask about it is whether it had to be that way, and in particular if you could build something like OpenBSD on top of Linux. I believe that the answer is no. Linux and the *BSDs have a significantly different model of what they are. BSDs have a 'base system' that provides an integrated and fully operational core Unix, covering the kernel, C library and compiler, and the normal Unix user level programs, all maintained and distributed by the particular BSD. Linux is not a single unit this way, and instead all of the component parts are maintained separately and assembled in various ways by various Linux distributions. Both approaches have their advantages, but one big one for the BSD approach is that it enables global changes. Making global changes is an important part of what makes OpenBSD's approach to improving security, code maintenance, and so on work. Because it directly maintains everything as a unit, OpenBSD is in a position to introduce new C library or kernel APIs (or change them) and then immediately update all sorts of things in user level programs to use the new API. This takes a certain amount of work, of course, but it's possible to do it at all. And because OpenBSD can do this sort of ambitious global change, it does. This goes further than just the ability to make global changes, because in theory you can patch in global changes on top of a bunch of separate upstream projects. Because OpenBSD is in control of its entire base system, it's not forced to try to reconcile different development priorities or integrate clashing changes. OpenBSD can decide (and has) that only certain sorts of changes will be accepted into its system at all, no matter what people want. If there are features or entire programs that don't fit into what OpenBSD will accept, they just lose out. FreeBSD Quarterly Status Report 2019Q4 (https://lists.freebsd.org/pipermail/freebsd-announce/2020-January/001923.html) Here is the last quarterly status report for 2019. As you might remember from last report, we changed our timeline: now we collect reports the last month of each quarter and we edit and publish the full document the next month. Thus, we cover here the period October 2019 - December 2019. If you thought that the FreeBSD community was less active in the Christmas' quarter you will be glad to be proven wrong: a quick glance at the summary will be sufficient to see that much work has been done in the last months. Have a nice read! News Roundup OPNsense 19.7.9 released (https://opnsense.org/opnsense-19-7-9-released/) As 20.1 nears we will be making adjustments to the scope of the release with an announcement following shortly. For now, this update brings you a GeoIP database configuration page for aliases which is now required due to upstream database policy changes and a number of prominent third-party software updates we are happy to see included. Archives are important to retain and pass on knowledge (https://dan.langille.org/2020/01/07/archives-are-important-to-retain-and-pass-on-knowledge/) Archives are important. When they are public and available for searching, it retains and passes on knowledge. It saves vast amounts of time. HardenedBSD Tor Onion Service v3 Nodes (https://hardenedbsd.org/article/shawn-webb/2020-01-30/hardenedbsd-tor-onion-service-v3-nodes) I've been working today on deploying Tor Onion Service v3 nodes across our build infrastructure. I'm happy to announce that the public portion of this is now completed. Below you will find various onion service hostnames and their match to our infrastructure. hardenedbsd.org: lkiw4tmbudbr43hbyhm636sarn73vuow77czzohdbqdpjuq3vdzvenyd.onion ci-01.nyi.hardenedbsd.org: qspcqclhifj3tcpojsbwoxgwanlo2wakti2ia4wozxjcldkxmw2yj3yd.onion ci-03.md.hardenedbsd.org: eqvnohly4tjrkpwatdhgptftabpesofirnhz5kq7jzn4zd6ernpvnpqd.onion ci-04.md.hardenedbsd.org: rfqabq2w65nhdkukeqwf27r7h5xfh53h3uns6n74feeyl7s5fbjxczqd.onion git-01.md.hardenedbsd.org: dacxzjk3kq5mmepbdd3ai2ifynlzxsnpl2cnkfhridqfywihrfftapid.onion Beastie Bits The Missing Semester of Your CS Education (MIT Course) (https://missing.csail.mit.edu/) An old Unix Ad (https://i.redd.it/503390rf7md41.png) OpenBSD syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170&w=2) OpenBSD/arm64 on Pinebook (https://twitter.com/bluerise/status/1220963106563579909) Reminder: First Southern Ontario BSD user group meeting, February 11th (this coming Tuesday!) 18:30 at Boston Pizza on Upper James st, Hamilton. (http://studybsd.com/) NYCBUG: March meeting will feature Dr. Paul Vixie and his new talk “Operating Systems as Dumb Pipes” (https://www.nycbug.org/) 8th Meetup of the Stockholm BUG: March 3 at 18:00 (https://www.meetup.com/de-DE/BSD-Users-Stockholm/events/267873938/) Polish BSD User Group meets on Feb 11, 2020 at 18:15 (https://bsd-pl.org/en) Feedback/Questions Sean - ZFS and Creation Dates (http://dpaste.com/3W5WBV0#wrap) Christopher - Help on ZFS Disaster Recovery (http://dpaste.com/3SE43PW) Mike - Encrypted ZFS Send (http://dpaste.com/00J5JZG#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
335: FreeBSD Down Under
Hyperbola Developer interview, why you should migrate from Linux to BSD, FreeBSD is an amazing OS, improving the ptrace(2) API in LLVM 10, First FreeBSD conference in Australia, and a guide to containers on FreeNAS. Headlines FreeBSD is an amazing operating System (https://www.unixsheikh.com/articles/freebsd-is-an-amazing-operating-system.html) Update 2020-01-21: Since I wrote this article it got posted on Hacker News, Reddit and Lobster, and a few people have emailed me with comments. I have updated the article with comments where I have found it needed. As an important side note I would like to point out that I am not a FreeBSD developer, there may be things going on in the FreeBSD world that I know absolutely nothing about. I am also not glued to the FreeBSD developer mailing lists. I am not a FreeBSD "fanboy". I have been using GNU/Linux a ton more for the past two decades than FreeBSD, mainly due to hardware incompatibility (lacking or buggy drivers), and I love both Debian GNU/Linux and Arch Linux just as much as FreeBSD. However, I am concerned about the development of GNU/Linux as of late. Also this article is not about me trying to make anyone switch from something else to FreeBSD. It's about why I like FreeBSD and that I recommend you try it out if you're into messing with operating systems. I think the year was late 1999 or mid 2000 when I one day was browsing computer books at my favorite bookshop and I discovered the book The Complete FreeBSD third edition from 1999 by Greg Lehey. With the book came 4 CD Roms with FreeBSD 3.3. I had already familiarized myself with GNU/Linux in 1998, and I was in the process of migrating every server and desktop operating system away from Microsoft Windows, both at home and at my company, to GNU/Linux, initially Red Hat Linux and then later Debian GNU/Linux, which eventually became my favorite GNU/Linux distribution for many years. When I first saw The Complete FreeBSD book by Greg Lehey I remember noticing the text on the front page that said, "The Free Version of Berkeley UNIX" and "Rock Solid Stability", and I was immediately intrigued! What was that all about? A free UNIX operating system! And rock solid stability? That sounded amazing. Hyperbola Dev Interview (https://itsfoss.com/hyperbola-linux-bsd/) In late December 2019, Hyperbola announced that they would be making major changes to their project. They have decided to drop the Linux kernel in favor of forking the OpenBSD kernel. This announcement only came months after Project Trident announced that they were going in the opposite direction (from BSD to Linux). Hyperbola also plans to replace all software that is not GPL v3 compliant with new versions that are. To get more insight into the future of their new project, I interviewed Andre, co-founder of Hyperbola. News Roundup Improving the ptrace(2) API and preparing for LLVM-10.0 (https://blog.netbsd.org/tnf/entry/improving_the_ptrace_2_api) This month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt. As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0). The first FreeBSD conference in Australia (https://rubenerd.com/the-first-freebsd-conference-in-australia/) FreeBSD has existed as an operating system, project, and foundation for more than twenty years, and its earlier incantations have exited for far longer. The old guard have been developing code, porting software, and writing documentation for longer than I’ve existed. I’ve been using it for more than a decade for personal projects, and professionally for half that time. While there are many prominent Australian FreeBSD contributors, sysadmins, and users, we’ve always had to venture overseas for conferences. We’re always told Australians are among the most ardent travellers, but I always wondered if we could do a domestic event as well. And on Tuesday, we did! Deb Goodkin and the FreeBSD Foundation graciously organised and chaired a dedicated FreeBSD miniconf at the long-running linux.conf.au event held each year in a different city in Australia and New Zealand. A practical guide to containers on FreeNAS for a depraved psychopath (https://medium.com/@andoriyu/a-practical-guide-to-containers-on-freenas-for-a-depraved-psychopath-c212203c0394) This is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11. But muh jails? You know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years… So jails are dead then? No, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD. Why you should migrate everything from Linux to BSD (https://www.unixsheikh.com/articles/why-you-should-migrate-everything-from-linux-to-bsd.html) As an operating system GNU/Linux has become a real mess because of the fragmented nature of the project, the bloatware in the kernel, and because of the jerking around by commercial interests. Response Should you migrate from Linux to BSD? It depends. (https://fediverse.blog/~/AllGoodThings/should-you-migrate-from-linux-to-bsd-it-depends) Beastie Bits Using the OpenBSD ports tree with dedicated users (https://dataswamp.org/~solene/2020-01-11-privsep.html) broot on FreeBSD (https://vermaden.wordpress.com/2020/01/10/run-broot-on-freebsd/) A Trip down Memory Lane (https://svnweb.freebsd.org/base/head/share/misc/bsd-family-tree?view=co) Running syslog-ng in BastilleBSD (https://www.syslog-ng.com/community/b/blog/posts/running-syslog-ng-in-bastillebsd) NASA : Using Software Packages in pkgsrc (https://www.nas.nasa.gov/hecc/support/kb/using-software-packages-in-pkgsrc_493.html) Feedback/Questions All of our questions this week were pretty technical in nature so I'm going to save those for the next episode so Allan can weigh in on them, since if we cover them now we're basically going to be deferring to Allan anyway. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
334: Distrowatch Running FreeBSD
Upgrading FreeBSD from 11.3 to 12.1, Distrowatch switching to FreeBSD, Torvalds says don’t run ZFS, iked(8) removed automatic IPv6 blocking, working towards LLDB on i386, and memory-hard Argon2 hashing scheme in NetBSD. Headlines Upgrading FreeBSD from 11.3 to 12.1 (https://blog.bimajority.org/2020/01/13/upgrading-freebsd-from-11-3-to-12-1/) Now here’s something more like what I was originally expecting the content on this blog to look like. I’m in the process of moving all of our FreeBSD servers (about 30 in total) from 11.3 to 12.1. We have our own local build of the OS, and until “packaged base” gets to a state where it’s reliably usable, we’re stuck doing upgrades the old-fashioned way. I created a set of notes for myself while cranking through these upgrades and I wanted to share them since they are not really work-specific and this process isn’t very well documented for people who haven’t been doing this sort of upgrade process for 25 years. Our source and object trees are read-only exported from the build server over NFS, which causes things to be slow. /etc/make.conf and /etc/src.conf are symbolic links on all of our servers to the master copies in /usr/src so that make installworld can find the configuration parameters the system was built with. Switching Distrowatch over to BSD (https://www.reddit.com/r/freebsd/comments/eodhit/switching_distrowatch_over_to_freebsd_ama/) This may be a little off-topic for this board (forgive me if it is, please). However, I wanted to say that I'm one of the people who works on DistroWatch (distrowatch.com) and this past week we had to deal with a server facing hardware failure. We had a discussion about whether to continue running Debian or switch to something else. The primary "something else" option turned out to be FreeBSD and it is what we eventually went with. It took a while to convert everything over from working with Debian GNU/Linux to FreeBSD 12 (some script incompatibilities, different paths, some changes to web server configuration, networking IPv6 troubles). But in the end we ended up with a good, FreeBSD-based experience. Since the transition was successful, though certainly not seamless, I thought people might want to do a Q&A on the migration process. Especially for those thinking of making the same switch. News Roundup iked(8) automatic IPv6 blocking removed (https://www.openbsd.org/faq/current.html#r20200114) iked(8) no longer automatically blocks unencrypted outbound IPv6 packets. This feature was intended to avoid accidental leakage, but in practice was found to mostly be a cause of misconfiguration. If you previously used iked(8)'s -6 flag to disable this feature, it is no longer needed and should be removed from /etc/rc.conf.local if used. Linus says dont run ZFS (https://itsfoss.com/linus-torvalds-zfs/) “Don’t use ZFS. It’s that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.” This is what Linus Torvalds said in a mailing list to once again express his disliking for ZFS filesystem specially over its licensing. To avoid unnecessary confusion, this is more intended for Linux distributions, kernel developers and maintainers rather than individual Linux users. GSoC 2019 Final Report: Incorporating the memory-hard Argon2 hashing scheme into NetBSD (https://blog.netbsd.org/tnf/entry/gsoc_2019_final_report_incorporating) We successfully incorporated the Argon2 reference implementation into NetBSD/amd64 for our 2019 Google Summer of Coding project. We introduced our project here and provided some hints on how to select parameters here. For our final report, we will provide an overview of what changes were made to complete the project. The Argon2 reference implementation, available here, is available under both the Creative Commons CC0 1.0 and the Apache Public License 2.0. To import the reference implementation into src/external, we chose to use the Apache 2.0 license for this project. Working towards LLDB on i386 NetBSD (https://blog.netbsd.org/tnf/entry/working_towards_lldb_on_i386) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support. Throughout December I've continued working on our build bot maintenance, in particular enabling compiler-rt tests. I've revived and finished my old patch for extended register state (XState) in core dumps. I've started working on bringing proper i386 support to LLDB. Beastie Bits An open source Civilization V (https://github.com/yairm210/UnCiv) BSD Groups in Italy (https://bsdnotizie.blogspot.com/2020/01/gruppi-bsd-in-italia.html) Why is Wednesday, November 17, 1858 the base time for OpenVMS? (https://www.slac.stanford.edu/~rkj/crazytime.txt) Benchmarking shell pipelines and the Unix “tools” philosophy (https://blog.plover.com/Unix/tools.html) LPI and BSD working together (https://youtu.be/QItb5aoj7Oc) Feedback/Questions Pat - March Meeting (http://dpaste.com/2BMGZVV#wrap) Madhukar - Overheating Laptop (http://dpaste.com/17WNVM8#wrap) Warren - R vs S (http://dpaste.com/3AZYFB1#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
333: Unix Keyboard Joy
Your Impact on FreeBSD in 2019, Wireguard on OpenBSD Router, Amazon now has FreeBSD/ARM 12, pkgsrc-2019Q4, The Joys of UNIX Keyboards, OpenBSD on Digital Ocean, and more. Headlines Your Impact on FreeBSD in 2019 (https://www.freebsdfoundation.org/blog/your-impact-on-freebsd-in-2019/) It’s hard to believe that 2019 is nearly over. It has been an amazing year for supporting the FreeBSD Project and community! Why do I say that? Because as I reflect over the past 12 months, I realize how many events we’ve attended all over the world, and how many lives we’ve touched in so many ways. From advocating for FreeBSD to implementing FreeBSD features, my team has been there to help make FreeBSD the best open source project and operating system out there. In 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we’ve stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we’re on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams. Our advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees. Our travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD user/contributors. We’ve seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund. Wireguard on OpenBSD Router (https://obscurity.xyz/bsd/open/wireguard.html) wireguard (wg) is a modern vpn protocol, using the latest class of encryption algorithms while at the same time promising speed and a small code base. modern crypto and lean code are also tenants of openbsd, thus it was a no brainer to migrate my router from openvpn over to wireguard. my setup : a collection of devices, both wired and wireless, that are nat’d through my router (openbsd 6.6) out via my vpn provider azire* and out to the internet using wg-quick to start wg. running : doubtless this could be improved on, but currently i start wg manually when my router boots. this, and the nat'ing on the vpn interface mean its impossible for clients to connect to the internet without the vpn being up. as my router is on a ups and only reboots when a kernel patch requires it, it’s a compromise i can live with. run wg-quick (please replace vpn with whatever you named your wg .conf file.) and reload pf rules. News Roundup Amazon now has FreeBSD/ARM 12 (https://aws.amazon.com/marketplace/pp/B081NF7BY7) AWS, the cloud division of Amazon, announced in December the next generation of its ARM processors, the Graviton2. This is a custom chip design with a 7nm architecture. It is based on 64-bit ARM Neoverse cores. Compared to first-generation Graviton processors (A1), today’s new chips should deliver up to 7x the performance of A1 instances in some cases. Floating point performance is now twice as fast. There are additional memory channels and cache speed memory access should be much faster. The company is working on three types of Graviton2 EC2 instances that should be available soon. Instances with a “g” suffix are powered by Graviton2 chips. If they have a “d” suffix, it also means that they have NVMe local storage. General-purpose instances (M6g and M6gd) Compute-optimized instances (C6g and C6gd) Memory-optimized instances (R6g and R6gd) You can choose instances with up to 64 vCPUs, 512 GiB of memory and 25 Gbps networking. And you can see that ARM-powered servers are not just a fad. AWS already promises a 40% better price/performance ratio with ARM-based instances when you compare them with x86-based instances. AWS has been working with operating system vendors and independent software vendors to help them release software that runs on ARM. ARM-based EC2 instances support Amazon Linux 2, Ubuntu, Red Hat, SUSE, Fedora, Debian and FreeBSD. It also works with multiple container services (Docker, Amazon ECS, and Amazon Elastic Kubernetes Service). Coverage of AWS Announcement (https://techcrunch.com/2019/12/03/aws-announces-new-arm-based-instances-with-graviton2-processors/) Announcing the pkgsrc-2019Q4 release (https://mail-index.netbsd.org/pkgsrc-users/2020/01/06/msg030130.html) The pkgsrc developers are proud to announce the 65th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 20,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/ In total, 190 packages were added, 96 packages were removed, and 1,868 package updates (to 1388 unique packages) were processed since the pkgsrc-2019Q3 release. As usual, a large number of updates and additions were processed for packages for go (14), guile (11), perl (170), php (10), python (426), and ruby (110). This continues pkgsrc's tradition of adding useful packages, updating many packages to more current versions, and pruning unmaintained packages that are believed to have essentially no users. The Joys of UNIX Keyboards (https://donatstudios.com/UNIX-Keyboards) I fell in love with a dead keyboard layout. A decade or so ago while helping a friends father clean out an old building, we came across an ancient Sun Microsystems server. We found it curious. Everything about it was different from what we were used to. The command line was black on white, the connectors strange and foreign, and the keyboard layout was bizarre. We never did much with it; turning it on made all the lights in his home dim, and our joint knowledge of UNIX was nonexistent. It sat in his bedroom for years supporting his television at the foot of his bed. I never forgot that keyboard though. The thought that there was this alternative layout out there seemed intriguing to me. OpenBSD on Digital Ocean (https://www.going-flying.com/blog/openbsd-on-digitalocean.html) Last night I had a need to put together a new OpenBSD machine. Since I already use DigitalOcean for one of my public DNS servers I wanted to use them for this need but sadly like all too many of the cloud providers they don't support OpenBSD. Now they do support FreeBSD and I found a couple writeups that show how to use FreeBSD as a shim to install OpenBSD. They are both sort of old at this point and with OpenBSD 6.6 out I ran into a bit of a snag. The default these days is to use a GPT partition table to enable EFI booting. This is generally pretty sane but it looks to me like the FreeBSD droplet doesn't support this. After the installer rebooted the VM failed to boot, being unable to find the bootloader. Thankfully DigitalOcean has a recovery ISO that you can boot by simply switching to it and powering off and then on your Droplet. Beastie Bits FreeBSD defaults to LLVM on PPC (https://svnweb.freebsd.org/base?view=revision&revision=356111) Theo De Raadt Interview between Ottawa 2019 Hackathon and BSDCAN 2019 (https://undeadly.org/cgi?action=article;sid=20191231214356) Bastille Poll about what people would like to see in 2020 (https://twitter.com/BastilleBSD/status/1211475103143251968) Notes on the classic book : The Design of the UNIX Operating System (https://github.com/suvratapte/Maurice-Bach-Notes) Multics History (https://www.multicians.org/) First meeting of the Hamilton BSD user group, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St (http://studybsd.com/) Feedback/Questions Bill - 1.1 CDROM (http://dpaste.com/2H9CW6R) Greg - More 50 Year anniversary information (http://dpaste.com/2SGA3KY) Dave - Question time for Allan (http://dpaste.com/3ZAEKHD#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
332: The BSD Hyperbole
Announcing HyperbolaBSD, IPFW In-Kernel NAT setup on FreeBSD, Wayland and WebRTC enabled for NetBSD 9/Linux, LLDB Threading support ready for mainline, OpenSSH U2F/FIDO support in base, Dragonfly drm/i915: Update, and more. Headlines HyperbolaBSD Announcement (https://www.hyperbola.info/news/announcing-hyperbolabsd-roadmap/) Due to the Linux kernel rapidly proceeding down an unstable path, we are planning on implementing a completely new OS derived from several BSD implementations. This was not an easy decision to make, but we wish to use our time and resources to create a viable alternative to the current operating system trends which are actively seeking to undermine user choice and freedom. This will not be a "distro", but a hard fork of the OpenBSD kernel and userspace including new code written under GPLv3 and LGPLv3 to replace GPL-incompatible parts and non-free ones. Reasons for this include: Linux kernel forcing adaption of DRM, including HDCP. Linux kernel proposed usage of Rust (which contains freedom flaws and a centralized code repository that is more prone to cyber attack and generally requires internet access to use.) Linux kernel being written without security and in mind. (KSPP is basically a dead project and Grsec is no longer free software) Many GNU userspace and core utils are all forcing adaption of features without build time options to disable them. E.g. (PulseAudio / SystemD / Rust / Java as forced dependencies) As such, we will continue to support the Milky Way branch until 2022 when our legacy Linux-libre kernel reaches End of Life. Future versions of Hyperbola will be using HyperbolaBSD which will have the new kernel, userspace and not be ABI compatible with previous versions. HyperbolaBSD is intended to be modular and minimalist so other projects will be able to re-use the code under free license. Forum Post (https://forums.hyperbola.info/viewtopic.php?id=315) A simple IPFW In-Kernel NAT setup on FreeBSD (https://www.neelc.org/posts/freebsd-ipfw-nat/) After graduating college, I am moving from Brooklyn, NY to Redmond, WA (guess where I got a job). I always wanted to re-do my OPNsense firewall (currently a HP T730) with stock FreeBSD and IPFW’s in-kernel NAT. Why IPFW? Benchmarks have shown IPFW to be faster which is especially good for my Tor relay, and because I can! However, one downside of IPFW is less documentation vs PF, even less without natd (which we’re not using), and this took me time to figure this out. But since my T730 is already packed, I am testing this on a old PC with two NICs, and my laptop [1] as a client with an USB-to-Ethernet adapter. News Roundup HEADS UP: Wayland and WebRTC enabled for NetBSD 9/Linux (https://mail-index.netbsd.org/pkgsrc-users/2020/01/05/msg030124.html) This is just a heads up that the Wayland option is now turned on by default for NetBSD 9 and Linux in cases where it peacefully coexists with X11. Right now, this effects the following packages: graphics/MesaLib devel/SDL2 www/webkit-gtk x11/gtk3 The WebRTC option has also been enabled by default on NetBSD 9 for two Firefox versions: www/firefox, www/firefox68 Please keep me informed of any fallout. Hopefully, there will be none. If you want to try out Wayland-related things on NetBSD 9, wm/velox/MESSAGE may be interesting for you. LLDB Threading support now ready for mainline (https://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. OpenSSH U2F/FIDO support in base (https://www.undeadly.org/cgi?action=article;sid=20191115064850) Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step. You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time. So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. drm/i915: Update to Linux 4.8.17 (http://lists.dragonflybsd.org/pipermail/commits/2019-December/720257.html) drm/i915: Update to Linux 4.8.17 Broxton, Valleyview and Cherryview support improvements Broadwell and Gen9/Skylake support improvements Broadwell brightness fixes from OpenBSD Atomic modesetting improvements Various bug fixes and performance enhancements Beastie Bits Visual Studio Code port for FreeBSD (https://github.com/tagattie/FreeBSD-VSCode) OpenBSD syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170&w=2) Peertube on OpenBSD (https://www.22decembre.eu/en/2019/12/09/peertube-14-openbsd/) Fuzzing Filesystems on NetBSD via AFL+KCOV by Maciej Grochowski (https://www.youtube.com/watch?v=bbNCqFdQEyk&feature=youtu.be) Twitter Bot for Prop65 (https://twitter.com/prop65bot/status/1199003319307558912) Interactive vim tutorial (https://www.openvim.com/) First BSD user group meeting in Hamilton, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St (http://studybsd.com/) *** Feedback/Questions Samir - cgit (http://dpaste.com/2B22M24#wrap) Russell - R (http://dpaste.com/0J5TYY0#wrap) Wolfgang - Question (http://dpaste.com/3MQAH27#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
331: Why Computers Suck
How learning OpenBSD makes computers suck a little less, How Unix works, FreeBSD 12.1 Runs Well on Ryzen Threadripper 3970X, BSDCan CFP, HardenedBSD Infrastructure Goals, and more. Headlines Why computers suck and how learning from OpenBSD can make them marginally less horrible (https://telegra.ph/Why-OpenBSD-is-marginally-less-horrible-12-05) How much better could things actually be if we abandoned the enterprise development model? Next I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase. One of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD. To prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively. How Unix Works: Become a Better Software Engineer (https://neilkakkar.com/unix.html) Unix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system. Every fancy thing you want done is one google search away. But understanding why the solution does what you want is not the same. That’s what gives you real power, the power to not be afraid. And since it rhymes, it must be true. News Roundup FreeBSD 12.1 Runs Refreshingly Well With AMD Ryzen Threadripper 3970X (https://www.phoronix.com/scan.php?page=article&item=freebsd-amd-3970x&num=1) For those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform. It was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround. While the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1. FreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system. BSDCan 2020 CFP (https://lists.bsdcan.org/pipermail/bsdcan-announce/2019-December/000180.html) BSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu). NOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June. We are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue. See http://www.bsdcan.org/2020/ If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include: How we manage a giant installation with respect to handling spam. and/or sysadmin. and/or networking. Cool new stuff in BSD Tell us about your project which runs on BSD other topics (see next paragraph) From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples. Both users and developers are encouraged to share their experiences. HardenedBSD Infrastructure Goals (https://github.com/lattera/articles/blob/master/hardenedbsd/2019-12-01_infrastructure/article.md) 2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion. The last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity. My available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them. My primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub. As I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor. Transparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor. Integrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published. I hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired. So, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows. Beastie Bits FuryBSD - KDE plasma flavor now available (https://www.furybsd.org/kde-plasma-flavor-now-available/) DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud (http://lists.dragonflybsd.org/pipermail/commits/2019-November/719945.html) LPI is looking for BSD Specialist learning material writers (https://wiki.lpi.org/wiki/BSD_Specialist_Objectives_V1.0) ZFS sync/async + ZIL/SLOG, explained (https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/) BSD-Licensed Combinatorics library/utility (https://lists.freebsd.org/pipermail/freebsd-announce/2019-December/001921.html) SSL client vs server certificates and bacula-fd (https://dan.langille.org/2019/11/29/ssl-client-vs-server-certificates-and-bacula-fd/) MaxxDesktop planning to come to FreeBSD (https://www.facebook.com/maxxdesktop/posts/2761326693888282) Project Page (https://www.facebook.com/maxxdesktop/) Feedback/Questions Tom - ZFS Mirror with different speeds (http://dpaste.com/3ZGYNS3#wrap) Jeff - Knowledge is power (http://dpaste.com/1H9QDCR#wrap) Johnny - Episode 324 response to Jacob (http://dpaste.com/1A7Q9EV) Pat - NYC*BUG meeting Jan Meeting Location (http://dpaste.com/0QPZ2GC) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
330: Happy Holidays, All(an)
Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more. Headlines Authentication vulnerabilities in OpenBSD (https://www.openwall.com/lists/oss-security/2019/12/04/5) We discovered an authentication-bypass vulnerability in OpenBSD's authentication system: this vulnerability is remotely exploitable in smtpd, ldapd, and radiusd, but its real-world impact should be studied on a case-by-case basis. For example, sshd is not exploitable thanks to its defense-in-depth mechanisms. From the manual page of login.conf: OpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are: passwd Request a password and check it against the password in the master.passwd file. See loginpasswd(8). skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See loginskey(8). yubikey Authenticate using a Yubico YubiKey token. See loginyubikey(8). For any given style, the program /usr/libexec/auth/loginstyle is used to perform the authentication. The synopsis of this program is: /usr/libexec/auth/login_style [-v name=value] [-s service] username class This is the first piece of the puzzle: if an attacker specifies a username of the form "-option", they can influence the behavior of the authentication program in unexpected ways. login_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program. The allowed protocols are login, challenge, and response. (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based). This is the second piece of the puzzle: if an attacker specifies the username "-schallenge" (or "-schallenge:passwd" to force a passwd-style authentication), then the authentication is automatically successful and therefore bypassed. Case study: smtpd Case study: ldapd Case study: radiusd Case study: sshd Acknowledgments: We thank Theo de Raadt and the OpenBSD developers for their incredibly quick response: they published patches for these vulnerabilities less than 40 hours after our initial contact. We also thank MITRE's CVE Assignment Team. First release candidate for NetBSD 9.0 available! (https://blog.netbsd.org/tnf/entry/first_release_candidate_for_netbsd) Since the start of the release process four months ago a lot of improvements went into the branch - more than 500 pullups were processed! This includes usbnet (a common framework for usb ethernet drivers), aarch64 stability enhancements and lots of new hardware support, installer/sysinst fixes and changes to the NVMM (hardware virtualization) interface. We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year). Here are a few highlights of the new release: Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA) Enhanced hardware support for Armv7-A Updated GPU drivers (e.g. support for Intel Kabylake) Enhanced virtualization support Support for hardware-accelerated virtualization (NVMM) Support for Performance Monitoring Counters Support for Kernel ASLR Support several kernel sanitizers (KLEAK, KASAN, KUBSAN) Support for userland sanitizers Audit of the network stack Many improvements in NPF Updated ZFS Reworked error handling and NCQ support in the SATA subsystem Support a common framework for USB Ethernet drivers (usbnet) More information on the RC can be found on the NetBSD 9 release page (https://www.netbsd.org/releases/formal-9/NetBSD-9.0.html) News Roundup Running FreeNAS on a Digitalocean droplet (https://www.shlomimarco.com/post/running-freenas-on-a-digitalocean-droplet) ZFS is awesome. FreeBSD even more so. FreeNAS is the battle-tested, enterprise-ready-yet-home-user-friendly software defined storage solution which is cooler then deep space, based on FreeBSD and makes heavy use of ZFS. This is what I (and soooooo many others) use for just about any storage-related task. I can go on and on and on about what makes it great, but if you're here, reading this, you probably know all that already and we can skip ahead. I've needed an offsite FreeNAS setup to replicate things to, to run some things, to do some stuff, basically, my privately-owned, tightly-controlled NAS appliance in the cloud, one I control from top to bottom and with support for whatever crazy thing I'm trying to do. Since I'm using DigitalOcean as my main VPS provider, it seemed logical to run FreeNAS there, however, you can't. While DO supports many many distos and pre-setup applications (e.g OpenVPN), FreeNAS isn't a supported feature, at least not in the traditional way :) Before we begin, here's the gist of what we're going to do: Base of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device. Part 1: re-image our boot block device to boot FreeNAS install media. Part 2: Install FreeNAS on the second block-device Part 3: Re-image the boot block device using the FreeNAS-installed block device NomadBSD 1.3 is now available (https://nomadbsd.org/) From the release notes: The base system has been changed to FreeBSD 12.1-RELEASE-p1 Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse The GPT layout has been changed to MBR. This prevents problems with Lenovo systems that refuse to boot from GPT if "lenovofix" is not set, and systems that hang on boot if "lenovofix" is set. Support for ZFS installations has been added to the NomadBSD installer. The rc-script for setting up the network interfaces has been fixed and improved. Support for setting the country code for the wlan device has been added. Auto configuration for running in VirtualBox has been added. A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead. NVIDIA driver version 440 has been added. nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the default user and autologin has been added. nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added. Martin Orszulik added Czech translations to the setup and installation wizard. The NomadBSD logo, designed by Ian Grindley, has been changed. Support for localized error messages has been added. Support for localizing the password prompts has been added. Some templates for starting other DEs have been added to ~/.xinitrc. The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved. A script that helps users to configure a multihead systems has been added. The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting". /proc has been added to /etc/fstab A D-Bus session issue has been fixed which prevented thunar from accessing samba shares. DSBBg which allows users to change and manage wallpapers has been added. The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed. Support for multiple keyboard layouts has been added. www/palemoon has been removed. mail/thunderbird has been removed. audio/audacity has been added. deskutils/orage has been added. the password manager fpm2 has been replaced by KeePassXC mail/sylpheed has been replaced by mail/claws-mail multimedia/simplescreenrecorder has been added. DSBMC has been changed to DSBMC-Qt Many small improvements and bug fixes. At e2k19 nobody can hear you scream (https://undeadly.org/cgi?action=article;sid=20191204170908) After 2 years it was once again time to pack skis and snowshoes, put a satellite dish onto a sledge and hike through the snowy rockies to the Elk Lakes hut. I did not really have much of a plan what I wanted to work on but there were a few things I wanted to look into. One of them was rpki-client and the fact that it was so incredibly slow. Since Bob beck@ was around I started to ask him innocent X509 questions ... as if there are innocent X509 questions! Mainly about the abuse of the X509STORE in rpki-client. Pretty soon it was clear that rpki-client did it all wrong and most of the X509 verification had to be rewritten. Instead of only storing the root certificates in the store and passing the intermediate certs as a chain to the verification function rpki-client threw everything into it. The X509STORE is just not built for such an abuse and so it was no wonder that this was slow. Lucky me I pulled benno@ with me into this dark hole of libcrypto code. He managed to build up an initial diff to pass the chains as a STACKOF(X509) and together we managed to get it working. A big thanks goes to ingo@ who documented most of the functions we had to use. Have a look at STACKOF(3) and skpopfree(3) to understand why benno@ and I slowly turned crazy. Our next challenge was to only load the necessary certificate revocation list into the X509STORECTX. While doing those changes it became obvious that some of the data structures needed better lookup functions. Looking up certificates was done using a linear lookup and so we replaced the internal certificate and CRL tables with RB trees for fast lookups. deraadt@ also joined the rpki-client commit fest and changed the output code to use rename(2) so that files are replaced in an atomic operation. Thanks to this rpki-client can now be safely run from cron (there is an example in the default crontab). I did not plan to spend most of my week hacking on rpki-client but in the end I'm happy that I did and the result is fairly impressive. Working with libcrypto code and especially X509 was less than pleasant. Our screams of agony died away in the snowy rocky mountains and made Bob deep dive into UVM with a smile since he knew that benno@ and I had it worse. In case you wonder thanks to all changes at e2k19 rpki-client improved from over 20min run time to validate all VRPS to roughly 1min to do the same job. A factor 20 improvement! Thanks to Theo, Bob and Howie to make this possible. To all the cooks for the great food and to Xplornet for providing us with Internet at the hut. Beastie Bits FOSDEM 2020 BSD Devroom schedule (https://fosdem.org/2020/schedule/track/bsd/) Easy Minecraft Server on FreeBSD Howto (https://www.freebsdfoundation.org/freebsd/how-to-guides/easy-minecraft-server-on-freebsd/) stats(3) framework in the TCP stack (https://svnweb.freebsd.org/base?view=revision&revision=355304) 4017 days of uptime (https://twitter.com/EdwinKremer/status/1203071684535889921) sysget - A front-end for every package manager (https://github.com/emilengler/sysget) PlayOnBSD’s Cross-BSD Shopping Guide (https://www.playonbsd.com/shopping_guide/) Feedback/Questions Pat asks about the proper disk drive type for ZFS (http://dpaste.com/2FDN26X#wrap) Brad asks about a ZFS rosetta stone (http://dpaste.com/2X8PBMC#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Mariusz Zaborski.
329: Lucas’ Arts
In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book. Interview - Michael Lucas Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Michael W Lucas.
328: EPYC Netflix Stack
LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more. Headlines LLDB Threading support now ready for mainline (https://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Multiple IPSec VPN tunnels with FreeBSD (https://blog.socruel.nu/text-only/how-to-multiple-ipsec-vpn-tunnels-on-freebsd.txt) The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html) But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below. The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE). Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C). VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C). News Roundup Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance (https://www.phoronix.com/scan.php?page=news_item&px=Netflix-NUMA-FreeBSD-Optimized) Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel. Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made. For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%. unwind(8); "happy eyeballs" (https://marc.info/?l=openbsd-tech&m=157475113130337&w=2) In case you are wondering why happy eyeballs: It's a variation on this: https://en.wikipedia.org/wiki/Happy_Eyeballs unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers. This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix. One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E): 17 files changed, 385 insertions(+), 1683 deletions(-) Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals. Amazon now has FreeBSD ARM 12 (https://aws.amazon.com/marketplace/pp/B081NF7BY7) Product Overview FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years. FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems. OpenSSH U2F/FIDO support in base (https://www.undeadly.org/cgi?action=article;sid=20191115064850) I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys. Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step. You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time. So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too. Please test this thoroughly - it's a big change that we want to have stable before the next release. Beastie Bits DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud (http://lists.dragonflybsd.org/pipermail/commits/2019-November/719945.html) Really fast Markov chains in ~20 lines of sh, grep, cut and awk (https://0x0f0f0f.github.io/posts/2019/11/really-fast-markov-chains-in-~20-lines-of-sh-grep-cut-and-awk/) FreeBSD Journal Sept/Oct 2019 (https://www.freebsdfoundation.org/past-issues/security-3/) Michael Dexter is raising money for Bhyve development (https://twitter.com/michaeldexter/status/1201231729228308480) syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170) FreeBSD Forums Howto Section (https://forums.freebsd.org/forums/howtos-and-faqs-moderated.39/) Feedback/Questions Jeroen - Feedback (http://dpaste.com/0PK1EG2#wrap) Savo - pfsense ports (http://dpaste.com/0PZ03B7#wrap) Tin - I want to learn C (http://dpaste.com/2GVNCYB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
327: ZFS Rename Repo
We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more. Headlines FreeBSD third quarterly status report for 2019 (https://www.freebsd.org/news/status/report-2019-07-2019-09.html) This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work). Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner. Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October. Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January. OpenBSD on Sparc64 (https://eerielinux.wordpress.com/2019/10/10/openbsd-on-sparc64-6-0-to-6-5/) OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available. First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release. Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions. News Roundup ZoL repo move to OpenZFS (https://zfsonlinux.topicbox.com/groups/zfs-discuss/T13eedc32607dab41/zol-repo-move-to-openzfs) Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs. The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020. Mcclure111 Sun Thread (https://twitter.com/mcclure111/status/1196557401710837762) A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback. GEOM NOP (https://oshogbo.vexillium.org/blog/71/) Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device. GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks. Keeping NetBSD up-to-date with pkg_comp 2.0 (https://jmmv.dev/2017/02/pkg_comp-2.0-tutorial-netbsd.html) This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure. This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform. Beastie Bits DragonFly - Radeon Improvements (http://lists.dragonflybsd.org/pipermail/commits/2019-November/720070.html) NomadBSD review (https://www.youtube.com/watch?v=7DglP7SbnlA&feature=share) Spongebob OpenBSD Security Comic (https://files.yukiisbo.red/openbsd_claim.png) Forth : The Early Years (https://colorforth.github.io/HOPL.html) LCM+L PDP-7 booting and running UNIX Version 0 (https://www.youtube.com/watch?v=pvaPaWyiuLA) Feedback/Questions Chris - Ctrl-T (http://dpaste.com/284E5BV) Improved Ctrl+t that shows kernel backtrace (https://asciinema.org/a/xfSpvPT61Cnd9iRgbfIjT6kYj) Brian - Migrating NexentaStore to FreeBSD/FreeNAS (http://dpaste.com/05GDK8H#wrap) Avery - How to get involved (http://dpaste.com/26KW801#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
326: Certified BSD
LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more. Headlines Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group (https://www.lpi.org/articles/linux-professional-institute-releases-bsd-specialist-certification) Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018. G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.” OpenZFS Trip Report (https://www.ixsystems.com/blog/openzfs-dev-summit-2019/) The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling. News Roundup Using FreeBSD with Ports (2/2): Tool-assisted updating (https://eerielinux.wordpress.com/2019/09/12/using-freebsd-with-ports-2-2-tool-assisted-updating/) Part 1 here: https://eerielinux.wordpress.com/2019/08/18/using-freebsd-with-ports-1-2-classic-way-with-tools/ In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient. In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today. LLDB Threading support now ready for mainline (http://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Linux VS open source UNIX (https://www.adminbyaccident.com/politics/linux-vs-open-source-unix/) Beastie Bits Support for Realtek RTL8125 2.5Gb Ethernet controller (https://marc.info/?l=openbsd-tech&m=157380442230074&w=2) Computer Files Are Going Extinct (https://onezero.medium.com/the-death-of-the-computer-file-doc-43cb028c0506) FreeBSD kernel hacking (https://www.youtube.com/watch?v=4FUub_UtF3c) Modern BSD Computing for Fun on a VAX! Trying to use a VAX in today's world by Jeff Armstrong (https://youtu.be/e7cJ7v2lYdE) MidnightBSD 1.2 Released (https://www.justjournal.com/users/mbsd/entry/33779) Feedback/Questions Paulo - Zfs snapshots (http://dpaste.com/0WQRP43#wrap) Phillip - GCP (http://dpaste.com/075ZQE1#wrap) A Listener - Old episodes? (http://dpaste.com/3YJ4119#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
325: Cracking Rainbows
FreeBSD 12.1 is here, A history of Unix before Berkeley, FreeBSD development setup, HardenedBSD 2019 Status Report, DNSSEC, compiling RainbowCrack on OpenBSD, and more. Headlines FreeBSD 12.1 (https://www.freebsd.org/releases/12.1R/announce.html) Some of the highlights: BearSSL has been imported to the base system. The clang, llvm, lld, lldb, compiler-rt utilities and libc++ have been updated to version 8.0.1. OpenSSL has been updated to version 1.1.1d. Several userland utility updates. For a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html A History of UNIX before Berkeley: UNIX Evolution: 1975-1984. (http://www.darwinsys.com/history/hist.html) Nobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today. Our title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question. Part One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at Research'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as nowthe Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time. News Roundup My FreeBSD Development Setup (https://adventurist.me/posts/00296) I do my FreeBSD development using git, tmux, vim and cscope. I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd OPNsense 19.7.6 released (https://opnsense.org/opnsense-19-7-6-released/) As we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available. LDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis. HardenedBSD November 2019 Status Report. (https://hardenedbsd.org/article/shawn-webb/2019-11-09/hardenedbsd-status-report) We at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work. DNSSEC enabled in default unbound(8) configuration. (https://undeadly.org/cgi?action=article;sid=20191110123908) DNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@) How to Install Shopware with NGINX and Let's Encrypt on FreeBSD 12 (https://www.howtoforge.com/how-to-install-shopware-with-nginx-and-lets-encrypt-on-freebsd-12/) Shopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server. Requirements Make sure your system meets the following minimum requirements: + Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed. + PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended. + MySQL 5.5.0 or higher. + Possibility to set up cron jobs. + Minimum 4 GB available hard disk space. + IonCube Loader version 5.0.0 or higher (optional). How to Compile RainbowCrack on OpenBSD (https://cromwell-intl.com/open-source/compiling-rainbowcrack-on-openbsd.html) Project RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still just his or if it's even been maintained for a while. His page seems to have been last updated in August 2007. The Project RainbowCrack web page now has just binaries for Windows XP and Linux, both 32-bit and 64-bit versions. Earlier versions were available as source code. The version 1.2 source code does not compile on OpenBSD, and in my experience it doesn't compile on Linux, either. It seems to date from 2004 at the earliest, and I think it makes some version-2.4 assumptions about Linux kernel headers. You might also look at ophcrack, a more modern tool, although it seems to be focused on cracking Windows XP/Vista/7/8/10 password hashes Feedback/Questions Reese - Amature radio info (http://dpaste.com/2RDG9K4#wrap) Chris - VPN (http://dpaste.com/2K4T2FQ#wrap) Malcolm - NAT (http://dpaste.com/138NEMA) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
324: Emergency Space Mode
Migrating drives and zpool between hosts, OpenBSD in 2019, Dragonfly’s new zlib and dhcpcd, Batch renaming images and resolution with awk, a rant on the X11 ICCCM selection system, hammer 2 emergency space mode, and more. Headlines Migrating drives and the zpool from one host to another. (https://dan.langille.org/2019/10/26/migrating-drives-and-the-zpool-from-one-host-to-another/) Today is the day. Today I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host. Fortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail. Now it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up. In this post: FreeBSD 12.0 Dell R710 (r710-01) Dell R720 (r720-01) drive caddies from eBay and now I know the difference between SATA and SATAu PLEASE READ THIS first: Migrating ZFS Storage Pools (https://docs.oracle.com/cd/E19253-01/819-5461/gbchy/index.html) OpenBSD in 2019 (https://blog.habets.se/2019/10/OpenBSD-in-2019.html) I’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again. What triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine. I don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux. That and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things. This is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem. Verdict Ouch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD. And maybe in a year I’ll have a review of OpenBSD on a laptop. News Roundup New zlib, new dhcpcd (https://www.dragonflydigest.com/2019/10/29/23683.html) zlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed. DHCPCD Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719768.html ZLIB Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719772.html Batch renaming images, including image resolution, with awk (https://victoria.dev/verbose/batch-renaming-images-including-image-resolution-with-awk/) The most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code: $ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done IMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg IMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg IMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg IMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg IMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg IMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg IMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg IMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg IMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg IMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg IMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg IMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg IMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg // ... etc etc The last item on the aforementioned list is “TODO: come up with a shorter title for this list.” I hate the X11 ICCCM selection system, and you should too - A Rant (http://www.call-with-current-continuation.org/rants/icccm.txt) d00d, that document is devilspawn. I've recently spent my nights in pain implementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program. I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates? So what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misunderstanding it. If you didn't misunderstand it, you wouldn't be here complaining about it would you? HAMMER2 emergency space mode (https://www.dragonflydigest.com/2019/10/22/23652.html) As anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on! Beastie Bits The BastilleBSD community has started work on over 100 automation templates (https://twitter.com/BastilleBSD/status/1186659762458501120) PAM perturbed (https://www.dragonflydigest.com/2019/10/23/23654.html) OpenBSD T-Shirts now available (https://teespring.com/stores/openbsd) FastoCloud (Opensource Media Service) now available on FreeBSD (https://old.reddit.com/r/freebsd/comments/dlyqtq/fastocloud_opensource_media_service_now_available/) Unix: A History and a Memoir by Brian Kernighan now available (https://www.cs.princeton.edu/~bwk/) OpenBSD Moonlight game streaming client from a Windows + Nvidia PC (https://www.reddit.com/r/openbsd_gaming/comments/d6xboo/openbsd_moonlight_game_streaming_client_from_a/) *** Feedback/Questions Tim - Release Notes for Lumina 1.5 (http://dpaste.com/38DNSXT#wrap) Answer Here (http://dpaste.com/3QJX8G3#wrap) Brad - vBSDcon Trip Report (http://dpaste.com/316MGVX#wrap) Jacob - Using terminfo on FreeBSD (http://dpaste.com/131N05J#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
323: OSI Burrito Guy
The earliest Unix code, how to replace fail2ban with blacklistd, OpenBSD crossed 400k commits, how to install Bolt CMS on FreeBSD, optimized hammer2, appeasing the OSI 7-layer burrito guys, and more. Headlines The Earliest Unix Code: An Anniversary Source Code Release (https://computerhistory.org/blog/the-earliest-unix-code-an-anniversary-source-code-release/) What is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system. 2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs. This man sent the first online message 50 years ago (https://www.cbc.ca/radio/thecurrent/the-current-for-oct-29-2019-1.5339212/this-man-sent-the-first-online-message-50-years-ago-he-s-since-seen-the-web-s-dark-side-emerge-1.5339244) As many of you have heard in the past, the first online message ever sent between two computers was "lo", just over 50 years ago, on Oct. 29, 1969. It was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in." The CBC Radio show, “The Current” has a half-hour interview with the man who sent that message, Leonard Kleinrock, distinguished professor of computer science at UCLA "The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there," 50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it. Q: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later? A: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible. Well, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element. Q: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another? A: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it. So I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting. Q: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated. A: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls. When the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side. But in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways. Q: Is there any part of you that regrets giving birth to this? A: Absolutely not. The greater good is much more important. News Roundup How to use blacklistd(8) with NPF as a fail2ban replacement (https://www.unitedbsd.com/d/63-how-to-use-blacklistd8-with-npf-as-a-fail2ban-replacement) blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy. The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use. Unfortunately (dont' ask me why ??) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen. FreeBSD’s handbook chapter on blacklistd (https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-blacklistd.html) OpenBSD crossed 400,000 commits (https://marc.info/?l=openbsd-tech&m=157059352620659&w=2) Sometime in the last week OpenBSD crossed 400,000 commits (*) upon all our repositories since starting at 1995/10/18 08:37:01 Canada/Mountain. That's a lot of commits by a lot of amazing people. (*) by one measure. Since the repository is so large and old, there are a variety of quirks including ChangeLog missing entries and branches not convertible to other repo forms, so measuring is hard. If you think you've got a great way of measuring, don't be so sure of yourself -- you may have overcounted or undercounted. Subject to the notes Theo made about under and over counting, FreeBSD should hit 1 million commits (base + ports + docs) some time in 2020 NetBSD + pkgsrc are approaching 600,000, but of course pkgsrc covers other operating systems too How to Install Bolt CMS with Nginx and Let's Encrypt on FreeBSD 12 (https://www.howtoforge.com/how-to-install-bolt-cms-nginx-ssl-on-freebsd-12/) Bolt is a sophisticated, lightweight and simple CMS built with PHP. It is released under the open-source MIT-license and source code is hosted as a public repository on Github. A bolt is a tool for Content Management, which strives to be as simple and straightforward as possible. It is quick to set up, easy to configure, uses elegant templates. Bolt is created using modern open-source libraries and is best suited to build sites in HTML5 with modern markup. In this tutorial, we will go through the Bolt CMS installation on FreeBSD 12 system by using Nginx as a web server, MySQL as a database server, and optionally you can secure the transport layer by using acme.sh client and Let's Encrypt certificate authority to add SSL support. Requirements The system requirements for Bolt are modest, and it should run on any fairly modern web server: PHP version 5.5.9 or higher with the following common PHP extensions: pdo, mysqlnd, pgsql, openssl, curl, gd, intl, json, mbstring, opcache, posix, xml, fileinfo, exif, zip. Access to SQLite (which comes bundled with PHP), or MySQL or PostgreSQL. Apache with mod_rewrite enabled (.htaccess files) or Nginx (virtual host configuration covered below). A minimum of 32MB of memory allocated to PHP. hammer2 - Optimize hammer2 support threads and dispatch (http://lists.dragonflybsd.org/pipermail/commits/2019-September/719632.html) Refactor the XOP groups in order to be able to queue strategy calls, whenever possible, to the same CPU as the issuer. This optimizes several cases and reduces unnecessary IPI traffic between cores. The next best thing to do would be to not queue certain XOPs to an H2 support thread at all, but I would like to keep the threads intact for later clustering work. The best scaling case for this is when one has a large number of user threads doing I/O. One instance of a single-threaded program on an otherwise idle machine might see a slightly reduction in performance but at the same time we completely avoid unnecessarily spamming all cores in the system on the behalf of a single program, so overhead is also significantly lower. This will tend to increase the number of H2 support threads since we need a certain degree of multiplication for domain separation. This should significantly increase I/O performance for multi-threaded workloads. You know, we might as well just run every network service over HTTPS/2 and build another six layers on top of that to appease the OSI 7-layer burrito guys (http://boston.conman.org/2019/10/17.1) I've seen the writing on the wall, and while for now you can configure Firefox not to use DoH, I'm not confident enough to think it will remain that way. To that end, I've finally set up my own DoH server for use at Chez Boca. It only involved setting up my own CA to generate the appropriate certificates, install my CA certificate into Firefox, configure Apache to run over HTTP/2 (THANK YOU SO VERY XXXXXXX MUCH GOOGLE FOR SHOVING THIS HTTP/2 XXXXXXXX DOWN OUR THROATS!—no, I'm not bitter) and write a 150 line script that just queries my own local DNS, because, you know, it's more XXXXXXX secure or some XXXXXXXX reason like that. Sigh. Beastie Bits An Oral History of Unix (https://www.princeton.edu/~hos/Mahoney/unixhistory) NUMA Siloing in the FreeBSD Network Stack [pdf] (https://people.freebsd.org/~gallatin/talks/euro2019.pdf) EuroBSDCon 2019 videos available (https://www.youtube.com/playlist?list=PLskKNopggjc6NssLc8GEGSiFYJLYdlTQx) Barbie knows best (https://twitter.com/eksffa/status/1188638425567682560) For the #OpenBSD #e2k19 attendees. I did a pre visit today. (https://twitter.com/bob_beck/status/1188226661684301824) Drawer Find (https://twitter.com/pasha_sh/status/1187877745499561985) Slides - Removing ROP Gadgets from OpenBSD - AsiaBSDCon 2019 (https://www.openbsd.org/papers/asiabsdcon2019-rop-slides.pdf) Feedback/Questions Bostjan - Open source doesn't mean secure (http://dpaste.com/1M5MVCX#wrap) Malcolm - Allan is Correct. (http://dpaste.com/2RFNR94) Michael - FreeNAS inside a Jail (http://dpaste.com/28YW3BB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.