Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Similar Podcasts
Elixir Outlaws
Elixir Outlaws is an informal discussion about interesting things happening in Elixir. Our goal is to capture the spirit of a conference hallway discussion in a podcast.
The Cynical Developer
A UK based Technology and Software Developer Podcast that helps you to improve your development knowledge and career,
through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer.
Programming Throwdown
Programming Throwdown educates Computer Scientists and Software Engineers on a cavalcade of programming and tech topics. Every show will cover a new programming language, so listeners will be able to speak intelligently about any programming language.
149: The bhyve has been disturbed, and a wild Dexter appears!
Today on the show, we are going to be chatting with Michael Dexter about a variety of topics, but of course including bhyve! That plus This episode was brought to you by Headlines NetBSD Introduction (https://bsdmag.org/netbsd_intr/) We start off today’s episode with a great new NetBSD article! Siju Oommen George has written an article for BSDMag, which provides a great overview of NetBSD’s beginnings and what it is today. Of course you can’t start an article about NetBSD without mentioning where the name came from: “The four founders of the NetBSD project, Chris Demetriou, Theo de Raadt, Adam Glass, and Charles Hannum, felt that a more open development model would benefit the project: one centered on portable, clean and correct code. They aimed to produce a unified, multi-platform, production-quality, BSD-based operating system. The name “NetBSD” was suggested by de Raadt, based on the importance and growth of networks, such as the Internet at that time, the distributed and collaborative nature of its development.” From there NetBSD has expanded, and keeping in line with its motto “Of course it runs NetBSD” it has grown to over 57 hardware platforms, including “IA-32, Alpha, PowerPC,SPARC, Raspberry pi 2, SPARC64 and Zaurus” From there topics such as pkgsrc, SMP, embedded and of course virtualization are all covered, which gives the reader a good overview of what to expect in the modern NetBSD today. Lastly, in addition to mentioning some of the vendors using NetBSD in a variety of ways, including Point-Of-Sale systems, routers and thin-clients, you may not have known about the research teams which deploy NetBSD: NASA Lewis Research Center – Satellite Networks and Architectures Branch use NetBSD almost exclusively in their investigation of TCP for use in satellite networks. KAME project – A research group for implementing IPv6, IPsec and other recent TCP/IP related technologies into BSD UNIX kernels, under BSD license. NEC Europe Ltd. established the Network Laboratories in Heidelberg, Germany in 1997, as NEC’s third research facility in Europe. The Heidelberg labs focus on software-oriented research and development for the next generation Internet. SAMS-II Project – Space Acceleration Measurement System II. NASA will be measuring the microgravity environment on the International Space Station using a distributed system, consisting of NetBSD.“ My condolences, you’re now the maintainer of a popular open source project (https://runcommand.io/2016/06/26/my-condolences-youre-now-the-maintainer-of-a-popular-open-source-project/) A presentation from a Wordpress conference, about what it is like to be the maintainer of a popular open source project The presentation covers the basics: Open Source is more than just the license, it is about community and involvement The difference between Maintainers and Contributors It covers some of the reasons people do not open up their code, and other common problems people run into: “I'm embarrassed by my code” (Hint: so is everyone else, post it anyway, it is the best way to learn) “I'm discouraged that I can't finish releases on time” “I'm overwhelmed by the PR backlog” “I'm frustrated when issues turn into flamewars” “I'm overcommitted on my open source involvement” “I feel all alone” Each of those points is met with advice and possible solutions So, there you have it. Open up your code, or join an existing project and help maintain it *** FreeBSD Committer Allan Jude Discusses the Advantages of FreeBSD and His Role in Keeping Millions of Servers Running (http://www.hostingadvice.com/blog/freebsd-project-under-the-hood/) An interesting twist on our normal news-stories today, we have an article featuring our very own Allan Jude, talking about why FreeBSD and the advantages of working on an open-source project. “When Allan started his own company hosting websites for video streaming, FreeBSD was the only operating system he had previously used with other hosts. Based on his experience and comfort with it, he trusted the system with the future of his budding business.A decade later, the former-SysAdmin went to a conference focused on the open-source operating system, where he ran into some of the folks on its documentation team. “They inspired me,” he told our team in a recent chat. He began writing documentation but soon wanted to contribute improvements beyond the docs.Today, Allan sits as a FreeBSD Project Committer. It’s rare that you get to chat with someone involved with a massive-scale open-source project like this — rare and awesome.” From there Allan goes into some of the reasons “Why” FreeBSD, starting with Code Organization being well-maintained and documented: “The FreeBSD Project functions like an extremely well-organized world all its own. Allan explained the environment: “There’s a documentation page that explains how the file system’s laid out and everything has a place and it always goes in that place.”” + In addition, Allan gives us some insight into his work to bring Boot-Environments to the loader, and other reasons why FreeBSD “just makes sense” + In summary Allan wraps it up quite nicely: “An important take-away is that you don’t have to be a major developer with tons of experience to make a difference in the project,” Allan said — and the difference that devs like Allan are making is incredible. If you too want to submit the commit that contributes to the project relied on by millions of web servers, there are plenty of ways to get involved! We’re especially talking to SysAdmins here, as Allan noted that they are the main users of FreeBSD. “Having more SysAdmins involved in the actual build of the system means we can offer the tools they’re looking for — designed the way a SysAdmin would want them designed, not necessarily the way a developer would think makes the most sense” A guide to saving electricity and time with poudriere and bhyve (http://justinholcomb.me/blog/2016/07/03/poudriere-in-bhyve-and-bare-metal.html) “This article goes over running poudriere to built packages for a Raspberry Pi with the interesting twist of running it both as a bhyve guest and then switching to running on bare metal via Fiber Channel via ctld by sharing the same ZFS volume.” “Firstly, poudriere can build packages for different architectures such as ARM. This can save hours of build time compared to building ports from said ARM device.” “Secondly, let’s say a person has an always-on device (NAS) running FreeBSD. To save power, this device has a CPU with a low clock-rate and low core count. This low clock-rate and core count is great for saving power but terrible for processor intensive application such as poudriere. Let’s say a person also has another physical server with fast processors and a high CPU count but draws nearly twice the power and a fan noise to match.” “To get the best of both worlds, the goal is to build the packages on the fast physical server, power it down, and then start the same ZFS volume in a bhyve environment to serve packages from the always-on device.” The tutorial walks through setting up ‘ahost’, the always on machine, ‘fhost’ the fast but noisy build machine, and a raspberry pi It also includes creating a zvol, configuring iSCSI over fibre channel and exporting the zvol, booting an iSCSI volume in bhyve, plus installing and setting up poudriere This it configures booting over fibre channel, and cross-building armv6 (raspberry pi) packages on the fast build machine Then the fast machine is shut down, and the zvol is booted in bhyve on the NAS Everything you need to know to make a hybrid physical/virtual machine The same setup could also work to run the same bhyve VM from either ahost or fhost bhyve does not yet support live migration, but when it does, having common network storage like the zvol will be an important part of that *** Interview - Michael Dexter - editor@callfortesting.org (mailto:editor@callfortesting.org) / @michaeldexter (https://twitter.com/michaeldexter) The RoloDexter *** iXSystems Children's Minnesota Star Studio Chooses iXsystems' TrueNAS Storage (https://www.youtube.com/watch?v=FFbdQ_05e-0) *** News Roundup FreeBSD Foundation June 2016 Update (https://www.freebsdfoundation.org/wp-content/uploads/2016/06/FreeBSD-Foundation-June-2016-Update.pdf) The FreeBSD Foundation’s June newsletter is out Make sure you submit the FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) by July 7th: In addition to the opening message from the executive director of the foundation, the update includes details to sponsored work on the FreeBSD VM system, reports from a number of conferences the Foundation attended, including BSDCan The results of the foundation's yearly board meeting People the foundation recognized for their contributions to FreeBSD at BSDCan And an introduction to their new “Getting Started with FreeBSD” project *** [How-To] Building the FreeBSD OS from scratch (http://www.all-nettools.com/forum/showthread.php?34422-Building-the-FreeBSD-OS-from-scratch) A tutorial over at the All-NetTools.com forums that walks through building FreeBSD from scratch I am not sure why anyone would want to build Xorg from source, but you can It covers everything in quite a bit of detail, from the installation process through adding Xorg and a window manager from source It also includes tweaking some device node permissions for easier operation as a non-root user, and configuring the firewall *** Window Systems Should Be Transparent (http://doc.cat-v.org/bell_labs/transparent_wsys/) + Rob Pike of AT&T Labs writes about why Window Systems should be transparent This is an old paper (undated, but I think from the late 80s), but may contain some timeless insights “UNIX window systems are unsatisfactory. Because they are cumbersome and complicated, they are unsuitable companions for an operating system that is appreciated for its technical elegance” “A good interface should clarify the view, not obscure it” “Mux is one window system that is popular and therefore worth studying as an example of good design. (It is not commercially important because it runs only on obsolete hardware.) This paper uses mux as a case study to illustrate some principles that can help keep a user interface simple, comfortable, and unobtrusive. When designing their products, the purveyors of commercial window systems should keep these principles in mind.” There are not many commercial window systems anymore, but “open source” was not really a big thing when this paper was written *** Roger Faulkner, of Solaris fame passed away (http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/12877) “RIP Roger Faulkner: creator of the One and True /proc, slayer of the M-to-N threading model -- and the godfather of post-AT&T Unix” @bcantrill: Another great Roger Faulkner story (https://twitter.com/bcantrill/status/750442169807171584) The story of how pgrep -w saved a monitor -- if not a life (https://news.ycombinator.com/item?id=4306515) @bcantrill: With Roger Faulkner, Tim led an engineering coup inside Sun that saved Solaris circa 2.5 (https://twitter.com/bcantrill/status/750442169807171584) *** Beastie Bits: Developer Ed Maste is requesting information from those who are users of libvgl. (https://lists.freebsd.org/pipermail/freebsd-stable/2016-June/084843.html) HEADS UP: DragonFly 4.5 world reneeds rebuilding (http://lists.dragonflybsd.org/pipermail/users/2016-June/249748.html) Chris Buechler is leaving the pfSense project, the entire community thanks you for your many years of service (https://blog.pfsense.org/?p=2095) GhostBSD 10.3-BETA1 now available (http://ghostbsd.org/10.3_BETA1) DragonFlyBSD adds nvmectl (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500671.html) OPNsense 16.1.18 released (https://opnsense.org/opnsense-16-1-18-released/) bhyve_graphics hit CURRENT (https://svnweb.freebsd.org/base?view=revision&revision=302332) BUG Update FreeBSD Central Twitter account looking for a new owner (https://twitter.com/freebsdcentral/status/750053703420350465) NYCBUG meeting : Meet the Smallest BSDs: RetroBSD and LiteBSD, Brian Callahan (http://lists.nycbug.org/pipermail/talk/2016-July/016732.html) NYCBUG install fest @ HOPE (http://lists.nycbug.org/pipermail/talk/2016-June/016694.html) SemiBUG is looking for presentations for September and beyond (http://lists.nycbug.org/pipermail/semibug/2016-June/000107.html) Caleb Cooper is giving a talk on Crytpo at KnoxBUG on July 26th (http://knoxbug.org/content/2016-07-26) Feedback/Questions Leif - ZFS xfer (http://pastebin.com/vvASr64P) Zach - Python3 (http://pastebin.com/SznQHq7n) Dave - Versioning (http://pastebin.com/qkpjKEr0) David - Encrypted Disk Images (http://pastebin.com/yr7BUmv2) Eli - TLF in all the wrong places (http://pastebin.com/xby81NvC) ***
148: The place to B...A Robot!
This week on the show, Allan and I are going to be showing you a very interesting interview we did talking about using FreeBSD to drive This episode was brought to you by Headlines FreeBSD Core Team Election (https://www.freebsd.org/administration.html#t-core) Core.9 has been elected, and will officially take over from Core.8 on Wednesday, 6 July 2016 Many thanks to the outgoing members of the core team for their service over the last 2 years 214 out of 325 eligible voters (65.8%) cast their votes in an election counting 14 candidates. The top nine candidates are, in descending order of votes received: 180 84.1% Ed Maste (incumbent) 176 82.2% George V. Neville-Neil (incumbent) 171 79.9% Baptiste Daroussin (incumbent) 168 78.5% John Baldwin 166 77.6% Hiroki Sato (incumbent) 147 68.7% Allan Jude 132 61.7% Kris Moore 121 56.5% Benedict Reuschling 108 50.5% Benno Rice There was no tie for ninth. BSDNow and the entire community would also like to extend their thanks to all those who stood for election to the core team Next week’s core meeting will encompass the members of Core.8 and Core.9, as responsibility for any outstanding items will be passed from outgoing members of core to the new incoming members *** Why I run OpenBSD (http://deftly.net/posts/2016-05-31-why-i-run-openbsd.html) This week we have a good article / blog post talking about why the posted has moved to OpenBSD from Linux. “One thing I learned during my travels between OSs: consistency is everything. Most operating systems seem to, at least, keep a consistent interface between themselves and binaries / applications. They do this by keeping consistent APIs (Application Programming Interfaces) and ABIs (Application Binary Interfaces). If you take a binary from a really old version of Linux and run or build it on a brand-spanking new install of Linux, it will likely Just Work™. This is great for applications and developers of applications. Vendors can build binaries for distribution and worry less about their product working when it gets out in the wild (sure this binary built in 2016 will run on RedHat AS2.1!!).“ The author then goes through another important part of the consistency argument, with what he calls “UPI” or “User Program Interfaces”. In other words, while the ABI may be stable, what about the end-user tooling that the user directly has to interact with on a daily basis? “This inconsistency seems to have come to be when Linux started getting wireless support. For some reason someone (vendors, maybe?) decided that ifconfig wasn’t a good place to let users interact with their wireless device. Maybe they felt their device was special? Maybe there were technical reasons? The bottom line is, someone decided to create a new utility to manage a wireless device… and then another one came along… pretty soon there was iwconfig(8), iw(8), ifconfig(8), some funky thing that let windows drivers interface with Linux.. and one called ip(8) I am sure there are others I am forgetting, but I prefer to forget. I have moved onto greener pastures and the knowledge of these programs no longer serves me.” The article then goes through the rundown of how he evaluated the various BSD’s and ultimately settled on OpenBSD: “OpenBSD won the showdown. It was the most complete, simple, and coherent system. The documentation was thorough, the code was easy to follow and understand. It had one command to configure all of the network interfaces! I didn’t have wireless, but I was able to find a cheap USB adapter that worked by simply running man -k wireless and reading about the USB entries. It didn’t have some of the applications I use regularly, so I started reading about ports (intuitively, via man ports!).” The ultimate NetBSD Router (http://blog.tbrodel.me/2016/#netbsd-router) “So yesterday I spent the day setting up a new firewall at home here, based off of this BSD Now tutorial. Having set up a couple of OpenBSD routers before, either based on old laptops, bulky old power-sucking desktops or completely over-specced machines like the Intel NUC, I wanted to get some kind of BSD onto a low-powered ARM board and use that instead.” “I've had a couple of Cubietrucks lying around for a while now, I've used them in a couple of art installations, running Debian and Pure Data, but over all they've been a bit disappointing. It's more the manufacturer's fault but they require blobs for the graphics and audio, which Debian won't allow, so as a multimedia board they're dud for video, and only passable for audio work with a usb sound card. So they've been collecting dust.” “Only thing missing is a second NIC, luckily I had an Apple USB->Ethernet dongle lying around, which when I bought it was the cheapest thing I could find on eBay that OpenBSD definitely supported. There, and on NetBSD, it's supported by the axe(4) driver. USB 2.0 works fine for me as I live in Australia and my ISP can only give me 30Mbps, so this should do for the forseeable future.” + The article then walks through installing and configuring NetBSD + Configuration includes: pf, unbound, and dhcpd “This project has been really fun, I started with basically no experience with NetBSD and have finished with a really useful, low-powered and robust appliance. It's a testament to the simplicity of the NetBSD system, and the BSD design principles in general, that such a novice as myself could figure this out. The NetBSD project has easily the most polished experience on Allwinner ARM boards, even Debian doesn't make it this easy. It's been a joy running the system, it has the bits I love from OpenBSD; ksh(1), tmux(1), an http daemon in base and of course, pf(4). This is mixed with some of the pragmatism I see in FreeBSD; a willingness to accept blobs if that really is the only way to boot, or get audio, or a video console.” bhyve-Bootable Boot Environments (http://callfortesting.org/bhyve-boot-environments/) We have a lengthy article also today from our friend Michael Dexter, who asks the basic question “What if multibooting and OS upgrades weren't horrible?” No doubt if you’ve been a frequent listener to this show, you’ve heard Allan or Myself talking about ZFS Boot Environments, and how they can “change your life”. Well today Michael goes further into detail on how the BE’s work, and how they can be leveraged to do neat things, like installing other versions of an operating system from the original running system. “If you are reading this, you have probably used a personal computer with a BSD or GNU/Linux operating system and at some point attempted to multiboot between multiple operating systems on the same computer. This goal is typically attempted with complex disk partitioning and a BSD or GNU/Linux boot loader like LILO or GRUB, plus several hours of frustrating experimentation and perhaps data loss. While exotic OS experimentation has driven my virtualization work since the late 1990s, there are very pragmatic reasons for multibooting the same OS on the same hardware, notable for updates and failback to "known good" versions. To its credit, FreeBSD has long had various strategies including the NanoBSD embedded system framework with primary and secondary root partitions, plus the nextboot(8) utility for selecting the "next" kernel with various boot parameters. Get everything set correctly and you can multiboot "with impunity". “That's a good start, and over time we have seen ZFS "boot environments" be used by PC-BSD and FreeNAS to allow for system updates that allow one to fall back to previous versions should something go wrong. Hats off to these efforts but they exist in essentially purpose-built appliance environments. I have long sensed that there is more fun to be had here and a wonderful thing happened with FreeBSD 10.3 and 11.0: Allan Jude added a boot environment menu to the FreeBSD loader” From here Michael takes us through the mechanical bits of actually creating a new ZFS dataset (BE) and performing a fresh FreeBSD 10.3 installation into this new boot-environment. The twist comes at the end, where he next sets up the BE to be a root NFS for booting in bhyve! This is interesting and gives you a way to test booting into your new environment via a VM, before rebooting the host directly into it. *** Interview - Edicarla Andrade & Vinícius Zavam - @egypcio (https://twitter.com/egypcio) BSD-Powered Robots News Roundup Tomohiro Kasumi explains what “@@” means, in the context of the Hammer filesystem (http://lists.dragonflybsd.org/pipermail/users/2016-June/249717.html) A post from the Dragonfly users’ mailing list about what the @@ construct means in the Hammer filesystem “@@ represents the existence of a PFS which is logically separated pseudo filesystem space within HAMMER's B-Tree” “HAMMER only has 1 large B-Tree per filesystem (not per PFS), so all the PFS exist within that single B-Tree. PFS are separated by localization parameter which is one of the B-Tree keys used to lookup the tree.” Each substring in "@@-1:00001" means: "@@" means it's a PFS or snapshot. "-1" means it's a master. ":" is just a separator. "00001" means it's PFS#1, where PFS#0 is the default PFS created on newfs. There is no "00000" because that's what's mounted on /HAMMER. PFS# is used for localization parameter. “Localization parameter has the highest priority when inserting or looking up B-Tree elements, so fs elements that belong to the same PFS# tend to be localized (clustered) within the B-Tree” There is also a note about how snapshots are named: "@@0x00..." A user points out that having : in the path can confuse some applications, such as in the case of adding the current directory or a relative path to the $PATH environment variable, which is a colon delimited list of paths This seems quite a bit more confusing that the datasets created by ZFS, but they might have other useful properties *** FreeBSD 11.0 nearing RC1 (https://www.freebsd.org/releases/11.0R/schedule.html) We’ve all been eagerly awaiting the pending release of FreeBSD 11.0, and the schedule has now been updated! The first release candidate is slated for July 29th! If all goes well (and we stick to schedule) there will be another RC2 and possible RC3 release, before 11.0 officially drops near the end of August. Start playing with those builds folks, be sure to send your feedback to the team to make this the best .0 release ever! *** TensorFlow on FreeBSD (http://ecc-comp.blogspot.com/2016/06/tensorflow-on-freebsd.html) Next we have a blog post about the experience of a “new” FreeBSD user trying to deploy some non-ported software to his new system. Specifically he was interested in running TensorFlow, but not doing a port himself, because in his words: “First, I apologize for not supplying a port archive myself. After reading the FreeBSD handbook for creating a port, it's too complex of a task for me right now. I've only been using FreeBSD for two weeks. I would also not like to waste anyone's time giving them a terrible port archive and mess up their system.” First of all, good ports are often born out of bad ports! Don’t let the porting framework daunt you, give it a go, since that's the only way you are going to learn how to write “good” ports over time. The porters-handbook is a good first place to start, plus the community usually is very helpful in providing feedback. He then walks us through the changes made to the TensorFlow code (starting with the assumption that OSX was a good “flavor” to begin porting from) and ultimately compiling. This ends up with the creation of a pip package which works! A good tutorial, and also very similar to what goes on in the porting process. With this write-up perhaps somebody will take up creating a port of it… hint hint! *** NetBSD: A New Beginning? (http://jamesdeagle.blogspot.ca/2016/06/netbsd-new-beginning.html) We don’t get enough NetBSD news at times, but this post by James Deagle talks about his adventure with NetBSD 7.0 and making it his “new beginning” “After a few months of traipsing around the worlds of SunOS and Linux, I'm back to NetBSD for what I hope will be a lengthy return engagement. And while I'm enamored of NetBSD for all the previously-mentioned reasons, I'm already thinking ahead to some problems to solve, some of which have also been mentioned before.” He then goes through and lists some of the small nits he’s still running into during the daily workflow YouTube audio - Specifically he mentions that no audio is playing, but wonders if Flash plays some part. (Ideally you’re not using Flash though, in which case you need to check the audio backend FF is using. Try PulseAudio since it seems the best supported. If pulse is already enabled, install ‘pavucontrol’ to make sure audio is playing to the correct sound device) Slow gaming performance (TuxKart and Celestia) - Check DRI / Xorg? Or is it CPU bound? Lastly some unspecified Wireless issues, which typically end up being driver related. (Or use another chipset) Beastie Bits Reproducible NetBSD? 77.7% of the way there (https://reproducible.debian.net/netbsd/netbsd.html) Create FreeBSD virtual machine using qemu. Run the VM using xhyve. (https://gist.github.com/zg/38a3afa112ddf7de4912aafc249ec82f) FreeBSD PowerPC 32bit pkg repository (unofficial). ~19,500 packages, more to come (https://joshcummings.net/pub/FreeBSD) NetBSD machines at Open Source Conference 2016 Gunma (http://mail-index.netbsd.org/netbsd-advocacy/2016/05/16/msg000706.html) Adam Leventhal (of ZFS and DTrace) does an analysis of APFS (http://arstechnica.com/apple/2016/06/a-zfs-developers-analysis-of-the-good-and-bad-in-apples-new-apfs-file-system/) SemiBug June meeting summary (http://lists.nycbug.org/pipermail/semibug/2016-June/000106.html) KnoxBug Meeting (http://knoxbug.org/content/2016-07-26) Feedback/Questions Andrew - iocage (http://pastebin.com/nuYTzaG6) Florian - Arm + GitHub (http://pastebin.com/PzY68hNS) Clint - Synth (http://pastebin.com/JESGZjLu) Leonardo - Translations (http://pastebin.com/b4LAiPs4) Zachary - Moving things to VMs (http://pastebin.com/VRc8fvBk) ***
147: Release all the things!
On this episode of BSDNow, we will be talking to Glen Barber and Peter Wemm of the FreeBSD RE and Cluster Admin teams! That plus our This episode was brought to you by Headlines 2016 FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) We often get comments from our listeners, “I’m not a developer, how can I help out”? Well today is your chance to do something. The FreeBSD Foundation has its 2016 Community Survey online, where they are asking for feedback from you! I just did the survey, it’ll take you about 5 minutes, but gives you a chance to provide valuable feedback to the foundation about things that are important to you. Be sure to answer in as much detail as possible and the foundation will review and use this feedback for its operations going forward. *** ART, OpenBSDs new routing table, single thread performances (http://www.grenadille.net/post/2016/06/17/ART-single-thread-performances) OpenBSD has changed the way routes are looked up in the kernel as part of their path to an SMP networking stack The “Allotment Routing Table” (ART) is a performance tradeoff, where more memory is used to store the routing table, in exchange for faster lookups With this new arrangement, a full BGP routing table will grow from 130MB to 180MB of memory “ART is a free multibit trie based routing table. To keep it simple, it can be seen as using more memory for fewer CPU cycles. In other words, we get a faster lookup by wasting memory. The original paper (http://www.hariguchi.org/art/art.pdf) presents some performance comparisons between two ART configurations and the BSD Radix. But how does this apply to OpenBSD?” “I asked Hrvoje Popovski to run his packet forwarding test on his Xeon box (E5-2620 v2 @ 2.10GHz, 2400.34 MHz) with ix(4) (82599) interfaces. The test setup consist of three machines with the OpenBSD box in the middle” “The simulations have been performed with an OpenBSD -current from June 9th. The machine is configured with pf(4) disabled in order to force a single route lookup for every IPv4 packet. Based on the result of the lookup the kernel decide if it should forward, deliver or drop the packet” *** BSDCan 2016 Playlist (https://www.youtube.com/playlist?list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC) The complete set of videos from BSDCan is online and ready to be consumed Remember the good-ole days where we would wait months (or years) to get videos posted from conferences? Well, who are we kidding, some conferences STILL do that, but we can’t count BSDCan among them. Only two weeks out from this years exciting BSDCan, and all the videos have now landed on YouTube. Granted, this is no substitute for actually being at the conference, but even if you attended you probably missed quite a few of the talks. There are no videos of the hallway track, which is the best part of the conference Except the dinner discussion of course. and don’t forget the hacker lounge *** Should you be scared of Unix signals? (http://jvns.ca/blog/2016/06/13/should-you-be-scared-of-signals/) Do you know much about UNIX Signals? Are you afraid of their complexity? Do you know there are signals other than SIGKILL? This article talks about the practical implications of signals from a programming perspective The things you need to consider when dealing with signals Basically, you register a “signal handler”, the function that will be run when a signal arrives As you program is running, if a signal arrives, your program will be interrupted. Its current state will be saved and any system calls in progress will return EINTR (Error, Interrupted), then your signal handler will be run. Once the signal handler is complete, the state of your application will be restored, and execution will resume As long as your program properly handles this interruption, and errors that might result from it (getting EINTR from a read() call, instead of the data you expected), then everything should be fine. Of course, you need to be careful what you do inside your signal handler, as if you modify any variables or state in your application, it might be very confused when it resumes. *** Interview - Glen and Peter- News Roundup Unik - The Unikernel Compilation and Deployment Platform (uses NetBSD's Rump) (https://github.com/emc-advanced-dev/unik) We’ve talked a bit about NetBSD’s RUMP (unikernel) in the past, including articles on how to deploy services using it. Now we have an interesting project which makes the process super-easy, and dare-we-say almost “Docker-Like?” The Unik project has a fairly complete walkthrough right on their GitHub project page, including details on installation and creating your own unikernel containers. In addition, it provides instructions on boot-strapping your own Go/Node.js/Python/Java applications, and supports out of Box VCenter / AWS / Qemu / VirtualBox providers. *** PkgSrc 50th Release Highlights () pkgsrc is celebrating its 50th release, and to highlight this, they have posted a series of interviews from people who have been active in the project pkgsrc 50th release interviews - Jonathan Perkin (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_jonathan) pkgsrc 50th release interviews - Ryo ONODERA (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_ryo) pkgsrc 50th release interviews - Joerg Sonnenberg (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interview_with) pkgsrc 50th release interviews - Sevan Janiyan (https://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_sevan) *** Migrating to FreeBSD from Solaris 11 (http://justinholcomb.me/blog/2016/02/28/migration-to-freebsd-part1.html) Part 2 (http://justinholcomb.me/blog/2016/03/12/migration-to-freebsd-part2.html) Part 3 (http://justinholcomb.me/blog/2016/03/19/migration-to-freebsd-part3.html) Part 4 (http://justinholcomb.me/blog/2016/03/26/migration-to-freebsd-part4.html) Part 5 (http://justinholcomb.me/blog/2016/04/03/migration-to-freebsd-part5.html) *** How to chroot www/firefox on NetBSD (https://github.com/alnsn/localpkgsrc/tree/master/firefox-chroot) Looking for a jail-like method of running FireFox on NetBSD? (Or possibly other BSDs?) We have a github repo with details on how to setup and run FireFox using a chroot using a “webuser” account for safety. Think of this as a jail alternative, may be useful on systems with no jail support. Of interest is the method used to do X forwarding. It uses Xorg TCP listen option (which is often off by default for security reasons). Perhaps SSH X forwarding would be a better alternative. (Or nullfs mounts of /tmp) *** Beastie Bits Tredly - V1 Release Candidate (https://github.com/tredly/tredly/releases/tag/v1.0.0-rc.1) Call for Testing - ypldap testing against OpenLDAP and Microsoft Active Directory (http://lists.freebsd.org/pipermail/freebsd-current/2016-June/061775.html) BSD Magazine, June 2016 Out Now (https://bsdmag.org/) Hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500610.html) chyves pre-announcement (http://justinholcomb.me/blog/2016/06/14/chyves-project-preannouncement.html) *** Feedback/Questions Michael - Versioning (http://pastebin.com/1hpGrmuL) Michael - Removing Encryption (http://pastebin.com/2PkrMGGx) Bostjan - PC-BSD Questions (http://pastebin.com/q5VdmNxG) Fong - ZFS Rollback (http://pastebin.com/2aedLV7d) Jochen - Docker on FBSD (http://pastebin.com/dneVZkXc) ***
146: Music to Beastie’s ears
Kris is on vacation this week, so allan flies solo, provides a recap of BSDCan & cover's a boatload of news including Microsoft This episode was brought to you by Headlines BSDCan Recap and Live Stream Videos (http://www.bsdcan.org/2016/) OpenBSD BSDCan 2016 papers now available (http://www.openbsd.org/papers) Allan’s slides (http://allanjude.com/bsd/BSDCan2016_-_GELIBoot.pdf) and Paper (http://allanjude.com/bsd/AsiaBSDCon2016_geliboot_pdf1a.pdf) Michael W Lucas presents Allan with a gift (https://www.youtube.com/watch?v=LFgxAHkrSTg) “FreeBSD Mastery: Advanced ZedFS” (http://blather.michaelwlucas.com/archives/2698) Highlighted Tweets: Groff Arrives at BSDCan (https://twitter.com/Keltounet/status/740344735194320896) FreeBSD Foundation recognizes the contributions of Bryan Drewery, Rod Grimes, Warren Block, & Gleb Smirnoff (https://twitter.com/freebsdfndation/status/742456950676393984) A moment of silence and shots in memory in Benjamin Perrault @creepingfur (https://twitter.com/__briancallahan/status/741854476340858880) @gvnn3 sells the FreeBSD Foundation shirt off of his back for Charity (https://twitter.com/Keltounet/status/741763867471155201) Michael W. Lucas asks Matt Ahrens how to pronounce ZFS, “You can pronounce ZFS however you like, but if you pronounce it 'reiserfs', people might be confused.” (https://twitter.com/cperciva/status/741375414967410688) Sysadmin T-Shirt (https://twitter.com/BSDCan/status/741420633007874050) FreeBSD Dev Summit ran out of room on the chalkboards listing accomplishments of 11.0 (https://twitter.com/SeanChittenden/status/740904105388978176) List of things people have or want for FreeBSD 12 (https://twitter.com/Keltounet/status/740928627471159296) Matt Ahrens signing Allan’s ZFS book (https://twitter.com/kprovst/status/741322268480049152?cn=bWVudGlvbg%3D%3D&refsrc=email) FreeBSD’s new marketing strategy (https://twitter.com/cperciva/status/741707948469157889) Charity Auction: systemd whoopie cushion (https://twitter.com/HippyWizard/status/741768670704066560) Embarass OpenBSD’s @HenningBrauer by donating $10 to charity for a selfie with him wearing a Linux t-shirt (https://twitter.com/juliefriday/status/741948048788586496) @GroffTheBSDGoat changes handlers, from @HenningBrauer to @GavinAtkinson (https://twitter.com/GroffTheBSDGoat/status/742415390798716928) Day 1 Video (https://www.youtube.com/watch?v=AOidjSS7Hsg) Day 2 Video (https://www.youtube.com/watch?v=z7pDnBO5wSM) Allan’s GELIBoot talk (day 2) (https://www.youtube.com/watch?v=z7pDnBO5wSM&feature=youtu.be&list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC&t=4440) *** Media Coverage of Microsoft + FreeBSD story (https://azure.microsoft.com/en-us/blog/freebsd-now-available-in-azure-marketplace/) Microsoft has released their own custom image of FreeBSD 10.3 for the Azure Cloud “This means that not only can you quickly bring-up a FreeBSD VM in Azure, but also that in the event you need technical support, Microsoft support engineers can assist.” “Microsoft is the publisher of the FreeBSD image in the marketplace rather than the FreeBSD Foundation. The FreeBSD Foundation is supported by donations from the FreeBSD community, including companies that build their solutions on FreeBSD. They are not a solution provider or an ISV with a support organization but rather rely on a very active community that support one another. In order to ensure our customers have an enterprise SLA for their FreeBSD VMs running in Azure, we took on the work of building, testing, releasing and maintaining the image in order to remove that burden from the Foundation. We will continue to partner closely with the Foundation as we make further investments in FreeBSD on Hyper-V and in Azure.” "It's quite a significant milestone for FreeBSD community and for Microsoft to publish a supported FreeBSD image on Azure Marketplace. We really appreciate Microsoft's commitment and investment in FreeBSD project". - Justin T. Gibbs, President of FreeBSD Foundation Microsoft took a FreeBSD 10.3-RELEASE image and added additional patches, most of which they have upstreamed but that were too late for the regular 10.3 release cycle. Rather than requiring users to use a snapshot of the stable/10 branch, which would complicate the user experience, and complicate the job of the Microsoft support engineers, they created their own “certified” release This allows Microsoft to selectively deploy errata fixes to the image as well It is not clear how this affects update mechanisms like freebsd-update(8) The Register (http://www.theregister.co.uk/2016/06/09/microsoft_freebsd/) The Inquirer (http://www.theinquirer.net/inquirer/news/2461070/microsoft-creates-own-distribution-of-freebsd-for-azure-developers) Infoworld (http://www.infoworld.com/article/3082090/open-source-tools/is-microsoft-publishing-its-own-freebsd-yes-and-no.html) The Hacker News (http://thehackernews.com/2016/06/microsoft-azure-freebsd.html) Windows Report (http://windowsreport.com/microsoft-freebsd-10-3-ready-made-vm-image-azure/) Windows Club (http://news.thewindowsclub.com/microsoft-freebsd-operating-system-84375/) *** Select works poorly (http://www.tedunangst.com/flak/post/select-works-poorly) “At the bottom of the OpenBSD man page for select is a little note. “Internally to the kernel, select() and pselect() work poorly if multiple processes wait on the same file descriptor.” There’s a similar warning in the poll man page. Where does this warning come from and what does it mean?” Ted found that at first glance, OpenBSD’s select() appears to be quite bad: “whenever some data gets written, we call wakeup(&selwait);. Based on what we’ve seen so far, one can conclude that this is likely to be inefficient. Every time any socket has some data available, we wake up every selecting process in the system. Works poorly indeed.” After further investigation, it turns out to not be quite as bad When the select() is first setup, the PID of the process that cares about the FD is recorded in the selinfo struct If a second process runs select() on the same FD, the SI_COLL (Select Collision) flag is set on the selinfo struct When selwakeup() is called, if SI_COLL is set, all select()ing processes are woken up, and the sysctl kern.nselcoll is incremented. If the flag is not set, and only a single PID is waiting for activity on that FD, only that process is woken up “This is not an intractable problem. kevent avoids it entirely. Other implementations may too. But practically, does it need to be solved? My laptop says it’s happened 43 times. A server with substantially more uptime says 0. Doesn’t seem so bad.” *** Interview - Hans Petter Selasky - hps@freebsd.org (mailto:hps@freebsd.org) / @twitter (https://twitter.com/user) Designing FreeBSD’s USB drivers, hooking up a piano to FreeBSD & more! *** News Roundup Timeline of libexpat random vulnerability (http://www.tedunangst.com/flak/post/timeline-of-libexpat-random-vulnerability) Do you use FreeBSD as web server? Why or why not? (https://news.ycombinator.com/item?id=11804565) 20 years of NetBSD code Bloat (http://kristerw.blogspot.sg/2016/05/20-years-of-netbsd-code-bloat.html) HP Chromebook 13 now booting OpenBSD (https://jcs.org/statuses/2016/06/08/740606952149942272/) UNIX for Poets (https://web.stanford.edu/class/cs124/lec/124-UnixForPoets.pdf) Comparing live version upgrade methods (https://distrowatch.com/weekly.php?issue=20160530#upgrades) My life with FreeBSD on a Thinkpad X220 (https://www.reddit.com/r/BSD/comments/4n3flx/my_life_with_freebsd_on_a_thinkpad_x220/)
145: At the Core of it all
It’s BSDCan time! Allan and I are both enjoying what is sure to be a super-busy week, but don’t think we’ve forgotten about This episode was brought to you by Interview - Benno Rice - benno@freebsd.org (mailto:benno@freebsd.org) / @jeamland (https://twitter.com/jeamland) Manager, OS & Networking at EMC Isilon Emily Dunham: Community Automation (https://www.youtube.com/watch?v=dIageYT0Vgg) iXsystems 1U Rackmount Server - 4 Bay Hot-Swap SAS/SATA Drive Bays 400W Redundant Power Supply - Single Socket Embedded CPU (48 cores) - 8 DIMM Slots with 16GB DIMMs for a total of 128GB RAM – Dual Gigabit LAN, Dual 10GbE SFP+ and 1 x 40Gb QSFP+ port, (1) PCI-E Expansion Slots + IPMI Dedicated LAN - Cavium ThunderX ARM CN8890 48 Core ThunderX CPU - 2.5GHz per core System has 128GB RAM, 4 x 2TB SATA HDD, Additional Intel i350 (2 x 1GbE) Beastie Bits file considered harmful (http://www.tedunangst.com/flak/post/file-considered-harmful) An open source talk on ZFS. “Intro to ZFS” as a set of open source slides for the community to build on, and to reuse. Go give this talk at your local conference. (https://github.com/problame/talkintrozfs2016) ARMv7 now has a bootloader (http://undeadly.org/cgi?action=article&sid=20160529145411) SHA256/512 speed improvements in FreeBSD 11 (https://svnweb.freebsd.org/base?view=revision&revision=300966) pkgsrc 50th release interviews - Joerg Sonnenberg (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interview_with) DFly versus PC-BSD on a Laptop (http://lists.dragonflybsd.org/pipermail/users/2016-May/249636.html) FreeBSD ifconfig can print subnet masks in CIDR or dotted-quad, finally (https://svnweb.freebsd.org/base?view=revision&revision=301059) Feedback/Questions Eli - Getting rid of ports? (http://pastebin.com/4Y6VYSyN) Morgan - Best way to admin jails? (http://pastebin.com/w8hsMtbc) Simon - Use existing pkgs in poudriere (http://pastebin.com/mqSJk0pP) Pete - Lots of Q’s (http://pastebin.com/1M7HLAXs) Van - Made the switch (http://pastebin.com/NTVBvtC5) ***
144: The PF life
It’s only one-week away from BSDCan, both Allan and I are excited to meet some of you in person! However, the show keeps on This episode was brought to you by Headlines dotSecurity 2016 - Theo de Raadt - Privilege Separation and Pledge (http://www.dotsecurity.io/) Video (https://www.youtube.com/watch?v=a_EYdzGyNWs) Slides (https://www.openbsd.org/papers/dot2016.pdf) Interested in Privilege Separation and security in general? If so, then you are in for a treat, we have both the video and slides from Theo de Raadt at dotSecurity 2016. Specifically the the talk starts off looking at Pledge (no copyright issues with the pictures I hope??) and how their NTP daemon uses it. After going through some internals, Theo reveals that around 10% of programs “pledged” so far were found to be trying to do actions outside of their security scope. On the future-work side, they mention going back and looking at OpenSSH privilege separation next, as well as working with other OS’s that may want pledge support. *** bhyve now supports UEFI GOP (https://lists.freebsd.org/pipermail/freebsd-virtualization/2016-May/004471.html) The log awaited UEFI GOP (Graphics Output Protocol (https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface#GOP)) features has landed in bhyve This provides emulated graphics via an internal VNC server, allowing users to have full graphical access to the guest OS This allows installation of Windows guests without needing to create a modified ISO with an unattended installation script The code has not actually landed in FreeBSD head yet, but has been committed to a project branch Following a few simple commands, you can compile the new bhyve binary on your -CURRENT system and get started right away This feature is expected to be included in the upcoming FreeBSD 11.0 This commit drop also brings with it: XHCI -- an emulated usb tablet device that provides exact mouse positioning in supported OSs PS2 mouse for fallback if the guest does not support XHCI (Windows 7) PS2 keyboard “The code has been tested with Windows 7/8/8.1/10 and Server 2k12/2k16, Ubuntu 15.10, and FreeBSD 10.3/11-CURRENT” “For VNC clients, TightVNC, TigherVNC, and RealVNC (aka VNC Viewer) have been tested on various hosts. The OSX VNC client is known not to work.” The VNC server supports an optional ‘wait’ parameter, that causes the VM to not actually boot until the VNC client connects, allowing you to interrupt the boot process if need be Related user blog post (http://justinholcomb.me/blog/2016/05/28/bhyve-uefi-gop-support.html) SVN commit (https://svnweb.freebsd.org/base?view=revision&revision=300829) *** zfsd lands in FreeBSD HEAD, in time for 11.0-RELEASE (https://svnweb.freebsd.org/base?view=revision&revision=300906) zfsd has been committed to FreeBSD -CURRENT in time to be included in FreeBSD 11.0 zfsd is the missing piece required to make ‘hot spares’ work properly in FreeBSD ZFS “zfsd attempts to resolve ZFS faults that the kernel can't resolve by itself. It listens to devctl(4) events, which is how the kernel notifies of events such as I/O errors and disk removals. Zfsd attempts to resolve these faults by activating or deactivating hotspares and onlining offline vdevs.” “The administrator never interacts with zfsd directly. Instead, he controls its behavior indirectly through zpool configuration. There are two ways to influence zfsd: assigning hotspares and setting pool properties. Currently, only the autoreplace property has any effect. See zpool(8) for details.” So, what example does it do? Device Removal: “When a leaf vdev disappears, zfsd will activate any available hotspare.” Device Arrival: “When a new GEOM device appears, zfsd will attempt to read its ZFS label, if any. If it matches a previously removed vdev on an active pool, zfsd will online it. Once resilvering completes, any active hotspare will detach automatically.” So if you disconnect a drive, then reconnect it, it will automatically be brought back online. Since ZFS is smart, the resilver will only have to copy data that has changed since the device went offline. “If the new device has no ZFS label but its physical path matches the physical path of a previously removed vdev on an active pool, and that pool has the autoreplace property set, then zfsd will replace the missing vdev with the newly arrived device. Once resilvering completes, any active hotspare will detach automatically.” If the new drive is in the same slot in your hot swap array as a failed device, it will be used as a replacement immediately. vdev degrade or fault events: “If a vdev becomes degraded or faulted, zfsd will activate any available hotspare. If a leaf vdev generates more than 50 I/O errors in a 60 second period, then zfsd will mark that vdev as FAULTED. zfs(4) will no longer issue any I/Os to it. zfsd will activate a hotspare if one is available.” Same for checksum errors. So if zfsd detects a drive is going bad, it brings the hotspare online before it is too late Spare addition: “If the system administrator adds a hotspare to a pool that is already degraded, zfsd will activate the spare.” Resilver complete: “zfsd will detach any hotspare once a permanent replacement finishes resilvering.” Physical path change: “If the physical path of an existing disk changes, zfsd will attempt to replace any missing disk with the same physical path, if its pool's autoreplace property is set.” In general, this tool means less reliance on the system administrator to keep the pool healthy *** W^X now mandatory in OpenBSD (http://undeadly.org/cgi?action=article&sid=20160527203200) We’ve talked a bit about W^X in the past. (Refresher: Memory being writable and executable at once) Well, this major security no-no is no-more on OpenBSD. Theo has committed a change which now prevents violations of this policy: “W^X violations are no longer permitted by default. A kernel log message is generated, and mprotect/mmap return ENOTSUP. If the sysctl(8) flag kern.wxabort is set then a SIGABRT occurs instead, for gdb use or coredump creation.” There are a few cases where you may still need W^X, which Theo points out can be enabled on a file-system basis. “W^X violating programs can be permitted on a ffs/nfs filesystem-basis, using the "wxallowed" mount option. One day far in the future upstream software developers will understand that W^X violations are a tremendously risky practice and that style of programming will be banished outright. Until then, we recommend most users need to use the wxallowed option on their /usr/local filesystem. At least your other filesystems don't permit such programs.” This is a great ability to grow, since now users can begin doing auditing of programs that violate this principle and making noise to upstream. *** Interview - Kristof Provost - kp@freebsd.org (mailto:kp@freebsd.org) @kprovst (https://twitter.com/kprovst) pf improvements on FreeBSD *** News Roundup GELI Support for the EFI Loader (https://ericmccorkleblog.wordpress.com/2016/05/28/freebsd-geli-support/) We’ve had Allan’s work to bring GELI support to the GPT / BIOS / ZFS loader for a while now, but the missing piece has been support for EFI. No longer, Eric McCorkle has posted a blog entry (with relevant github links) introducing us to his work to bring GELI encryption support to EFI. First the bad-news. This won’t make it into 11.0. (Maybe PC-BSD, TBD) Next he explains why this is more than just a new feature, but a re-factor of the EFI boot code: I have already written extensively about my EFI refactoring here. The reason for undertaking this effort, however, was driven by GELI support. Early in my work on this, I had implemented a non-EFI “providers” framework in boot1 in order to support the notion of disk partitions that may contain sub-partitions. This was deeply unsatisfying to me for several reasons: It implemented a lot of the same functionality that exists in the EFI framework. It involved implementing a GPT partition driver to deal with partition tables inside GELI partitions (GPT detection and support is guaranteed by the EFI spec). The interface between the EFI framework and the custom “providers” framework was awkward. The driver was completely boot1-specific, and exporting it to something like GRUB probably involved a total rewrite. Implementing it within loader was going to involve a lot of code duplication. There was no obvious was to pass keys between boot1, loader, and the kernel. With the issues known, Eric seems pleased with the results of the conversion so far: The GELI driver can be extracted from the FreeBSD codebase without too much trouble. While I was unable to go all the way to the EFI driver model, the only blocker is the bcache code, and once that is resolved, we can have hotplug support in the boot loader! The boot1 and loader codebases are now sharing all the backend drivers, and boot1 has been reduced to one very small source file. An interesting read, looking forward to playing with EFI more in the future! *** Faces of FreeBSD 2016: Michael W. Lucas (https://www.freebsdfoundation.org/blog/faces-of-freebsd-2016-michael-lucas/) On this edition of “Faces of FreeBSD”, Michael W Lucas tells the story of how he got started with FreeBSD After an amusing re-telling of his childhood (The words “Purina Monkey Chow” were mentioned), he then tells us how he got into BSD. His being thrown into the project may sound familiar to many: I came in at 11 PM one night and was told “The DNS administrator just got walked out the door. You’re the new lead DNS administrator. Make those servers work. Good luck.” From there (because he wanted more sleep), he began ripping out the systems that had been failing and waking him up at night. Good-bye UnixWare, Good-bye Solaris, hello BSD! A very amusing read, check it out! *** High Availability with PostgreSQL on FreeBSD (https://www.youtube.com/watch?v=ugct9-Mm7Ls) A talk by Sean Chittenden, who we interviewed previously on episode Episode 95 (http://www.bsdnow.tv/episodes/2015_06_24-bitrot_group_therapy) Explains how to setup Multi Data Center High Availability for PostgreSQL using consul Goes into how consul works, how it does the election, the gossip protocol, etc The HA setup uses DNS Failover, and the pros and cons of that approach are discussed Then he walks through the implementation details, and example configuration *** New FreeBSD i915 testing images (http://www.bsddesktop.com/images/) Still need users to test the Linux Kernel 4.6 DRM update to FreeBSD’s graphics stack Download the test image and write it to a USB stick and boot from it It will not modify your installed system, it runs entirely off of the USB drive Allows you to test the updated drivers without having to install the development branch on your device you can tell them that ATI/AMD support will be coming shortly and that stability has been steadily improving and that I'll do another announcement as soon as I've had a chance to test the newest Xorg bits *** Beastie Bits Comfortable on the CLI: Series Part 1 (https://www.cotcli.com/post/The-Very-Basics/) FreeBSD Booting on the Netgate uFW, a smaller-than-a-raspberry-pi dual port firewall (https://gist.github.com/gonzopancho/8e7df7a826e9a2949b36ed2a9d30312e) Picture of uFW (https://twitter.com/gonzopancho/status/737874921435594753) uFW OpenSSL Benchmarks (https://gist.github.com/gonzopancho/8f20b50487a4f7de56e99448866a147d) ***
143: One small step for DRM, one giant leap for BSD
This week on BSDNow, we have an interview with Matthew Macy, who has some exciting news to share with us regarding the state of graphics This episode was brought to you by Headlines How the number of states affects pf’s performance of FreeBSD (http://blog.cochard.me/2016/05/playing-with-freebsd-packet-filter.html) Our friend Olivier of FreeNAS and BSDRP fame has an interesting blog post this week detailing his unique issue with finding a firewall that can handle upwards of 4 million state table entries. He begins in the article with benchmarking the defaults, since without that we don’t have a framework to compare the later results. All done on his Netgate RCC-VE 4860 (4 cores ATOM C2558, 8GB RAM) under FreeBSD 10.3. “We notice a little performance impact when we reach the default 10K state table limit: From 413Kpps with 128 states in-used, it lower to 372Kpps.” With the initial benchmarks done and graphed, he then starts the tuning process by adjusting the “net.pf.states_hashsize”sysctl, and then playing with the number of states for the firewall to keep. “For the next bench, the number of flow will be fixed for generating 9800 pf state entries, but I will try different value of pf.states_hashsize until the maximum allowed on my 8GB RAM server (still with the default max states of 10k):” Then he cranks it up to 4 million states “There is only 12% performance penalty between pf 128 pf states and 4 million pf states.” “With 10M state, pf performance lower to 362Kpps: Still only 12% lower performance than with only 128 states” He then looks at what this does of pfsync, the protocol to sync the state table between two redundant pf firewalls Conclusions: There need to be a linear relationship between the pf hard-limit of states and the pf.stateshashsize; RAM needed for pf.stateshashsize = pf.stateshashsize * 80 Byte and pf.stateshashsize should be a power of 2 (from the manual page); Even small hardware can manage large number of sessions (it's a matter of RAM), but under too lot's of pressure pfsync will suffer. Introducing the BCHS Stack = BSD, C, httpd, SQLite (http://www.learnbchs.org/) Pronounced Beaches “It's a hipster-free, open source software stack for web applications” “Don't just write C. Write portable and secure C.” “Get to know your security tools. OpenBSD has systrace(4) and pledge(2). FreeBSD has capsicum(4).” “Statically scan your binary with LLVM” and “Run your application under valgrind” “Don't forget: BSD is a community of professionals. Go to conferences (EuroBSDCon, AsiaBSDCon, BSDCan, etc.)” This seems like a really interesting project, we’ll have to get Kristaps Dzonsons back on the show to talk about it *** Installing OpenBSD's httpd server, MariaDB, PHP 5.6 on OpenBSD 5.9 (https://www.rootbsd.net/kb/339/Installing-OpenBSDandsharp039s-httpd-server-MariaDB-PHP-56-on-OpenBSD-59.html) Looking to deploy your next web-stack on OpenBSD 5.9? If so this next article from rootbsd.net is for you. Specifically it will walk you through the process of getting OpenBSD’s own httpd server up and running, followed by MariaDB and PHP 5.6. Most of the setup is pretty straight-forward, the httpd syntax may be different to you, if this is your first time trying it out. Once the various packages are installed / configured, the rest of the tutorial will be easy, walking you through the standard hello world PHP script, and enabling the services to run at reboot. A good article for those wanting to start hosting PHP/DB content (wordpress anyone?) on your OpenBSD system. *** The infrastructure behind Varnish (https://www.varnish-cache.org/news/20160425_website.html) Dogfooding. It’s a term you hear often in the software community, which essentially means to “Run your own stuff”. Today we have an article by PKH over at varnish-cache, talking about what that means to them. Specifically, they recently went through a website upgrade, which will enable them to run more of their own stuff. He has a great quote on what OS they use:“So, dogfood: Obviously FreeBSD. Apart from the obvious reason that I wrote a lot of FreeBSD and can get world-class support by bugging my buddies about it, there are two equally serious reasons for the Varnish Project to run on FreeBSD: Dogfood and jails.Varnish Cache is not “software for Linux”, it is software for any competent UNIX-like operating system, and FreeBSD is our primary “keep us honest about this” platform.“ He then goes through the process of explaining how they would setup a new Varnish-cache website, or upgrade it. All together a great read, and if you are one of the admin-types, you really should pay attention to how they build from the ground up. Some valuable knowledge here which every admin should try to replicate. I can not reiterate the value of having your config files in a private source control repo strongly enough The biggest take-away is: “And by doing it this way, I know it will work next time also.” *** Interview - Matt Macy - mmacy@nextbsd.org (mailto:mmacy@nextbsd.org)Graphics Stack Update (https://lists.freebsd.org/pipermail/freebsd-x11/2016-May/017560.html) News Roundup Followup on packaging base with pkg(8) (https://lists.freebsd.org/pipermail/freebsd-pkgbase/2016-May/000238.html) In spite of the heroic last minute effort by a team of contributors, pkg’d base will not be ready in time for FreeBSD 11.0 There are just too many issues that were discovered during testing The plan is to continue using freebsd-update in the meantime, and introduce a pkg based upgrade mechanism in FreeBSD 11.1 With the new support model for the FreeBSD 11 branch, 11.1 may come sooner than with previous major releases *** FreeBSD Core Election (https://www.freebsd.org/internal/bylaws.html) It is time once again for the FreeBSD Core Election Application period begins: Wednesday, 18 May 2016 at 18:00:00 UTC Application period ends: Wednesday, 25 May 2016 at 18:00:00 UTC Voting begins: Wednesday, 25 May 2016 at 18:00:00 UTC Voting ends: Wednesday, 22 June 2016 at 18:00:00 UTC Results announced Wednesday, 29 June 2016 New core team takes office: Wednesday, 6 July 2016 As of the time I was writing these notes, 3 hours before the application deadline, the candidates are: Allan Jude: Filling in the potholes Marcelo Araujo: We are not vampires, but we need new blood. Baptiste Daroussin (incumbent): Keep on improving Benedict Reuschling: Learn and Teach Benno Rice: Revitalising The Community Devin Teske: Here to help Ed Maste (incumbent): FreeBSD is people George V. Neville-Neil (incumbent): There is much to do… Hiroki Sato (incumbent): Keep up with our good community and technical strength John Baldwin: Ready to work Juli Mallett: Caring for community. Kris Moore: User-Focused Mathieu Arnold: Someone ask for fresh blood ? Ollivier Robert: Caring for the project and you, its developers The deadline for applications is around the time we finish recording the live show We welcome any of the candidates to schedule an interview in the next few weeks. We will make an attempt to hunt many of them down at BSDCan as well. *** Wayland/Weston with XWayland works on DragonFly (http://lists.dragonflybsd.org/pipermail/users/2016-May/249620.html) We haven’t talked a lot about Wayland on BSD recently (or much at all), but today we have a post from Peter to the dragonfly mailing list, detailing his experience with it. Specifically he talks about getting XWayland working, which provides the compat bits for native X applications to run on WayLand displays. So far on the working list of apps: “gtk3: gedit nautilus evince xfce4: - xfce4-terminal - atril firefox spyder scilab” A pretty impressive list, although he said “chrome” failed with a seg-fault This is something I’m personally interested in. Now with the newer DRM bits landing in FreeBSD, perhaps it’s time for some further looking into Wayland. Broadcom WiFi driver update (http://adrianchadd.blogspot.ca/2016/05/updating-broadcom-softmac-driver-bwn-or.html) In this blog post Adrian Chadd talks about his recent work on the bwn(4) driver for Broadcom WiFi chips This work has added support for a number of older 802.11g chips, including the one from 2009-era Macbooks Work is ongoing, and the hope is to add 802.11n and 5ghz support as well Adrian is mentoring a number of developers working on embedded or wifi related things, to try to increase the projects bandwidth in those areas If you are interested in driver development, or wifi internals, the blog post has lots of interesting details and covers the story of Adrian’s recent adventures in bringing the drivers up *** Beastie Bits The Design of the NetBSD I/O Subsystems (2002) (http://arxiv.org/abs/1605.05810) ZFS, BTRFS, XFS, EXT4 and LVM with KVM – a storage performance comparison (http://www.ilsistemista.net/index.php/virtualization/47-zfs-btrfs-xfs-ext4-and-lvm-with-kvm-a-storage-performance-comparison.html?print=true) Swift added to FreeBSD Ports (http://www.freshports.org/lang/swift/) misc@openbsd: 'NSA addition to ifconfig' (http://marc.info/?l=openbsd-misc&m=146391388912602&w=2) Papers We Love: Memory by the Slab: The Tale of Bonwick's Slab Allocator (http://paperswelove.org/2015/video/ryan-zezeski-memory-by-the-slab/) Feedback/Questions Lars - Poudriere (http://pastebin.com/HRRyfxev) Warren - .NET (http://pastebin.com/fESV1egk) Eddy - Sys Init (http://pastebin.com/kQecpA1X) Tim - ZFS Resources (http://pastebin.com/5096cGXr) Morgan - Ports and Kernel (http://pastebin.com/rYr1CDcV) ***
142: Diving for BSD Perls
This week on the show, we have all the latest news and stories! Plus an interview with BSD developer Alfred Perlstein, that you This episode was brought to you by Headlines The May issus of BSDMag is now out (https://bsdmag.org/download/reusing_openbsd/) GhostBSD Reusing OpenBSD's arc4random in multi-threaded user space programs Securing VPN's with GRE / Strongswan Installing XFCE 4.12 on NetBSD 7 Interview with Fernando Rodriguez, the co-founder of KeepCoding *** A rundown of the FPTW^XEXT.1 security reqiurement for General Purpose Operating Systems by the NSA (http://blog.acumensecurity.net/fpt_wx_ext-1-a-rundown/) NIST/NSA Validation Scheme Report (https://www.commoncriteriaportal.org/files/ppfiles/pp_os_v4.1-vr.pdf) The SFR or Security Functional Requirement requires that; "The OS shall prevent allocation of any memory region with both write and execute permissions except for [assignment: list of exceptions]." While nearly all operating systems currently support the use of the NX bit, or the equivalent on processors such as SPARC and ARM, and will correctly mark the stack as non-executable, the fact remains that this in and of itself is deemed insufficient by NIST and NSA. OpenBSD 5.8, FreeBSD, Solaris, RHEL, and most other Linux distro have failed. HardenedBSD passes all three tests out of the box. NetBSD will do so with a single sysctl tweak. Since they are using the PaX model, anything else using PaX, such as a grsecurity-enabled Linux distribution pass these assurance activities as well. OpenBSD 5.9 does not allow memory mapping due to W^X being enforced by the kernel, however the kernel will panic if there are any attempts to create such mappings. *** DistroWatch reviews new features in FreeBSD 10.3 (https://distrowatch.com/weekly.php?issue=20160516#freebsd) DistroWatch did a review of FreeBSD 10.3 They ran into a few problems, but hopefully those can be fixed An issue with beadm setting the canmount property incorrectly causing the ZFS BE menu to not work as expected should be resolved in the next version, thanks to a patch from kmoore The limitations of the Linux 64 support are what they are, CentOS 6 is still fairly popular with enterprise software, but hopefully some folks are interested in working on bringing the syscall emulation forward In a third issue, the reviewer seemed to have issues SSHing from inside the jail. This likely has to do with how they got a console in the jail. I remember having problems with this in the past, something about a secure console. *** BSD Unix: Power to the people, from the code (https://www.salon.com/2000/05/16/chapter_2_part_one/) Salon.com has a very long article, chronicling much of the history behind BSD UNIX. It starts with detailing the humble origins of BSD, starting with Bill Joy in the mid-70’s, and then goes through details on how it rapidly grew, and the influence that the University of Berkeley had on open-source. “But too much focus on Joy, a favorite target for business magazine hagiography, obscures the larger picture. Berkeley’s most important contribution was not software; it was the way Berkeley created software. At Berkeley, a small core group — never more than four people at any one time — coordinated the contributions of an ever-growing network of far-flung, mostly volunteer programmers into progressive releases of steadily improving software. In so doing, they codified a template for what is now referred to as the “open-source software development methodology.” Put more simply, the Berkeley hackers set up a system for creating free software.” The article goes on to talk about some of the back and forth between Linux and BSD, and why Linux has captured more of the market in recent years, but BSD is far from throwing in the towel. “BSD patriots argue that the battle is far from over, that BSD is technically superior and will therefore win in the end. That’s for the future to determine. What’s indisputable is BSD’s contribution in the past. Even if, by 1975, Berkeley’s Free Speech Movement was a relic belonging to a fast-fading generation, on the fourth floor of Evans Hall, where Joy shared an office, the free-software movement was just beginning.” An excellent article (If a bit long), but well worth your time to understand the origins of what we consider modern day BSD, and how the University of Berkley helped shape it. *** iXsystems (http://ixsystems.com) #ServerEnvy: It's over 10,000 Terabytes! (https://www.ixsystems.com/blog/serverenvy-10000-terabytes/) *** Interview - Alfred Perlstein - alfred@freebsd.org (mailto:alfred@freebsd.org) / @splbio (https://twitter.com/splbio) Using BSD for projects *** News Roundup .NET framework ported to NetBSD (https://github.com/dotnet/coreclr/pull/4504/files) This pull request adds basic support for the .NET framework on NetBSD 7.x amd64 It includes documentation on how to get the .NET framework installed It uses pkgsrc to bootstrap the required tools pkgsrc-wip is used to get the actual .NET framework, as porting is still in progress The .NET Core-CLR is now available for: FreeBSD, Linux, NetBSD, and OS X *** OpenBSD SROP mitigation – call for testing (https://marc.info/?l=openbsd-tech&m=146281531025185&w=2) A new technique for exploiting flaws in applications and operating systems has been developed, called SROP “we describe Sigreturn Oriented Programming (SROP), a novel technique for exploits and backdoors in UNIX-like systems. Like return-oriented programming (ROP), sigreturn oriented programming constructs what is known as a ‘weird machine’ that can be programmed by attackers to change the behavior of a process. To program the machine, attackers set up fake signal frames and initiate returns from signals that the kernel never really delivered. This is possible, because UNIX stores signal frames on the process’ stack.” “Sigreturn oriented programming is interesting for attackers, OS developers and academics. For attackers, the technique is very versatile, with pre-conditions that are different from those of existing exploitation techniques like ROP. Moreover, unlike ROP, sigreturn oriented programming programs are portable. For OS developers, the technique presents a problem that has been present in one of the two main operating system families from its inception, while the fixes (which we also present) are non-trivial. From a more academic viewpoint, it is also interesting because we show that sigreturn oriented programming is Turing complete.” Paper describing SROP (http://www.cs.vu.nl/~herbertb/papers/srop_sp14.pdf) OpenBSD has developed a mitigation against SROP “Utilizing a trick from kbind(2), the kernel now only accepts signal returns from the PC address of the sigreturn(2) syscall in the signal trampoline. Since the signal trampoline page is randomized placed per process, it is only known by directly returning from a signal handler.” “As well, the sigcontext provided to sigreturn(2) now contains a magic cookie constructed from a per-process cookie XOR'd against the address of the signal context.” This is just a draft of the patch, not yet considered production quality *** Running Tor in a NetBSD rump unikernel (https://github.com/supradix/rumprun-packages/tree/33d9cc3a65a39e32b4bc8034c151a5d7e0b89f66/tor) We’ve talked about “rump” kernels before, and also Tor pretty frequently, but this new github project combines the two! Specifically, this set of Makefile and scripts will prep a system to run Tor via the Unikernel through Qemu. The script mainly describes how to do the initial setup on Linux, using iptables, but could easily be adapted to a BSD if somebody wants to do so. (Send them a pull request with the instructions!) All in all, this is a fascinating way to run a Tor node or relay, in the most minimal operating environment possible. *** An update on SSH protocol 1 ("we're most of the way towards fully deprecating SSH protocol 1" (http://lists.mindrot.org/pipermail/openssh-unix-dev/2016-May/035069.html) Damien Miller has given us an update on the status of the “SSH protocol 1”, and the current plans to deprecate it in an upcoming version of openssh. “We've had this old protocol in various stages of deprecation for almost 10 years and it has been compile-time disabled for about a year. Downstream vendors, to their credit, have included this change in recent OS releases by shipping OpenSSH packages that disable protocol 1 by default and/or offering separate, non-default packages to enable it. This seems to have proceeded far more smoothly than even my most optimistic hopes, so this gives us greater confidence that we can complete the removal of protocol 1 soon. We want to do this partly to hasten the demise of this cryptographic trainwreck, but also because doing so removes a lot of legacy code from OpenSSH that inflates our attack surface. Having it gone will make our jobs quite a bit easier as we maintain and refactor.” The current time-line looks like removing server-size protocol 1 support this August after OpenSSH 7.4 is released, leaving client-side disabled. Then a year from now (June 2017) all protocol 1 code will be removed. Beastie Bits Last day to get your BSDNow Shirts! Order now, wear at BSDCan! (https://teespring.com/bsdnow) Move local government (Austin TX) from Microsoft Windows (incl. Office) to Linux and/or PC-BSD (https://github.com/atxhack4change/2016-project-proposals/issues/15) Plan9 boot camp is back... and already at capacity. Another opportunity may come in September (http://lists.nycbug.org/pipermail/talk/2016-May/016642.html) Smaller is better - building an openbsd based router (https://functionallyparanoid.com/2016/04/22/smaller-is-better/) Baby Unix (https://i.redditmedia.com/KAjSscL9XOUdpIEWBQF1qi3QMr7zWgeETzQM6m3B4mY.jpg?w=1024&s=e8c08a7d4c4cea0256adb69b1e7c1887) Security Update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:19.sendmsg.asc) & Another security update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:18.atkbd.asc) Feedback/Questions Eric - The iX experience (http://pastebin.com/ZknTuKGv) Mike - Building Ports (http://pastebin.com/M760ZmHQ) David - ZFS Backups (http://pastebin.com/Pi0AFghV) James - BSD VPS (http://pastebin.com/EQ7envez) Rich - ZFS Followup (http://pastebin.com/p0HPDisH) ***
141: BSD Likes Ike!
This week on the show, we have all the latest news and stories! Plus we’ll be hearing more about OpnSense from the man himself, Ike! This episode was brought to you by Headlines Regarding Embargoes (http://www.tedunangst.com/flak/post/regarding-embargoes) Our buddy TedU has a great thought piece today on the idea of “embargoes” for security advisories. This all stemmed from a recent incident with LibreSSL patches from embargoed OpenSSL vulns, that accidentally got committed too early. Ted makes a pretty good case on the difficulties of having embargos, and maybe the reason there shouldn’t be. Couple of quotes to give you a taste: “There are several difficulties maintaining embargoes. Keeping secrets is against human nature. I don’t want to be the one who leaks, but if I see something that looks like the secret is out, it’s a relief to be able to speak freely. There is a bias towards recognizing such signs where they may not really exist. (Exacerbated by broad embargoes where some parts leak but other parts don’t. It’s actually very hard to tell what’s not publicly known when you know everything.) The most thorough embargo and release timeline reconstruction is the heartbleed timeline. It’s another great case study. Who exactly decided who were the haves and have nots? Was it determined by who needed to know or who you needed to know? Eventually the dam started to crack.” “When Cloudflare brags that they get advance notice of vulnerabilities, attracting more customers, and therefore requiring even more early access, how are smaller players to compete? What happens if you’re not big enough to prenotify? Sometimes vulnerabilities are announced unplanned. Zero day cyber missiles are part of our reality, which means end users don’t really have the luxury of only patching on Tuesday. They need to apply patches when they appear. If applying patches at inconvenient times is a problem, make it not a problem. Not really a gripe about embargoes per se, but the scheduled timing of coordinated release at the end of the embargo is catering to a problem that shouldn’t exist.” I will admit that CloudFlare bragging around Heartbleed was upsetting The biggest issue here is the difficulty with coordinating so many open source projects, which are often done by volunteers, in different countries and time zones The other issue is determining when the secret is “out of the bag” *** MAJOR ABI BREAK: csu, ld.so, libc, libpthread update (http://www.openbsd.org/faq/current.html#r20160507) OpenBSD warns those following the -current (development) branch to be careful as they upgrade because of a major ABI break that will result in applications not working “Handling of single-threaded programs is now closer to multi-threaded, with ld.so and libc.a doing thread information base (TIB) allocation. Threaded programs from before the 2016/03/19 csu and ld.so update will no longer run. An updated ld.so must be built and installed before running make build.” A special note for those on PowerPC: “PowerPC has been updated to offset the TIB from the hardware register. As a result, all threaded programs are broken until they have been rebuilt with the new libc and libpthread. perl must be built after building the libraries and before building the rest of base.” “The definitions of environ and __progname for dynamically linked programs have been moved from the C startup code to ld.so(1). An updated ld.so must be built and installed before running make build.” The link provides instructions on how to update your system properly *** How to install FreeBSD 10.3 on VMWare Workstation 12 Pro (http://random-notes-of-a-sysadmin.blogspot.be/2016/04/howto-install-freebsd-103-on-vmware.html) This tutorial starts at the very basics, running through the FreeBSD installer But then it goes on to configuring the machine specifically for VMWare After the system has been booted, the tutorial walks through installing the VMWare tools Then networking is configured in both VMWare and FreeBSD A small hack is required to make the VMWare tools startup script wait until the network is up A very nice tutorial for people using VMWare I am working on a patch to bsdinstall to ensure that the swap partition is put before the main partition, so it can more easily be resized if you later decide you need more space in your VM the camcontrol reprobe subcommand has been added (https://svnweb.freebsd.org/base?view=revision&revision=299371), “This makes it possible to manually force updating capacity data after the disk got resized. Without it it might be necessary to reboot before FreeBSD notices updated disk size under eg VMWare.” *** BSD Router project releases v1.59 (https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/1.59/) We’ve talked about the BSD Router project a bit in the past, but today we have a brand new release to bring to you. For those who don’t remember, the BSDrp is a router aimed at replacing more of your big-commercial type systems. First up in the new hotness, we have it based upon recently released FreeBSD 10.3! In addition, there is a new package: New package: mlvpn (aggregated network links in order to benefit from the bandwidth of multiple links) Other packages have gotten a bump with this release as well: bsnmp-ucd to 0.4.2 dma to 0.11 dmidecode to 3.0 exabgp to 3.4.15 iperf3 to 3.1.2 monit to 5.17 mpd5 to 5.8 openvpn to 2.3.10 python to 2.7.11 quagga to 1.0.20160315 strongswan to 5.4.0 What are you waiting for? Amd64 and i386 images are ready for you to download now. Interview - Isaac (.Ike) Levy - See Ike again at SEMIBug in Troy, Michigan on May 17th (http://semibug.org/) *** News Roundup Tredly - Prebuilt containers on FreeBSD (https://github.com/tredly/) Discussion regarding its GPLv3 licensing (https://www.reddit.com/r/freebsd/comments/4gggw8/introducing_tredly_containers_for_unix_freebsd/) A new “container” solution called “Trendly” has started making some news around various tech sites. In particular, this new project uses FreeBSD as its base OS and jail functionality in the backend. Their solution seems based around the idea of shipping containers as manifests, such as lists of packages to install and configuration knobs. The project is still rather new, and we’ll be keeping an eye on it for the future. One notable change already though, it was (for some reason) released under GPLv3. Understandably this caused quite a ruckus with various folks in the community, since it’s built specifically on BSD. Since this, the code has been re-licensed as MIT, which is far more in the spirit of a traditional BSD license. *** NVMe driver added to NetBSD - ported from OpenBSD (https://www.netbsd.org/changes/changes-8.0.html#nvme%284%29) NetBSD has gained support for Non-Volatile Memory Express, the new standard for PCIe attached Flash Memory The change of interface from SATA to NVMe offers a number of advantages, mostly, it doesn’t require the device to pretend to be a spinning disk One of the biggest advantages is that it supports completing multiple operations at once, with the Intel hardware I have tested, 63 I/Os can happen concurrently, so a very large queue depth is required to keep the device busy. The 64th I/O channel is reserved for administrative commands, to keep them from being delayed by the large queue depth The device I tested could read at 3800 MB/s, and write 1700MB/s, something that wouldn’t be possible with a normal SSD It is interesting that NetBSD took the NVMe support from OpenBSD, whereas the FreeBSD implementation was contributed directly by Intel This may have to do with that fact that OpenBSD’s device model is closer to that of NetBSD Commit Log (http://mail-index.netbsd.org/source-changes/2016/05/01/msg074367.html) *** New BSDNow T-Shirts (https://teespring.com/bsdnow) By popular demand, we have created a more subtle BSDNow shirt Featuring only the smallish BSDNow logo over the left breast Available in a number of styles (T-Shirt, Women’s T-Shirt, Long Sleeve, and Hoodie) as well as a number of colours: Black, Blue, Grey, and White The hope is that enough orders come though so we can get them shipped in and your sweaty little hands in time for BSDCan. (I’ll be wearing mine, will you B...SD?) If you still want one of our now-famous “The Usual BSD’s” t-shirts, you can also indicate your interest here, and once 10 or more shirts are ordered, a reprint will happen automatically (https://teespring.com/bsd105) *** PC-BSD 11-CURRENT with Package Base (http://lists.pcbsd.org/pipermail/testing/2016-May/010616.html) Looking for a way to play with the new FreeBSD base package system? This month’s PC-BSD -CURRENT image now used packages for base system installation, and is asking for testers to help find bugs. Known issues so far: setuid binaries (Fix in works) Missing tzone files Distrib packages If all that doesn’t scare you away, then give it a whirl! Upgrades for previous APRIL images are now online also. *** BeastieBits HardenedBSD + LibreSSL (https://hardenedbsd.org/article/shawn-webb/2016-05-05/libressl-hardenedbsd-base) Michael Dexter's talk at LFNW 2016 is the 2nd highest youtube views from this years conference (https://www.youtube.com/watch?v=6k1Mf0c6YW8) Why OpenBSD is important to me (http://ggr.com/why-openbsd-is-important-to-me.html) Study of nginx-1.9.12 performance/latency on DragonFlyBSD-g67a73 (http://lists.dragonflybsd.org/pipermail/users/2016-May/249581.html) Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net (https://www.geeklan.co.uk/?p=2109) The interesting story of how IllumOS syscalls work (http://zinascii.com/2016/the-illumos-syscall-handler.html) The BeaST is the FreeBSD based dual-controller reliable storage system concept with aim to implement ZFS and in-memory cache. (https://mezzantrop.wordpress.com/portfolio/the-beast/) Francois Tigeot updates the drm/i915 driver to match what’s in Linux kernel 4.3 (http://lists.dragonflybsd.org/pipermail/commits/2016-May/500352.html) FreeBSD is working on the update to Linux Kernel 4.6, we may finally get ahead of Dragonfly! (https://twitter.com/ed_maste/status/730450314889924608) Feedback/Questions Oskar - Torrent Jail (http://pastebin.com/RT7tVtQ7) Shane - ZFS Delete (http://pastebin.com/VkpMeims) Adam - Zimbra Port (http://pastebin.com/MmQ00Sv1) Ray - PC-BSD - FrameBuffer (http://pastebin.com/Xx9TkX7A) Richard - ZFS Backups (http://pastebin.com/ncYxqpg3) ***
140: Tracing it back to BSD
This week on BSDNow, Allan is back in down from Europe! We’ll get to hear some of his wrap-up and get caught up on the latest BSD This episode was brought to you by Headlines FreeBSD Quarterly Report (http://www.freebsd.org/news/status/report-2016-01-2016-03.html) This quarterly status report starts with a rather interesting introduction by Warren Block ASLR Porting CEPH to FreeBSD RCTL I/O Rate Limiting The Graphics Stack on FreeBSD (Haswell is in, work is progressing on the next update) CAM I/O Scheduler NFS Server updates, working around the 16 group limit, and implementing pNFS, allowing NFS to scale beyond a single server Static Analysis of the FreeBSD Kernel with PVS Studio PCI-express HotPlug GitLab Port committed! WITHFASTDEPEND and other improvements to the FreeBSD build system Lots of other interesting stuff *** A Prog By Any Other Name (http://www.tedunangst.com/flak/post/a-prog-by-any-other-name) Ted Unangst looks at what goes into the name of a program “Sometimes two similar programs are really the same program with two names. For example, grep and egrep are two commands that perform very similar functions and are therefore implemented as a single program. Running ls -i and observing the inode number of each file will reveal that there is only one file. Calling the program egrep is a shorthand for -E and does the same thing.” So BSD provides __progname in libc, so a program can tell what its name is But, what if it has more than one name? “In fact, every program has three names: its name in the filesystem, the name it has been invoked with, and whatever it believes its own name to be.” Of course it is not that easy. “there’s another set of choices for each name, the full path and the basename” “It’s even possible on some systems for argv[0] to be NULL.” He then goes on to rename doas (the OpenBSD light replacement for sudo) to banana and discuss what happens “On that note, another possible bug is to realize that syslog by default uses progname. A user may be able to evade log monitoring by invoking doas with a different name. (Just fixed.)” Another interesting article from our friend Ted *** FreeBSD (https://summerofcode.withgoogle.com/organizations/4892834293350400/) and NetBSD (https://summerofcode.withgoogle.com/organizations/6246531984261120/) Google Summer of Code projects have been announced Some FreeBSD highlights: Add SCSI passthrough to CTL (share an optical drive via iSCSI) Add USB target mode driver based on CTL (share a USB device via iSCSI) API to link created /dev entries to sysctl nodes Implement Ethernet Ring Protection Switching (ERPS) HD Audio device model in userspace for bhyve Some NetBSD highlights: Implement Ext4fs support in ReadOnly mode NPF and blacklistd web interface Port U-Boot so it can be compiled on NetBSD Split debug symbols for pkgsrc builds *** libressl - more vague priomises (http://www.tedunangst.com/flak/post/libressl-more-vague-promises) We haven’t had a Ted U article on the show as of late, however this week we get several! In his next entry “LibreSSL, more vague promises” He then goes into some detail on what has happened with LibreSSL in the past while, as well as future plans going forward. “With an eye to the future, what new promises can we make? Some time ago I joked that we only promised to make a better TLS implementation, not a better TLS. Remains true, but fortunately there are people working on that, too. TLS 1.3 support is on the short term watchlist. The good news is we may be ahead of the game, having already removed compression. How much more work can there be?” “LibreSSL integrated the draft chacha20-poly1305 construction from BoringSSL. The IETF has since standardized a slightly different version because if it were the same it wouldn’t be different. Support for standard variant, and the beginning of deprecation for the existing code, should be landing very shortly. Incidentally, some people got bent out of shape because shipping chacha20 meant exposing non IANA approved numbers to Internet. No promises that won’t happen again.” *** Interview - Samy Al Bahra - @0xF390 (https://twitter.com/0xF390) Backtrace *** News Roundup systrace(1) is removed for OpenBSD 6.0 (http://marc.info/?l=openbsd-cvs&m=146161167911029&w=2) OpenBSD has removed systrace, an older mechanism for limiting what syscalls an application can make It is mostly replaced by the pledge() system OpenBSD was the first implementation, most others have been unmaintained for some time The last reported Linux version was for kernel 2.6.1 NetBSD removed systrace in 2007 *** pfSense Video Series: Comprehensive Guide To pfSense 2.3 (https://www.youtube.com/playlist?list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk) A series of videos (11 so far), about pfSense Covers Why you would use it, how to pick your hardware, and installation Then the series covers some networking basics, to make sure you are up to speed before configuring your pfSense Then a comprehensive tour of the WebUI Then goes on to cover graphing, backing up and restoring configuration There are also videos on running DHCP, NTP, and DNS servers *** DuckDuckGo announces its 2016 FOSS Donations (https://duck.co/blog/post/303/2016-foss-donations-announcement) The theme is “raising the standard of trust online” Supported projects include: OpenBSD Foundation announces DuckDuckGo as a Gold Sponsor (http://undeadly.org/cgi?action=article&sid=20160503085227&mode=expanded) the Freedom of the Press Foundation for SecureDrop the Freenet Project the CrypTech Project the Tor Project Fight for the Future for Save Security Open Source Technology Improvement Fund for VeraCrypt (based on TrueCrypt) Riseup Labs for LEAP (LEAP Encryption Access Project) GPGTools for GPGMail *** Larry the BSD Guy hangs up his hat at FOSS Force (http://fossforce.com/2016/04/bsd-linuxfest-northwest/) After 15 years, Larry the BSD Guy has decided to hang it up, and walk into the sunset! (Figuratively of course) After wrapping up coverage of recent LinuxFest NorthWest (Which he didn’t attend), Larry has decided it’s time for a change and is giving up his column over at FOSS Force, as well as stepping away from all things technical. His last write-up is a good one, and he has some nice plugs for both Dru Lavigne and Michael Dexter of the BSD community. He will be missed, but we wish him all the luck with the future! He also puts out the plug that FOSS Force will be needing a new columnist in the near future, so if you are interested please let them know! *** Beastie Bits If you sponsored “FreeBSD Mastery: Advanced ZFS”, check your mail box (http://blather.michaelwlucas.com/archives/2648) pkg-1.7.0 is an order of magnitude slower than pkg-1.6.4 (https://marc.info/?l=freebsd-ports&m=146001143408868&w=2) -- Caused by a problem not in pkg LinuxFest Northwest 2016 Recap (https://www.ixsystems.com/blog/linuxfest-northwest-2016/) Dru Lavigne's 'Doc like an Egyption' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/doc-egyptian) Michael Dexters' 'Switching to BSD from Linux' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/devil-details-switching-bsd-linux) Michael Dexters' 'Secrets to enduring user groups' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/20-year-and-counting-secrets-enduring-user-groups) January issue of Freebsd Journal online for free (https://www.freebsdfoundation.org/journal/) Ghost BSD releases 10.3 Alpha1 for testing (http://ghostbsd.org/10.3_alpha1) EuroBSDcon 2016 - Call for Papers - Dealine: May 8th (https://www.freebsdnews.com/2016/04/15/eurobsdcon-2016-call-for-papers/) KnoxBUG Initial Meeting (http://www.knoxbug.org/content/knoxbug-maiden-voyage) Photos, slides, and videos from the Open Source Data Center Conference (https://www.netways.de/en/events_trainings/osdc/archive/osdc2016/) *** Feedback/Questions Mohammad - Replication (http://pastebin.com/KDnyWf6Y) John - Rolling new packages (http://pastebin.com/mAbRwbEF) Clint - Unicast (http://pastebin.com/BNa6pyir) Bill - GhostBSD (http://pastebin.com/KDjS2Hxa) Charles - BSD Videos (http://pastebin.com/ABUUtzWM) ***
139: Cheri-picking BSD
This week, Allan is out of town, but since when has that ever stopped us from bringing you a new episode of BSDNow? We have news, This episode was brought to you by Headlines Unix's file durability problem (https://utcc.utoronto.ca/~cks/space/blog/unix/FileSyncProblem) Another article by Chris Siebenmann from the University of Toronto This time, the issue was a lost comment on his Python based blog which uses files on disk rather than a database After an unexpected restart of the system, a recently posted comment no longer existed The post goes on to investigate what the ‘right way’ to ensure file durability is The answer, as you might expect, is “it depends…” Normally, fsync() should work, but it seems with ext4 and some other file systems, you must also fsync() the directory where the file was created, or it might not be possible to find the file after a crash Do you need to fsync() the parent of that directory too? Then what is fdatasync() for? What about just calling sync()? “One issue is that unlike many other Unix API issues, it's impossible to test to see if you got it all correct and complete. If your steps are incomplete, you don't get any errors; your data is just silently sometimes at risk. Even with a test setup to create system crashes or abrupt power loss (which VMs make much easier), you need uncommon instrumentation to know things like if your OS actually issued disk flushes or just did normal buffered writes. And straightforward testing can't tell you if what you're doing will work all the time, because what is required varies by Unix, kernel version, and the specific filesystem involved.” Second post by author: How I'm trying to do durable disk writes (https://utcc.utoronto.ca/~cks/space/blog/python/HowISyncDataDWiki) Additional Discussion on Hacker News (https://news.ycombinator.com/item?id=11511269) The discussion on HN also gets into AIO and other more complicated facilities, but even those seem to be vague about when your data is actually safe At least ZFS ensures you never get half of your new data, and half of your old data. *** Build a FreeBSD 10.3-release Openstack Image with bsd-cloudinit (https://raymii.org/s/tutorials/FreeBSD_10.3-release_Openstack_Image.html) Are you using FreeBSD and OpenStack or would you like to be? We next have a great tutorial which explains the ins-and-outs of doing exactly that. Remy van Elst brings us a great walkthrough on his site on how to get started, and hint it involves just a few ‘pip’ commands. After getting the initial Python tools bootstrapped, next he shows us how to save our OpenStack settings in a sourceable shell command, which comes in handy before doing admin on a instance. Next the ‘glance’ and ‘cinder’ tools are used to upload the target OS ISO file and then create a volume for it to install onto. Next the VM is started and some specific steps are outlined on getting FreeBSD 10.3 installed into the instance. It includes some helpful hints as how to fix a mountroot error, if you installed to ada0, but need to mount via vtdb0 instead now. After the installation is finished, the prep for ‘cloudinit’ is done, and the resulting image is compressed and made ready for deployment. We’ve kinda stepped through some of the more gory steps here, but if OpenStack is something you work with, this tutorial should be at the top of your “must read” list. *** Undeadly and HTTPS (http://undeadly.org/cgi?action=article&sid=20160411201504) Undeadly, the OpenBSD journal, is thinking of moving to HTTPS only In order to do this, they would like some help rewriting part of the site Currently, when you login to post comments, this is done over HTTPS, but to an stunnel instance running a custom script that gives you a cookie, and sends you back to the non-HTTPS site They would like to better integrate the authentication system, and otherwise improve the code for the site There is some pushback as well, questioning whether it makes sense to block users who are unable to use HTTPS for one reason or another I think it makes sense to have the site default to HTTPS, but, maybe HTTPS only doesn’t make sense. There is nothing private on the site, other than the authentication system which is optional, not required to post a comment. There is also some discussion about the code for the site, including the fact that when the code was released, the salt for the password database was included This is not actually a security problem, but the discussion may be interesting to some viewers *** FreeBSD Journal March/April Edition (https://www.freebsdfoundation.org/journal/browser-based-edition/) The next issue of the FreeBSD Journal is here, and this time it is about Teaching with Operating Systems In addition to the usual columns, including: svn update, the ports report, a conference report from FOSDEM, a meetup report from PortsCamp Taipei, A book review of "The Algorithm Design Manual", and the Events Calendar; there are a set of feature articles about teaching Teaching with FreeBSD through Tracing, Analysis, and Experimentation CHERI: Building a foundation for secure, trusted computing bases A brief history of Fast Filesystems There is also an interview with Gleb Smirnoff, a member of the Core team, release engineering, and the deputy security officer, as well as a senior software developer at Netflix Get the latest issue from your favourite mobile store, or the “Desktop Edition” directly in your browser from the FreeBSD Foundation’s website *** Interview - Brooks Davis - brooks@FreeBSD.org (mailto:brooks@FreeBSD.org) / @brooksdavis (https://twitter.com/brooksdavis) CHERI and Capabilities *** TrueNAS Three-Peats!!! (https://www.ixsystems.com/blog/truenas-three-peats/) News Roundup UbuntuBSD Is Looking To Become An Official Ubuntu Flavor (http://linux.softpedia.com/blog/ubuntubsd-is-looking-to-become-an-official-ubuntu-flavor-502746.shtml) You may recall a few weeks back that we were a bit surprised by the UbuntuBSD project and its longevity / goals. However the project seems to be pushing forward, with news on softpedia.com that they are now seeking to become an ‘official’ Ubuntu Flavor. They’ve already released a forth beta, so it seems the project currently has some developers pushing it forward: "I would like to contribute all my work to Ubuntu Community and, if you think it is worthy, make ubuntuBSD an official Ubuntu project like Xubuntu or Edubuntu," said Jon Boden. "If you're interested, please let me know how would you like me to proceed." It's Just Bits (http://blog.appliedcompscilab.com/its_just_bits/index.html) We have next an interesting blog post talking about the idea that “It’s just all bits!” The author then takes us down the idea of no matter how old or mysterious the code may be, in the end it is ending up as bits arranged a certain way. Then the article transitions and takes us through the idea that old bits, and bits that have grown too large should often be good candidates for replacement by “simpler” bits, using OpenBSD as an example. “The OpenBSD community exemplifies this in many ways by taking existing solutions and simplifying them. Processing man pages is as old as Unix, and even in the 21st century OpenBSD has taken the time to rewrite the existing solution to be simpler and safer. It's just bits that need to be turned into other bits. Similarly, OpenBSD has introduced doas as an alternative to sudo. While not replacing sudo entirely, doas makes the 99.99% case of what people use sudo for easier and safer. They are just bits that need to be authenticated. “ All in all, a good read, and it reinforces the point that nothing is really truly “finished”. As computing advances and new technologies / practices are made available, sometimes it makes a lot of sense to go back and re-write things in order to simplify the complexity that has snuck in over time. *** Disk IO limiting is coming to FreeBSD (https://lists.freebsd.org/pipermail/svn-src-head/2016-April/084288.html) A much requested feature for both Jails and VM’s on FreeBSD has just landed with experimental support in -HEAD, Disk IO limiting! The Commit message states as follows: “Add four new RCTL resources - readbps, readiops, writebps and writeiops, for limiting disk (actually filesystem) IO. Note that in some cases these limits are not quite precise. It's ok, as long as it's within some reasonable bounds. Testing - and review of the code, in particular the VFS and VM parts - is very welcome.” Well, what are you waiting for? This is a fantastic new feature which I’m sure will get incorporated into other tools for controlling jails and VM’s down the road. If you give it a spin, be sure to report back bugs so they can get quashed in time for 11. *** BeastieBits PC-BSD 10.3 Is the Last in the Series, PC-BSD 11.0 Arrives Later This Year (http://news.softpedia.com/news/pc-bsd-10-3-is-the-last-in-the-series-pc-bsd-11-0-arrives-later-this-year-502570.shtml) ASLR now on by default in NetBSD amd64 (http://mail-index.netbsd.org/source-changes/2016/04/10/msg073939.html) Daniel Bilik's fix for hangs on Baytrail (http://lists.dragonflybsd.org/pipermail/users/2016-April/228682.html) Don’t forget about PGCon 2016 (http://www.pgcon.org/2016/) Get your paper in for EuroBSDCon 2016, deadline is May 8th (https://2016.eurobsdcon.org/call-for-papers/) Feedback/Questions John - Destroy all Dataset (http://pastebin.com/QdGWn0TW) Thomas - Misc Questions (http://pastebin.com/43YkwBjP) Ben - ZFS Copy (http://pastebin.com/gdi3pswe) Bryson - SysV IPC (http://pastebin.com/E9n938D1) Drin - IPSEC (http://pastebin.com/bgGTmbDG) ***
138: Rushing into BSD
This week on the show, we will be talking to Benedict Reushling about his role with the FreeBSD foundation and the journey that took him This episode was brought to you by Headlines HardenedBSD introduces full PIE support (https://hardenedbsd.org/article/shawn-webb/2016-04-15/introducing-full-pie-support) PIE base for amd64 and i386 Only nine applications are not compiled as PIEs Tested PIE base on several amd64 systems, both virtualized and bare metal Hoped to be to enabled it for ARM64 before or during BSDCan. Shawn will be bringing ten Raspberry Pi 3 devices (which are ARM64) with to BSDCan, eight of which will be given out to lucky individuals. “We want the BSD community to hack on them and get ARM64/Aarch64 fully functional on them.” *** Lessons learned from 30 years of MINIX (http://m.cacm.acm.org/magazines/2016/3/198874-lessons-learned-from-30-years-of-minix/fulltext) Eat your own dog food. By not relying on idiosyncratic features of the hardware, one makes porting to new platforms much easier. The Internet is like an elephant; it never forgets. When standards exist (such as ANSI Standard C) stick to them. Even after you have adopted a strategy, you should nevertheless reexamine it from time to time. Keep focused on your real goal, Einstein was right: Things should be as simple as possible but not simpler. *** pfSense 2.3 released (https://blog.pfsense.org/?p=2008) Rewrite of the webGUI utilizing Bootstrap TLS v1.0 disabled for the GUI Moved to a FreeBSD 10.3-RELEASE base PHP Upgraded to 5.6 The "Full Backup" feature has been deprecated Closed 760 total tickets of which 137 are fixed bugs Known Regressions OpenVPN topology change IP aliases with CARP IP parent lose their parent interface association post-upgrade IPsec IPComp does not work. IGMP Proxy does not work with VLAN interfaces. Many other updates and changes *** OPNsense 16.1.10 released (https://opnsense.org/opnsense-16-1-10-released/) openvpn: revive windows installer binaries system: improved config history and backup pages layout system: increased backup count default from 30 to 60 system: /var /tmp MFS awareness for crash dumps added trust: add “IP security IKE intermediate” to server key usage firmware: moved reboot, halt and defaults pages to new home languages: updates to Russian, French, German and Japanese Many other updates and changes *** Interview - Benedict Reuschling - bcr@freebsd.org (mailto:bcr@freebsd.org) FreeBSD Foundation in Europe *** News Roundup Write opinionated workarounds (http://www.daemonology.net/blog/2016-04-11-write-opinionated-workarounds.html) Colin Percival has written a great blog post this past week, specifically talking about his policy of writing “opinionated workarounds”. The idea came about due to his working on multi-platform software, and the frustrations of dealing with POSIX violations The crux of the post is how he deals with these workarounds. Specifically by only applying them to the particular system in which it was required. And doing so loudly. This has some important benefits. First, it doesn’t potentially expose other systems to bugs / security flaws when a workaround doesn’t “work” on a system for which it wasn’t designed. Secondly it’s important to complain. Loudly. This lets the user know that they are running on a system that doesn’t adhere to POSIX compliance, and maybe even get the attention of a developer who could remedy the situation. *** Privilege escalation in calendar(1) (http://ftp.netbsd.org/pub/NetBSD/security/advisories/NetBSD-SA2016-003.txt.asc) File this one under “Ouch that hurts” a new security vuln has been posted, this time against NetBSD’s ‘calendar’ command. Specifically it looks like some of the daily scripts uses the ‘-a’ flag, which requires super-user privs in order to process all users calendar files and mail the results. However the bug occurred because the calendar command didn’t drop priv properly before executing external commands (whoops!) To workaround you can set run_calendar=NO in the daily.conf file, or apply the fixed binary from upstream. *** PGCon 2016 (http://www.pgcon.org/2016/) PGCon 2016 is now only 4 weeks away The conference will be held at the University of Ottawa (same venue as BSDCan) from May 17th to 20th Tutorials: 17-18 May 2016 (Tue & Wed) Talks: 19-20 May 2016 (Thu-Fri) Wednesday is a developer unconference. Saturday is a user unconference. “PGCon is an annual conference for users and developers of PostgreSQL, a leading relational database, which just happens to be open source. PGCon is the place to meet, discuss, build relationships, learn valuable insights, and generally chat about the work you are doing with PostgreSQL. If you want to learn why so many people are moving to PostgreSQL, PGCon will be the place to find out why. Whether you are a casual user or you've been working with PostgreSQL for years, PGCon will have something for you.” New to PGSQL? Just a user? Long time developers? This conference has something for you. A great lineup of talks (https://www.pgcon.org/2016/schedule/events.en.html), plus unconference days focused on both users and developers *** CfP EuroBSDCon 2016 (https://2016.eurobsdcon.org/call-for-papers/) The call for papers has been issued for EuroBSDCon 2016 in Belgrade, Serbia The conference will be held from the 22nd to 25th of September, 2016 The deadline for talk submissions is: Sunday the 8th of May, 2016 Submit your talk or tutorial proposal before it is too late *** Beastie Bits “FreeBSD Mastery: Advanced ZFS” has officially been released (https://www.michaelwlucas.com/nonfiction/fmaz) Support of OpenBSD pledge(2) in programming Languages (https://gist.github.com/ligurio/f6114bd1df371047dd80ea9b8a55c104) pkgsrcCon 2016 -Call for Presentations (http://daemonforums.org/showthread.php?t=9781) Christos Zoulas talks about blacklistd (http://blog.netbsd.org/tnf/entry/talks_about_blacklistd) Penguicon 2016 Lucas Track Schedule (http://blather.michaelwlucas.com/archives/2617) Feedback/Questions Peter - NVME (http://pastebin.com/HiiDpGcT) Jeremy - Wireless Gear (http://pastebin.com/L5XeVS1H) Ted - Rpi2 Packages (http://pastebin.com/yrCEnkWt) - Cross Building Wiki (https://wiki.freebsd.org/FreeBSD/arm/crossbuild) Geoff - Jail Failover (http://pastebin.com/pYFC1vdQ) Zach - Graphical Bhyve? (http://pastebin.com/WEgN0ZVw) ***
137: FreeNAS Mini XL
This week on BSD Now, I’m out of town for the week, but we have a special unboxing video to share with you, that you won’t want to miss. That, plus the latest BSD news, is coming your way right now! This episode was brought to you by Headlines Example of a FreeBSD bug hunting session by a simple user (http://blog.cochard.me/2016/01/example-of-freebsd-bug-hunting-session.html) Don’t be fooled, Olivier Cochard-Labbé is a bit more than just a FreeBSD user Original founder of the FreeNAS project many years ago, and currently leads the BSD Router Project (designed as a replacement for “Big Iron” routers like Cisco’s etc) However, he is not actually a committer on any of the BSD projects, and is mostly focused on networking, rather than development, so it is fair to call him a user He walks us through a bug hunting session that started when he updated his wireless router “My wireless-router configuration was complex: it involves routing, wireless in hostap mode, ipfw, snort, bridge, openvpn, etc.” Provides helpful advice on writing problem reports to developers, including trying to reproduce your issue with as minimal a setup as possible. This both reduces the amount of setup a developer has to do to try to recreate your issue, and can often make it more obvious where the problem actually lies As you might expect, the more he researched the problem, the more questions he had The journey goes through the kernel debugger, learning dtrace, and reading some source code In the end it seems the problem is that the bridge interface marks itself as down if none of the interfaces are in an ‘UP’ state. The wireless interface was in the unknown state, and was actually up, but when the wired interface was disconnected, this caused the bridge to mark it self as down. *** How-to Install OpenBSD 5.9 plus XFCE desktop and basic applications (http://ribalinux.blogspot.com/2016/04/how-to-install-openbsd-59-plus-xfce.html) Now this is the way to do videos. Over at the RibaLinux blogspot site, we have a great video showing how to setup and install OpenBSD 5.9 with XFCE and basic desktop applications. Along with the video tutorial, another nicety is the commands-used script, so you can see exactly how the setup was done, without having to pause/rewind the video to keep up. How to install PC-BSD 10.3 (http://ribalinux.blogspot.com/2016/04/how-to-install-pc-bsd-103.html) In addition to the OpenBSD 5.9 setup video, they just published a PC-BSD 10.3 installation video as well, check it out! *** FreeBSD on xhyve tutorial (https://gist.github.com/tanb/f8fefa22332edc7a641d) Originally only able to boot linux, xhyve, a “sort of” port of bhyve to OS X, can now run FreeBSD This tutorial makes it much easier, providing a script There are a few small command line flag differences from bhyve on FreeBSD The tutorial also covers sharing a directory between the guest and the host, resizing and growing the disk for the guest, and converting a QEMU image to be run under xhyve *** How to Configure SSHguard With IPFW Firewall On FreeBSD (http://www.unixmen.com/configure-sshguard-ipfw-firewall-freebsd) It’s been a while, but UNIXMen has dropped on us another FreeBSD tutorial, this time on how to setup IPFW and ‘sshguard’ to protect your system. In this tutorial they first lay down the rationale for picking IPFW as the firewall, but the reasons mainly boil down to IPFW being developed primarily on FreeBSD, and as such isn’t lagging behind when it comes to features / support. Interestingly enough, they also go the route of adding their own /usr/local/etc/rc.firewall script which will be used to specify TCP/UDP ports to open through IPFW via the rc.conf file Once that setup is complete (which you can just copy-n-paste) they then move onto ‘sshguard’ setup. Specifically you’ll need to be sure to install the correct port/pkg, sshguard-ipfw in order to work in this setup, although sshguard-pf and friends are available also. The article mentions that the name ‘sshguard’ can also be misleading, since it can be used to detect brute force attempts into a number of services. From there a bunch of configuration is thrown at you, which will allow you to start making the most out of sshguard’s potential, well worth your read if you are using IPFW, or even PF and want to get the basics down of using sshguard properly. *** FreeNAS Mini XL Video Unboxing Beastie Bits Amazon lists FreeBSD as 'Other Linux' (https://i.imgur.com/NJ7lpso.png) sbin/hammer: Make hammer commands print root volume path (http://lists.dragonflybsd.org/pipermail/commits/2016-April/459667.html) sbin/hammer: Print volume list after volume-add|del (http://lists.dragonflybsd.org/pipermail/commits/2016-April/459674.html) Front cover reveal for the upcoming 'FreeBSD Mastery: Advanced ZFS" book (https://twitter.com/mwlauthor/status/716328414072872960) If you don’t already have one, get your FreeBSD Pillow (http://linuxpillow.blogspot.com/2016/03/world-backup-day.html) Feedback/Questions Daniel - SysVIPC (http://pastebin.com/raw/JBbMj87t) Shane - OpenToonz (http://pastebin.com/raw/54ngYVEN) ***
136: This is GNN
This week on the show, we will be interviewing GNN of the FreeBSD project to talk about the new TeachBSD initiative. That plus the latest BSD headlines, all coming your way right now! This episode was brought to you by Headlines FreeBSD 10.3-RELEASE Announcement (https://www.freebsd.org/releases/10.3R/announce.html) FreeBSD 10.3 has landed, with extended support until April 30, 2018 This is likely to be the last extended support release, as starting with 11, the new support model will encourage upgrading to the latest minor version by ending support for the previous minor version approximately 2 months after each point release. The Major version / stable branch will still be supported for the same 5 year term. This will allow the FreeBSD project to move forward more quickly, while still providing the same level of long term support The UEFI boot loader is much improved, and now supports booting root-on-ZFS, and the beastie menu The beastie menu itself has been updated with support for ZFS Boot Environments The CAM Target Layer (CTL) now supports High Availability, allowing the construction of much more advanced storage systems The 64bit Linux Emulation Layer was backported Reroot support was added, allowing the system to boot off of a minimal image, such as a mfsroot and then reload all of userland from a different root file system (such as iSCSI, NFS, etc) The version of xz(1) has been updated to support multi-threaded compression sesutil(8) has been introduced, making it easier to manage large storage nodes Various ZFS updates As usual, a huge number of driver updates are also included *** How to use OpenBSD with Libreboot: detailed instructions (https://lists.nongnu.org/archive/html/libreboot/2016-04/msg00010.html) This tutorial covers installing OpenBSD on a Thinkpad X200 using Libreboot, a replacement for the traditional BIOS/firmware that comes from the manufacturer “Since 5.9, OpenBSD supports EFI boot mode, which means that it also have had to support framebuffer out of the box, so lack of proprietary VGA BIOS blob is no longer a problem and you can boot it with unmodified Libreboot binary release 20150518.” “In order to install OpenBSD on such a machine you will need someadditional preparations, since regular install59.fs won't work because bsd.rd doesn't have a framebuffer console.” A few extra steps are required to get it going, but they are outlined in the post This may be very interesting to those who prefer not to depend on binary blobs *** Linking the FreeBSD base system with lld -- status update (http://lists.llvm.org/pipermail/llvm-dev/2016-March/096449.html) The FreeBSD Foundation’s Ed Maste provides an update on the LLVM mailing list about the progress of replacing the GNU linker with the lld in the FreeBSD base system “I'm pleased to report that I can now build a runnable FreeBSD system using lld as the linker (for buildworld), with a few workarounds and work-in-progress patches. I have not yet extensively tested the result but it is possible to login to the resulting system, and basic sanity tests I've tried are successful. Note that the kernel is still linked with ld.bfd.” Outstanding Issues Symbol version support (PR 23231). FreeBSD uses symbol versioning for backwards compatibility Linker script expression support (PR 26731). The FreeBSD kernel linker scripts contain expressions not currently supported by lld Library search paths. GNU LD automatically searches /lib, and lld does not the -N flag makes the text and data sections RW and does not page-align data. It is used by boot loader components. The -dc flag assigns space to common symbols when producing relocatable output (-r). It is used by the /rescue build, which is a single binary assembled from a collection of individual tools (sh, ls, fsck, ...) -Y adds a path to the default library search path. It is used by the lib32 build, which provides i386 builds of the system libraries for compatibility with i386 applications. With the ongoing work, it might be possible for FreeBSD 11 to use lld by default, although it might be best to wait to throw that particular switch *** Your favorite billion user company using BSD just flipped on encryption for all their users -- and it took 15 Engineers to do it (http://www.wired.com/2016/04/forget-apple-vs-fbi-whatsapp-just-switched-encryption-billion-people/) With the help of Moxie Marlinspike’s Open Whisper Systems, WhatsApp has integrated the ‘Signal’ encryption system for all messages, class, pictures, and videos sent between individuals or groups It uses public key cryptography, very similar to GPG, but with automated public key servers It also includes a system of QR codes to verify the identity of individuals in person, so you can be sure the person you are talking to is actually the person you met with WhatsApp runs their billion user network, using FreeBSD, with only about 50 engineers Only 15 of those engineers we needed to work on the project that has now deployed complete end-to-end encryption across the entire network The Wired article is very detailed and well worth the read *** Interview - George Neville-Neil - gnn@freebsd.org (mailto:gnn@freebsd.org) / @gvnn3 (https://twitter.com/gvnn3) Teaching BSD with Tracing News Roundup Faces of FreeBSD 2016: Scott Long (https://www.freebsdfoundation.org/blog/faces-of-freebsd-2016-scott-long/) It’s been awhile since we’ve had a new entry into the “Faces of FreeBSD” series, but due to popular demand it’s back! This installment features developer Scott Long, who currently works at NetFlix, previously at Yahoo and Adaptec. Scott got a very early start into BSD, first discovering i386BSD 0.1 on a FTP server at Berkeley, back at 1992. From there on it’s been a journey, following along with FreeBSD since version 1.0 in 1993. So what stuff can we blame Scott for? In his own words: I’ve been a source committer since 2000. I got my start by taking over maintainership of the Adaptec ‘aac’ RAID driver. From 2002-2006 I was the Release Engineer and was responsible for the 5.x and 6.x releases. Though the early 5.x releases were not great, they were necessary stepping stones to the success of FreeBSD 6.x and beyond. I’m exceptionally proud of my role in helping FreeBSD move forward during that time. I authored and maintained the ‘mfi’ and ‘mps’ storage drivers, the ‘udf’ filesystem driver, and several smaller sound and USB drivers. I’ve maintained, or at least touched, most of the storage device drivers in the system to some extent, and I implemented medium-grained locking on the CAM storage stack. Recently I’ve been working on overall system scalability and performance. ASCII Flow (http://asciiflow.com/) A website that lets to draw and share ASCII diagrams Great for network layout maps, rack diagrams, protocol analysis etc Use it in your presentations and slides Sample (https://drive.google.com/open?id=0BynxTTJrNUOKeWxCVm1ERExrNkU) *** System Under Test: FreeBSD (http://lowlevelbits.org/system-under-test-freebsd/) Part of a series looking at testing across a number of projects Outlines the testing framework of FreeBSD Provides a mini-tutorial on how to run the tests There are some other tests that are now covered, but this is due to a lack of documentation on the fact that the tests exist, and how to run them There is much ongoing work in this area *** Worst April Fools Joke EVER! (http://www.rhyous.com/2016/04/01/microsoft-announces-it-is-acquiring-freebsd-for-300-million/) While a bad April Fool’s joke, it also shows some common misconceptions The FreeBSD Foundation does not own the source repository, it is only the care taken of the trademark, and other things that require a single legal entity OpenBSD and NetBSD are not ‘sub brands’ of FreeBSD Bash was not ported to Windows, but rather Windows gained a system similar to FreeBSD’s linux_compat It would be nice to have ZFS on Windows *** Beastie Bits Credit where credit's due... (https://forums.freebsd.org/threads/55642/) M:Tier's OpenBSD packages and binpatches updated for 5.9 (https://stable.mtier.org/) NYC BUG Meeting (2016-04-06) - Debugging with LLVM, John Wolfe (http://www.nycbug.org/index.cgi) Need to create extremely high traffic loads? kq_sendrecv is worth checking out (http://lists.dragonflybsd.org/pipermail/commits/2016-March/459651.html) If you're in the Maryland region, CharmBug has a meetup next week (http://www.meetup.com/CharmBUG/events/230048300/) How to get a desktop on DragonFly (https://www.dragonflybsd.org/docs/how_to_get_to_the_desktop/) Linux vs BSD Development Models (https://twitter.com/q5sys/status/717509675630084096) Feedback/Question Paulo - ZFS Setup (http://pastebin.com/raw/GrM0jKZK) Jonathan - Installation (http://pastebin.com/raw/13KCkhMU) Andrew - Career / School (http://pastebin.com/wsx90L2m)
135: Speciality MWL
This week on the show, we interview author Michael W Lucas to discuss his new book in the FreeBSD This episode was brought to you by Headlines OpenBSD 5.9 Released early (http://undeadly.org/cgi?action=article&sid=20160329181346&mode=expanded) Finished ahead of schedule! OpenBSD 5.9 has officially landed We’ve been covering some of the ongoing changes as they landed in the tree, but with the official release it’s time to bring you the final list of the new hotness which landed. First up: Pledge - Over 70%! Of the userland utilities have been converted to use it, and the best part, you probably didn’t even notice UEFI - Laptops which are pre-locked down to boot UEFI only can now be installed and used - GPT support has also been greatly improved ‘Less’ was replaced with a fork from Illumos, and has been further improved Xen DomU support - OpenBSD now plays nice in the cloud X11 - Broadwell and Bay Trail are now supported Initial work on making the network stack better support SMP has been added, this is still ongoing, but things are starting to happen 802.11N! Specifically for the iwn/iwm drivers In addition to support for UTF-8, most other locales have been ripped out, leaving only C and UTF-8 left standing in the wake All and all, sounds like a solid new release with plenty of new goodies to play with. Go grab a copy now! *** New routing table code (ART) enabled in -current (http://undeadly.org/cgi?action=article&sid=20160324093944) While OpenBSD 5.9 just landed, we also have some interesting work landing right now in -CURRENT as well. Specifically the new routing table code (ART) has landed: “I just enabled ART in -current, it will be the default routing table backend in the next snapshots. The plan is to squash the possible regressions with this new routing table backend then when we're confident enough, take its route lookup out of the KERNEL_LOCK(). Yes, this is one of the big steps for our network SMP improvements. In order to make progress, we need your help to make sure this new backend works well on your setup. So please, go download the next snapshot and report back. If you encounter any routing table regression, please make sure that you cannot reproduce it with your old kernel and include the output of # route -n show for the 2 kernels as well as the dmesg in your report. I know that simple dhclient(8) based setups work with ART, so please do not flood us too much. It's always great to know that things work, but it's also hard to keep focus ;) Thank your very much for your support!” + There you have it folks! If 5.9 is already too stale for you, time to move over to -CURRENT and give the new routing tables a whirl. fractal cells - FreeBSD-based All-In-One solution for software development startups (https://forums.freebsd.org/threads/55561/) Fractal Cells is a suite that transforms a stock FreeBSD installation into an instant “Startup Software Development Platform” It Integrates ZFS, PostgreSQL, OpenSMTPD, NGINX, OpenVPN, Redmine, Jenkins, Zabbix, Gitlab, and Ansible, all under OpenLDAP common authentication The suite is available under the 2-clause BSD license Provides all of the tools and infrastructure to build your application, including code review, issue tracking, continuous integration, and monitoring An interesting way to make it easier for people to start building new applications and startups on top of FreeBSD *** LinuxSecrets publishes guide on installing FreeBSD ezJail (http://www.linuxsecrets.com/blog/51freebsd/2016/02/29/1726-installing) Covers all of the steps of setting up ezjail on FreeBSD Includes the instructions for updating the version of the OS in the jail In a number of places the tutorial uses: > cat << EOF >> /etc/rc.conf > setting=”value” Instead, use: sysrc setting=”value” It is safer, and easier to type When you create the jail, if you specify an IP address, it is expected that this IP address is already setup on the host machine If instead you specify: ‘em0|192.168.1.105’ (where em0 is your network interface), the IP address will be added as an alias when the jail starts, and removed from the host when the jail is stopped You can also comma separate a list of addresses to have multiple IPs (possibly on different interfaces) in the jail Although recently posted, this appears as if it might be an update to a previous tutorial, as there are a few old references that have not been updated (pkg_add, rc.d/ezjail.sh), while the start of the article clearly covers pkg(8) *** Interview - Michael W. Lucas - mwlucas@michaelwlucas.com (mailto:mwlucas@michaelwlucas.com) / @mwlauthor (https://twitter.com/mwlauthor) + New Book: “FreeBSD Mastery: Specialty Filesystems” News Roundup NetBSD on Dreamcast (https://github.com/fwbug/dreamcast-slides) Ahh the dreamcast, so much promise. So much potential. If you are still holding onto your beloved dreamcast hoping that someday Sega will re-enter the console market… Then give it up now! In the meantime, you can now do something more interesting with that box taking up space in the closet. We have a link to a GitHub repo where a user has uploaded his curses-based slide-show for the upcoming Fort-Wayne, Indiana meetup. Aside from the novelty of using a curses-based slide setup, the presenter will also be displaying them from his beloved dreamcast, which “of course” runs NetBSD 7 The slide source code is available, which you too can view / compile and find out details of getting NetBSD boot-strapped on the DC. *** OPNsense 16.1.7 Released (https://opnsense.org/opnsense-16-1-7-released/) captive portal: add session timeout to status info firewall: fix non-report of errors when filter reload errors couldn’t be parsed proxy: adjust category visibility as not all of them were shown before firmware: fix an overzealous upgrade run when the package tool only changes options firmware: fixed the binary upgrade patch from 15.7.x in FreeBSD’s package tool system: removed NTP settings from general settings access: let only root access status.php as it leaks too much info development: remove the automount features development: addition of “opnsense-stable” package on our way to nightly builds development: opnsense-update can now install locally available base and kernel sets *** “FreeBSD Mastery: Advanced ZFS” in tech review (http://blather.michaelwlucas.com/archives/2570) Most of the tech review is finished It was very interesting to hear from many ZFS experts that they learned something from reading the review copy of the book, I was not expecting this Many minor corrections and clarifications have been integrated The book is now being copy edited *** Why OpenBSD? (http://www.cambus.net/why-openbsd/) Frederic Cambus gives us a nice perspective piece today on what his particular reasons are for choosing OpenBSD. Frederic is no stranger to UNIX-Like systems, having used them for 20 years now. In particular starting on Slackware back in ‘96 and moving to FreeBSD from 2000-2005 (around the 4.x series) His adventure into OpenBSD began sometime after 2005 (specific time unknown), but a bunch of things left a very good impression on him throughout the years. First, was the ease of installation, with its very minimalistic layout, which was one of the fastest installs he had ever done. Second was the extensive documentation, which extends beyond just manpages, but into other forms of documentation, such as presentations and papers as well. He makes the point about an “ecosystem of quality” that surrounds OpenBSD: OpenBSD is an ecosystem of quality. This is the result of a culture of code auditing, reviewing, and a rigorous development process where each commit hitting the tree must be approved by other developers. It has a slower evolution pace and a more carefully planned development model which leads to better code quality overall. Its well deserved reputation of being an ultra secure operating system is the byproduct of a no compromise attitude valuing simplicity, correctness, and most importantly proactivity. OpenBSD also deletes code, a lot of code. Everyone should know that removing code and keeping the codebase modern is probably as important as adding new one. Quoting Saint-Exupery: "It seems that perfection is attained not when there is nothing more to add, but when there is nothing more to remove". The article then covers security mechanisms, as well as the defaults which are turned specifically with an eye towards security. All-in-all a good perspective piece about the reasons why OpenBSD is the right choice for Frederic, worth your time to read up on it if you want to learn more about OpenBSD’s differences. *** BeastieBits Call for 2016Q1 quarterly status reports (https://docs.freebsd.org/cgi/getmsg.cgi?fetch=9011+0+current/freebsd-hackers) FreeBSD Mastery: Advanced ZFS” sponsorships ending soon (http://blather.michaelwlucas.com/archives/2593) Shawn Webb from HardenedBSD talking about giving away RPi3’s at BSDCan and hacking on them to get FreeBSD working (https://docs.freebsd.org/cgi/getmsg.cgi?fetch=250105+0+archive/2016/freebsd-arm/20160306.freebsd-arm) xterm(1) now UTF-8 by default (http://undeadly.org/cgi?action=article&sid=20160308204011) Call For Artists: New Icon Theme (https://blog.pcbsd.org/2016/03/call-for-artists-new-icon-theme/) Happy 23rd Birthday, src! (http://blog.netbsd.org/tnf/entry/happy_23rd_birthday_src) Feedback/Questions Alison - Readahead and Wayland (http://slexy.org/view/s2oqRuXCYW) Kenny - Gear (http://slexy.org/view/s2sQ8MxNPh) Ben - IPFW2/3 (http://slexy.org/view/s20SRvXPZA) Brad - ZFS Writeback (http://slexy.org/view/s207mV2Ph1) Simon - BSD Toonz (http://slexy.org/view/s202loSWdf) ***