Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Similar Podcasts
Elixir Outlaws
Elixir Outlaws is an informal discussion about interesting things happening in Elixir. Our goal is to capture the spirit of a conference hallway discussion in a podcast.
The Cynical Developer
A UK based Technology and Software Developer Podcast that helps you to improve your development knowledge and career,
through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer.
Programming Throwdown
Programming Throwdown educates Computer Scientists and Software Engineers on a cavalcade of programming and tech topics. Every show will cover a new programming language, so listeners will be able to speak intelligently about any programming language.
322: Happy Birthday, Unix
Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more. Headlines Unix is 50 (https://www.bell-labs.com/unix50/) In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing. Hunting down Ken's PDP-7: video footage found (https://bsdimp.blogspot.com/2019/10/video-footage-of-first-pdp-7-to-run-unix.html) In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix. News Roundup OpenBSD 6.6 Released (https://openbsd.org/66.html) Announce: https://marc.info/?l=openbsd-tech&m=157132024225971&w=2 Upgrade Guide: https://openbsd.org/faq/upgrade66.html Changelog: https://openbsd.org/plus66.html OPNsense 19.7.5 released (https://opnsense.org/opnsense-19-7-5-released/) Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version 12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so. Here are the full patch notes: + system: show all swap partitions in system information widget + system: flatten services_get() in preparation for removal + system: pin Syslog-ng version to specific package name + system: fix LDAP/StartTLS with user import page + system: fix a PHP warning on authentication server page + system: replace most subprocess.call use + interfaces: fix devd handling of carp devices (contributed by stumbaumr) + firewall: improve firewall rules inline toggles + firewall: only allow TCP flags on TCP protocol + firewall: simplify help text for direction setting + firewall: make protocol log summary case insensitive + reporting: ignore malformed flow records + captive portal: fix type mismatch for timeout read + dhcp: add note for static lease limitation with lease registration (contributed by Northguy) + ipsec: add margintime and rekeyfuzz options + ipsec: clear $dpdline correctly if not set + ui: fix tokenizer reorder on multiple saves + plugins: os-acme-client 1.26[1] + plugins: os-bind will reload bind on record change (contributed by blablup) + plugins: os-etpro-telemetry minor subprocess.call replacement + plugins: os-freeradius 1.9.4[2] + plugins: os-frr 1.12[3] + plugins: os-haproxy 2.19[4] + plugins: os-mailtrail 1.2[5] + plugins: os-postfix 1.11[6] + plugins: os-rspamd 1.8[7] + plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks) + plugins: os-telegraf 1.7.6[8] + plugins: os-theme-cicada 1.21 (contributed by Team Rebellion) + plugins: os-theme-tukan 1.21 (contributed by Team Rebellion) + plugins: os-tinc minor subprocess.call replacement + plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz) + plugins: os-virtualbox 1.0 (contributed by andrewhotlab) Dealing with the misunderstandings of what is GhostBSD (http://ghostbsd.org/node/194) Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE. Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release. There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum. SHUTTLE – VPN over SSH | VPN Alternative (https://www.terminalbytes.com/sshuttle-vpn-over-ssh-vpn-alternative/) Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy. VPN over SSH – sshuttle sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN. OpenSSH 8.1 Released (http://www.openssh.com/txt/release-8.1) Security ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program. ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB). This release includes a number of changes that may affect existing configurations: ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ..."). New Features ssh(1): Allow %n to be expanded in ProxyCommand strings ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '^' character, E.g. "HostKeyAlgorithms ^ssh-ed25519" ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email). ssh-keygen(1): print key comment when extracting public key from a private key. ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too. All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's. Beastie Bits Say goodbye to the 32 CPU limit in NetBSD/aarch64 (https://twitter.com/jmcwhatever/status/1185584719183962112) vBSDcon 2019 videos (https://www.youtube.com/channel/UCvcdrOSlYOSzOzLjv_n1_GQ/videos) Browse the web in the terminal - W3M (https://www.youtube.com/watch?v=3Hfda0Tjqsg&feature=youtu.be) NetBSD 9 and GSoC (http://netbsd.org/~kamil/GSoC2019.html#slide1) BSDCan 2019 Videos (https://www.youtube.com/playlist?list=PLeF8ZihVdpFegPoAKppaDSoYmsBvpnSZv) NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders (https://www.nycbug.org/index?action=view&id=10673) FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open (https://www.freebsdfoundation.org/blog/freebsd-miniconf-at-linux-conf-au-2020-call-for-sessions-now-open/) FOSDEM 2020 - BSD Devroom Call for Participation (https://people.freebsd.org/~rodrigo/fosdem20/) University of Cambridge looking for Research Assistants/Associates (https://twitter.com/ed_maste/status/1184865668317007874) Feedback/Questions Trenton - Beeping Thinkpad (http://dpaste.com/0ZEXNM6#wrap) Alex - Per user ZFS Datasets (http://dpaste.com/1K31A65#wrap) Allan’s old patch from 2015 (https://reviews.freebsd.org/D2272) Javier - FBSD 12.0 + ZFS + encryption (http://dpaste.com/1XX4NNA#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
321: The Robot OS
An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more. Interview - Trenton Schulz - freenas@norwegianrockcat.com (mailto:freenas@norwegianrockcat.com) Robot OS on FreeBSD BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD? AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how? BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD? AJ: Was EuroBSDcon your first BSD conference? How did you like it? BR: Do you have some tips or advice on how to get started with the BSDs? AJ: Is there anything else you’d like to tell us before we let you go? Beastie Bits FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open (https://www.freebsdfoundation.org/blog/freebsd-miniconf-at-linux-conf-au-2020-call-for-sessions-now-open/) Portland BSD Pizza Night: Oct 24th, 19:00 @ Rudy’s Gourmet Pizza (http://calagator.org/events/1250476319) NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders (https://www.nycbug.org/index?action=view&id=10673) FOSDEM 2020 - BSD Devroom Call for Participation (https://people.freebsd.org/~rodrigo/fosdem20/) University of Cambridge looking for Research Assistants/Associates (https://twitter.com/ed_maste/status/1184865668317007874) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Trenton Shulz.
320: Codebase: Neck Deep
Headlines FreeBSD and custom firmware on the Google Pixelbook (https://unrelenting.technology/articles/FreeBSD-and-custom-firmware-on-the-Google-Pixelbook) FreeBSD and custom firmware on the Google Pixelbook Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something.. lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along); with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially); with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional); with USB-C ports; without a dGPU, especially without an NVIDIA GPU; assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good); supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers); how about something with open source firmware, that would be fun. I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less! And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge). So here it is, my new laptop, a Google Pixelbook. Conclusion Pixelbook, FreeBSD, coreboot, EDK2 good. Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :) Porting NetBSD to the AMD x86-64: a case study in OS portability (https://www.usenix.org/legacy/publications/library/proceedings/bsdcon02/full_papers/linden/linden_html/index.html) Abstract NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example. Portability Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way. General Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements. Types Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32t, int8t, etc. Conclusions and future work The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification. News Roundup ZFS performance really does degrade as you approach quota limits (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFullQuotaPerformanceIssue) Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones. In the September meeting notes, I read a very interesting (to me) agenda item: Relax quota semantics for improved performance (Allan Jude) Problem: As you approach quotas, ZFS performance degrades. Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased. This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time). Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.) With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again. Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.." (http://adrianchadd.blogspot.com/2019/09/fixing-up-ka9q-unix-or-neck-deep-in-30.html) I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there. Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC. So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be. But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool! I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda. HAMMER2 and fsck for review (https://www.dragonflydigest.com/2019/09/24/23540.html) HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process. [The return of startx(1) for non-root users with some caveats (https://undeadly.org/cgi?action=article;sid=20190917091236) Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation: ``` CVSROOT: /cvs Module name: src Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41 Modified files: etc/etc.amd64 : fbtab etc/etc.arm64 : fbtab etc/etc.hppa : fbtab etc/etc.i386 : fbtab etc/etc.loongson: fbtab etc/etc.luna88k: fbtab etc/etc.macppc : fbtab etc/etc.octeon : fbtab etc/etc.sgi : fbtab etc/etc.sparc64: fbtab Log message: Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges. ok jsg@, matthieu@ CVSROOT: /cvs Module name: xenocara Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08 Modified files: xserver/hw/xfree86/common: xf86AutoConfig.c Log message: Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus. This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)). ok jsg@, matthieu@ ``` Beastie Bits ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal? (https://bestasciitable.com/) Sourcehut makes BSD software better (https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/) Chaosnet for Unx (https://github.com/LM-3/chaos) The Vim-Inspired Editor with a Linguistic Twist (https://cosine.blue/2019-09-06-kakoune.html) bhyvearm64: CPU and Memory Virtualization on Armv8.0-A (https://papers.freebsd.org/2019/bsdcan/elisei-bhyvearm64_cpu_and_memory_virtualization_on_armv8.0_a/) DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities (https://www.youtube.com/watch?v=a2m56Yq-EIs) Feedback/Questions Tim - GSoC project ideas for pf rule syntax translation (http://dpaste.com/1RCSFK7#wrap) Brad - Steam on FreeBSD (http://dpaste.com/2SKA9YB#wrap) Ruslan - FreeBSD Quarterly Status Report - Q2 2019 (http://dpaste.com/0DQM3Q1) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
319: Lack Rack, Jack
Causing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more. Headlines Causing ZFS corruption for fun and profit (https://datto.engineering/post/causing-zfs-corruption) Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios. Causing Corruption Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets. Conclusion At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events. To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject. NetBSD Assembly Programming Tutorial (https://polprog.net/blog/netbsdasmprog/) A sparc64 version is also being prepared and will be added when done This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel. Why assembly? Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs Due to the nature of the guide, visit the site for the complete breakdown News Roundup The IKEA Lack Rack for Servers (https://wiki.eth0.nl/index.php/LackRack) The LackRack First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009. The LackRack will certainly make its appearance again this summer at eth0:2010 Summer. Summary When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks. The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented! Howto You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat! OmniOS Community Edition r151030 LTS - Published at May 6, 2019 (https://omniosce.org/article/release-030) The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28. For full relase notes including upgrade instructions; release notes (https://omniosce.org/releasenotes.html) upgrade instructions (https://omniosce.org/upgrade.html) List Block Devices on FreeBSD lsblk(8) Style (https://vermaden.wordpress.com/2019/09/27/list-block-devices-on-freebsd-lsblk8-style/) When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world. Example lsblk(8) output from Linux system: $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom sda 8:0 0 931.5G 0 disk |-sda1 8:1 0 500M 0 part /boot `-sda2 8:2 0 931G 0 part |-vg_local-lv_root (dm-0) 253:0 0 50G 0 lvm / |-vg_local-lv_swap (dm-1) 253:1 0 17.7G 0 lvm [SWAP] `-vg_local-lv_home (dm-2) 253:2 0 1.8T 0 lvm /home sdc 8:32 0 232.9G 0 disk `-sdc1 8:33 0 232.9G 0 part `-md1 9:1 0 232.9G 0 raid10 /data sdd 8:48 0 232.9G 0 disk `-sdd1 8:49 0 232.9G 0 part `-md1 9:1 0 232.9G 0 raid10 /data What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs. See the article for the rest of the guide Project Trident 19.10 Now Available (https://project-trident.org/post/2019-10-05_19.10_available/) This is a general package update to the CURRENT release repository based upon TrueOS 19.10 PACKAGE CHANGES FROM 19.08 New Packages: 601 Deleted Packages: 165 Updated Packages: 3341 Beastie Bits NetBSD building tools (https://imgur.com/gallery/0sG4b1K) Sponsorships open for SNMP Mastery (https://mwl.io/archives/4569) pkgsrc-2019Q3 release announcement (2019-10-03) (http://mail-index.netbsd.org/pkgsrc-users/2019/10/03/msg029485.html) pfetch - A simple system information tool written in POSIX sh (https://github.com/dylanaraps/pfetch) Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview) (https://netbsd.org/~kamil/eurobsdcon2019_fuzzing/presentation.html#slide1) Cracking Ken Thomson’s password (https://leahneukirchen.org/blog/archive/2019/10/ken-thompson-s-unix-password.html) Feedback/Questions Evilham - Couple Questions (http://dpaste.com/2JC85WV) Rob - APU2 alternatives and GPT partition types (http://dpaste.com/0SDX9ZX) Tom - FreeBSD journal article by A. Fengler (http://dpaste.com/2B43MY1#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
318: The TrueNAS Library
DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more. Headlines DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X (https://www.phoronix.com/scan.php?page=article&item=bsd-linux-3700x) For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04. Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs. We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs. For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round. All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test. JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives (https://www.ixsystems.com/blog/jfk-presidential-library-pr/) iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete. Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection. Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors. With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes. Youtube Video (https://www.youtube.com/watch?v=8rFjH5-0Fiw) News Roundup FreeBSD 12.1-beta available (https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-12.1-Beta-Released) FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system. FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes. For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures. The FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November. Announcement Link (https://lists.freebsd.org/pipermail/freebsd-stable/2019-September/091533.html) Cool, but obscure X11 tools. More suggestions in the source link (https://cyber.dabamos.de/unix/x11/) ASClock Free42 FSV2 GLXGears GMixer GVIM Micropolis Sunclock Ted TiEmu X026 X48 XAbacus XAntfarm XArchiver XASCII XBiff XBill XBoard XCalc XCalendar XCHM XChomp XClipboard XClock XClock/Cat Clock XColorSel XConsole XDiary XEarth XEdit Xev XEyes XFontSel XGalaga XInvaders 3D XKill XLennart XLoad XLock XLogo XMahjongg XMan XMessage XmGrace XMixer XmMix XMore XMosaic XMOTD XMountains XNeko XOdometer XOSView Xplore XPostIt XRoach XScreenSaver XSnow XSpread XTerm XTide Xv Xvkbd XWPE XZoom vBSDCon 2019 trip report from iXSystems (https://www.ixsystems.com/blog/vbsdcon-2019/) The fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn. Project Trident 12-U7 now available (https://project-trident.org/post/2019-09-21_stable12-u7_available/) Package Summary New Packages: 130 Deleted Packages: 72 Updated Packages: 865 Stable ISO - https://pkg.project-trident.org/iso/stable/Trident-x64-TOS-12-U7-20190920.iso A Couple new Unix Artifacts (https://minnie.tuhs.org//pipermail/tuhs/2019-September/018685.html) I fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems. So I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me: by two large organisations of great significance to Unix history who want me to keep "mum" about them as they are going to make announcements about them soon* and I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-) Cheers, Warren * for some definition of "soon" Beastie Bits NetBSD machines at Open Source Conference 2019 Hiroshima (https://mail-index.netbsd.org/netbsd-advocacy/2019/09/16/msg000813.html) Hyperbola a GNU/Linux OS is using OpenBSD's Xenocara (https://www.hyperbola.info/news/end-of-xorg-support/) Talos is looking for a FreeBSD Engineer (https://www.talosintelligence.com/careers/freebsd_engineer) GitHub - dylanaraps/pure-sh-bible: A collection of pure POSIX sh alternatives to external processes. (https://github.com/dylanaraps/pure-sh-bible) dsynth: you’re building it (https://www.dragonflydigest.com/2019/09/23/23523.html) Percy Ludgate, the missing link between Babbage’s machine and everything else (http://lists.sigcis.org/pipermail/members-sigcis.org/2019-September/001606.html) Feedback/Questions Bruce - Down the expect rabbithole (http://dpaste.com/147HGP3#wrap) Bruce - Expect (update) (http://dpaste.com/37MNVSW#wrap) David - Netgraph answer (http://dpaste.com/2SE1YSE) Mason - Beeps? (http://dpaste.com/00KKXJM) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
317: Bots Building Jails
Setting up buildbot in FreeBSD jails, Set up a mail server with OpenSMTPD, Dovecot and Rspamd, OpenBSD amateur packet radio with HamBSD, DragonFlyBSD's HAMMER2 gets fsck, return of startx for users. Headlines EuroBSDcon 2019 Recap (https://2019.eurobsdcon.org/) We’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials (https://2019.eurobsdcon.org/tutorial-speakers/), parallel to the FreeBSD Devsummit (https://wiki.freebsd.org/DevSummit/201909), followed by two days of talks (https://2019.eurobsdcon.org/program/). Some speakers uploaded their slides to papers.freebsd.org (https://papers.freebsd.org/2019/eurobsdcon/) already with more to come. The social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant. The opening keynote by Patricia Aas was very good. Her talk on embedded ethics, from her perspective as someone trying to defend the sanctity of Norwegian elections, and a former developer for the Opera web browser, provided a great deal of insight into the issues. Her points about how the tech community has unleashed a very complex digital work upon people with barely any technical literacy were well taken. Her stories of trying to explain the problems with involving computers in the election process to journalists and politicians struck a chord with many of us, who have had to deal with legislation written by those who do not truly understand the issues with technology. Setting up buildbot in FreeBSD jails (https://andidog.de/blog/2018-04-22-buildbot-setup-freebsd-jails) In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here. Setting up a mail server with OpenSMTPD, Dovecot and Rspamd (https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/) Self-hosting and encouraging smaller providers is for the greater good First of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps. It doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice. There is nothing wrong with Random Joes using a service that works. What is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined. News Roundup The HamBSD project aims to bring amateur packet radio to OpenBSD (https://hambsd.org/) The HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system. HamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code. The amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither). DragonFlyBSD's HAMMER2 Gets Basic FSCK Support (https://www.dragonflydigest.com/2019/09/24/23540.html) HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process. commit (https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/5554cc8b81fbfcfd347f50be3f3b1b9a54b871b) Add initial fsck support for HAMMER2, although CoW fs doesn't require fsck as a concept. Currently no repairing (no write), just verifying. Keep this as a separate command for now. https://i.redd.it/vkdss0mtdpo31.jpg The return of startx for users (http://undeadly.org/cgi?action=article;sid=20190917091236) Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus. This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)). Beastie Bits Ori Bernstein will be giving the October talk at NYCBUG (http://lists.nycbug.org:8080/pipermail/talk/2019-September/018046.html) BSD Pizza Night: 2019/09/26, 7–9PM, Portland, Oregon, USA (http://calagator.org/events/1250476200) Nick Wolff : Home Lab Show & Tell (http://knoxbug.org/2019-09-30) Installing the Lumina Desktop in DragonflyBSD (https://www.youtube.com/watch?v=eWkCjj4_xsk) dhcpcd 8.0.6 added (https://www.dragonflydigest.com/2019/09/20/23519.html) Feedback/Questions Bruce - FOSDEM videos (http://dpaste.com/15ABRRB#wrap) Lars - Super Cluster of BSD on Rock64Pr (http://dpaste.com/1X9FEJJ) Madhukar - Question (http://dpaste.com/0TWF1NB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
316: git commit FreeBSD
NetBSD LLVM sanitizers and GDB regression test suite, Ada—The Language of Cost Savings, Homura - a Windows Games Launcher for FreeBSD, FreeBSD core team appoints a WG to explore transition to Git, OpenBSD 6.6 Beta tagged, Project Trident 12-U5 update now available, and more. Headlines LLVM santizers and GDB regression test suite. (http://blog.netbsd.org/tnf/entry/llvm_santizers_and_gdb_regression) As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay. I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite. NetBSD distribution changes I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@). The following changes were committed to the sources: ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data 40e89e890d66 Fix build of tubsan/tubsanxx under MKSANITIZER b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data 5fafbe8b8f64 Add missing extern declaration of ibmachemips in installboot d134584be69a Add SANITIZERRENAMECLASSES in bsd.prog.mk 2d00d9b08eae Adapt tests/kernel/tsubrprf for MKSANITIZER ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7 d8671fba7a78 Set NODEBUG for LLVM sanitizers 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image 64361771c78d Enhance MKSANITIZER support 3b5608f80a2b Define targetnotsupported_body() in TSan, MSan and libFuzzer tests c27f4619d513 Avoids signedness bit shift in dbgetvalue() 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no) 4ecfbbba2f2a Rework the LLVM compiler_rt build rules 748813da5547 Correct the build rules of LLVM sanitizers 20e223156dee Enhance the support of LLVM sanitizers 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0. Homura - a Windows Games Launcher for FreeBSD (https://github.com/Alexander88207/Homura) Inspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD. Makes it easier to run games on FreeBSD, by providing the tweaks and dependencies for you Dependencies curl bash p7zip zenity webfonts alsa-utils (Optional) winetricks vulkan-tools mesa-demos i386-wine-devel on amd64 or wine-devel on i386 News Roundup Ada—The Language of Cost Savings? (https://www.electronicdesign.com/embedded-revolution/ada-language-cost-savings) Many myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust. Chris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solution.” In general, Ada already makes embedded “programming in the large” much easier by handling issues that aren’t even addressed in other languages. Though these features are often provided by third-party software, it results in inconsistent practices among developers. Ada also supports the gamut of embedded platforms from systems like Arm’s Cortex-M through supercomputers. Learning Ada isn’t as hard as one might think and the benefits can be significant. FreeBSD core team appoints a WG to explore transitioning from Subversion to Git. (https://www.freebsd.org/news/status/report-2019-04-2019-06.html#FreeBSD-Core-Team) The FreeBSD Core Team is the governing body of FreeBSD. Core approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph). The annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019. The core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic. There is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward. OpenBSD 6.6 Beta tagged (https://undeadly.org/cgi?action=article;sid=20190810123243) ``` CVSROOT: /cvs Module name: src Changes by: deraadt@cvs.openbsd.org 2019/08/09 21:56:02 Modified files: etc/root : root.mail share/mk : sys.mk sys/arch/macppc/stand/tbxidata: bsd.tbxi sys/conf : newvers.sh sys/sys : param.h usr.bin/signify: signify.1 Log message: move to 6.6-beta ``` Preliminary release notes (https://www.openbsd.org/66.html) Improved hardware support, including: clang(1) is now provided on powerpc. IEEE 802.11 wireless stack improvements: Generic network stack improvements: Installer improvements: Security improvements: + Routing daemons and other userland network improvements + The ntpd(8) daemon now gets and sets the clock in a secure way when booting even when a battery-backed clock is absent. + bgdp(8) improvements + Assorted improvements: + The filesystem buffer cache now more aggressively uses memory outside the DMA region, to improve cache performance on amd64 machines. The BER API previously internal to ldap(1), ldapd(8), ypldap(8), and snmpd(8) has been moved into libutil. See berreadelements(3). Support for specifying boot device in vm.conf(5). OpenSMTPD 6.6.0 LibreSSL 3.0.X API and Documentation Enhancements Completed the port of RSA_METHOD accessors from the OpenSSL 1.1 API. Documented undescribed options and removed unfunctional options description in openssl(1) manual. OpenSSH 8.0 Project Trident 12-U5 update now available (https://project-trident.org/post/2019-09-04_stable12-u5_available/) This is the fifth general package update to the STABLE release repository based upon TrueOS 12-Stable. Package changes from Stable 12-U4 Package Summary New Packages: 20 Deleted Packages: 24 Updated Packages: 279 New Packages (20) artemis (biology/artemis) : 17.0.1.11 catesc (games/catesc) : 0.6 dmlc-core (devel/dmlc-core) : 0.3.105 go-wtf (sysutils/go-wtf) : 0.20.0_1 instead (games/instead) : 3.3.0_1 lidarr (net-p2p/lidarr) : 0.6.2.883 minerbold (games/minerbold) : 1.4 onnx (math/onnx) : 1.5.0 openzwave-devel (comms/openzwave-devel) : 1.6.897 polkit-qt-1 (sysutils/polkit-qt) : 0.113.0_8 py36-traitsui (graphics/py-traitsui) : 6.1.2 rubygem-aws-sigv2 (devel/rubygem-aws-sigv2) : 1.0.1 rubygem-defaultvaluefor32 (devel/rubygem-defaultvaluefor32) : 3.2.0 rubygem-ffi110 (devel/rubygem-ffi110) : 1.10.0 rubygem-zeitwerk (devel/rubygem-zeitwerk) : 2.1.9 sems (net/sems) : 1.7.0.g20190822 skypat (devel/skypat) : 3.1.1 tvm (math/tvm) : 0.4.1440 vavoom (games/vavoom) : 1.33_15 vavoom-extras (games/vavoom-extras) : 1.30_4 Deleted Packages (24) geeqie (graphics/geeqie) : Unknown reason iriverter (multimedia/iriverter) : Unknown reason kde5 (x11/kde5) : Unknown reason kicad-doc (cad/kicad-doc) : Unknown reason os-nozfs-buildworld (os/buildworld) : Unknown reason os-nozfs-userland (os/userland) : Unknown reason os-nozfs-userland-base (os/userland-base) : Unknown reason os-nozfs-userland-base-bootstrap (os/userland-base-bootstrap) : Unknown reason os-nozfs-userland-bin (os/userland-bin) : Unknown reason os-nozfs-userland-boot (os/userland-boot) : Unknown reason os-nozfs-userland-conf (os/userland-conf) : Unknown reason os-nozfs-userland-debug (os/userland-debug) : Unknown reason os-nozfs-userland-devtools (os/userland-devtools) : Unknown reason os-nozfs-userland-docs (os/userland-docs) : Unknown reason os-nozfs-userland-lib (os/userland-lib) : Unknown reason os-nozfs-userland-lib32 (os/userland-lib32) : Unknown reason os-nozfs-userland-lib32-development (os/userland-lib32-development) : Unknown reason os-nozfs-userland-rescue (os/userland-rescue) : Unknown reason os-nozfs-userland-sbin (os/userland-sbin) : Unknown reason os-nozfs-userland-tests (os/userland-tests) : Unknown reason photoprint (print/photoprint) : Unknown reason plasma5-plasma (x11/plasma5-plasma) : Unknown reason polkit-qt5 (sysutils/polkit-qt) : Unknown reason secpanel (security/secpanel) : Unknown reason Beastie Bits DragonFlyBSD - msdosfs updates (https://www.dragonflydigest.com/2019/09/10/23472.html) Stand out as a speaker (https://science.sciencemag.org/content/365/6455/834.full) Not a review of the 7th Gen X1 Carbon (http://akpoff.com/archive/2019/not_a_review_of_the_lenovo_x1c7.html) FreeBSD Meets Linux At The Open Source Summit (https://www.tfir.io/2019/08/24/freebsd-meets-linux-at-the-open-source-summit/) QEMU VM Escape (https://blog.bi0s.in/2019/08/24/Pwn/VM-Escape/2019-07-29-qemu-vm-escape-cve-2019-14378/) Porting wine to amd64 on NetBSD, third evaluation report. (http://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on1) OpenBSD disabled DoH by default in Firefox (https://undeadly.org/cgi?action=article;sid=20190911113856) Feedback/Questions Reinis - GELI with UEFI (http://dpaste.com/0SG8630#wrap) Mason - Beeping (http://dpaste.com/1FQN173) [CHVT feedback] DJ - Feedback (http://dpaste.com/08M3XNH#wrap) Ben - chvt (http://dpaste.com/274RVCE#wrap) Harri - Marc's chvt question (http://dpaste.com/23R1YMK#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
315: Recapping vBSDcon 2019
vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more. Headlines vBSDcon Recap Allan and Benedict attended vBSDcon 2019, which ended last week. It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks. The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails. If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract (https://www.vbsdcon.com/schedule/2019-09-06.html#talk:132615) and the recent commit we covered in episode 313. Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs. David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD. Shawn Webb followed with his overview talk about the “State of the Hardened Union”. Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality. Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale. People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts. Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”. Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads. “By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms. Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”. Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”. A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night. We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees! humungus - an hg server (https://humungus.tedunangst.com/r/humungus) Features View changes, files, changesets, etc. Some syntax highlighting. Read only. Serves multiple repositories. Allows cloning via the obvious URL. Supports go get. Serves files for downloads. Online documentation via mandoc. Terminal based admin interface. News Roundup OpenBSD on fan-less Tuxedo InfinityBook 14″ v2. (https://hazardous.org/archive/blog/openbsd/2019/09/02/OpenBSD-on-Infinitybook14) The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.). I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop. The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta. See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader Unix at 50: How the OS that powered smartphones started from failure (https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/) Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey. It was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term. Trying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.” Within the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for. Still, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote. Cancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals. Some of Allan’s favourite excerpts: In the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor. And so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics. With the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget. McIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it. It was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.” Eventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document. Of course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles. In August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task. Thompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics. By the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer. It wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications. The computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy. The rest has quite literally made tech history. See the link for the rest of the article How to configure a network dump in FreeBSD? (https://www.oshogbo.vexillium.org/blog/68/) A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump. So, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0. Now, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client. See the link for the rest of the article Beastie Bits Sudo Mastery 2nd edition is not out (https://mwl.io/archives/4530) Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development (http://users.utu.fi/kakrind/publications/19/vulnfuzz_camera.pdf) soso (https://github.com/ozkl/soso) GregKH - OpenBSD was right (https://youtu.be/gUqcMs0svNU?t=254) Game of Trees (https://gameoftrees.org/faq.html) Feedback/Questions BostJan - Another Question (http://dpaste.com/1ZPCCQY#wrap) Tom - PF (http://dpaste.com/3ZSCB8N#wrap) JohnnyK - Changing VT without keys (http://dpaste.com/3QZQ7Q5#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
314: Swap that Space
Unix virtual memory when you have no swap space, Dsynth details on Dragonfly, Instant Workstation on FreeBSD, new servers new tech, Experimenting with streaming setups on NetBSD, NetBSD’s progress towards Steam support thanks to GSoC, and more. Headlines What has to happen with Unix virtual memory when you have no swap space (https://utcc.utoronto.ca/~cks/space/blog/unix/NoSwapConsequence) Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine): Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...] I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM. To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program. See link for the rest of the article Dsynth details on Dragonfly (https://www.dragonflydigest.com/2019/08/27/23398.html) First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load. Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes. dsynth DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's). The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied. The intent is to make dsynth compatible with synth's configuration files and directory structure. This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts). dsynth code (https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/usr.bin/dsynth/dsynth.1) News Roundup Instant Workstation (https://euroquis.nl/freebsd/2019/08/12/instant-workstation.html) Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly. So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager. The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think. In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around. Here is the script in my GitHub repository with notes-for-myself. (https://raw.githubusercontent.com/adriaandegroot/FreeBSDTools/master/bin/instant-workstation) New Servers, new Tech (https://www.dragonflydigest.com/2019/08/26/23396.html) Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure. New servers in the colo, monster is being retired (http://lists.dragonflybsd.org/pipermail/users/2019-August/358271.html) We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency. With SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner. Monster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be. But we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot. I would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly. Experimenting with streaming setups on NetBSD (https://dressupgeekout.blogspot.com/2019/08/experimenting-with-streaming-setups-on.html?m=1) Ever since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward. Capturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK. My laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen. NetBSD Made Progress Thanks To GSoC In Its March Towards Steam Support (https://www.phoronix.com/scan.php?page=news_item&px=NetBSD-Linux-DRM-Ioctl-GSoC2019) Ultimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support. Student developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer. These interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer. Those curious about this DRM ioctl GSoC project can learn more from the NetBSD blog (https://blog.netbsd.org/tnf/entry/gsoc_2019_report_implementation_of). NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux. Beastie Bits FreeBSD in Wellington? (https://twitter.com/MengTangmu/status/1163265206660694016) FreeBSD on GFE (https://twitter.com/onewilshire/status/1163792878642114560) Clarification (https://twitter.com/onewilshire/status/1166323112620826624) Distrotest.net now with BSDs (https://distrotest.net/) Lecture: Anykernels meet fuzzing NetBSD (https://fahrplan.events.ccc.de/camp/2019/Fahrplan/events/10334.html) Sun Microsystems business plan from 1982 [pdf] (https://www.khoslaventures.com/wp-content/uploads/SunMicrosystem_bus_plan.pdf) Feedback/Questions Alan - Questions (http://dpaste.com/1Z8EGTW) Rodriguez - Feedback and a question (http://dpaste.com/2PZFP4X#wrap) Jeff - OpenZFS follow-up, FreeBSD Adventures (http://dpaste.com/02ZM6YE#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
313: In-Kernel TLS
OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more. Headlines OpenBSD on the Thinkpad X1 Carbon 7th Gen (https://jcs.org/2019/08/14/x1c7) Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery. The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life. Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile. On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck. See link for the rest of the article How To Install FreeBSD On A MacBook 1,1 or 2,1 (http://lexploit.com/freebsdmacbook1-1-2-1/) FreeBSD Setup For MacBook 1,1 and 2,1 FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12. Installing FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is: A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work. A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks. An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks. Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots. See link for the rest of the guide News Roundup Patch for review: Kernel portion of in-kernel TLS (KTLS) (https://svnweb.freebsd.org/base?view=revision&revision=351522) One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later. https://reviews.freebsd.org/D21277 DragonFly Boot Enviroments (https://github.com/newnix/dfbeadm) This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl. See link for the rest of the details Project Trident Updates 19.08 Available (https://project-trident.org/post/2019-08-15_19.08_available/) This is a general package update to the CURRENT release repository based upon TrueOS 19.08. Legacy boot ISO functional again This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems. PACKAGE CHANGES FROM 19.07-U1 New Packages: 154 Deleted Packages: 394 Updated Packages: 4926 12-U3 Available (https://project-trident.org/post/2019-08-22_stable12-u3_available/) This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable. PACKAGE CHANGES FROM STABLE 12-U2 New Packages: 105 Deleted Packages: 386 Updated Packages: 1046 vBSDcon (https://www.vbsdcon.com/schedule/) vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. *** Beastie Bits The next NYCBUG meeting will be Sept 4 @ 18:45 (https://www.nycbug.org/index?action=view&id=10671) Feedback/Questions Tom - Questions (http://dpaste.com/1AXXK7G#wrap) Michael - dfbeadm (http://dpaste.com/0PNEDYT#wrap) Bostjan - Questions (http://dpaste.com/1N7T7BR#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
312: Why Package Managers
The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more. Headlines The UNIX Philosophy in 2019 (https://triosdevelopers.com/jason.eckert/blog/Entries/2019/6/1_Entry_1.html) Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973: We write programs that do one thing and do it well We write programs to work together And we write programs that handle text streams, because that is a universal interface Why Use Package Managers? (https://uwm.edu/hpc/software-management/) Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case. Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed. In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few. Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this? The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science. Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically. News Roundup Touchpad, Interrupted (https://jcs.org/2019/07/28/ihidev) For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it. It's been a long journey and it's a technical tale, but here it is. Porting wine to amd64 on NetBSD, second evaluation report (https://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on2) Summary Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LDLIBRARYPATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity. Enhancing Syzkaller Support for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd) As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period. You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancingsyzkallersupportfornetbsd July Update: All about the Pinebook Pro (https://www.pine64.org/2019/07/05/july-update-all-about-the-pinebook-pro/) "So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available." Killing a process and all of its descendants (http://morningcoffee.io/killing-a-process-and-all-of-its-descendants.html) Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned: Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number. Sending signals to all processes in a session is not trivial with syscalls. Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children. The answer to the “What happens with orphaned process groups” question is not trivial. Fast Software, the Best Software (https://craigmod.com/essays/fast_software/) I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness. Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why. But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools. A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.) Beastie Bits Register for vBSDCon 2019, Sept 5-7 in Reston VA (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions Paulo - FreeNAS Question (http://dpaste.com/2GDG7WR#wrap) Marc - Changing VT without function keys? (http://dpaste.com/1AKC7A1#wrap) Caleb - Patch, update, and upgrade management (http://dpaste.com/2D6J482#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
311: Conference Gear Breakdown
NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more. Headlines NetBSD 9.0 release process has started (https://mail-index.netbsd.org/netbsd-announce/2019/07/31/msg000301.html) If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on: + New AArch64 architecture support: + Symmetric and asymmetrical multiprocessing support (aka big.LITTLE) + Support for running 32-bit binaries + UEFI and ACPI support + Support for SBSA/SBBR (server-class) hardware. + The FDT-ization of many ARM boards: + the 32-bit GENERIC kernel lists 129 different DTS configurations + the 64-bit GENERIC64 kernel lists 74 different DTS configurations + All supported by a single kernel, without requiring per-board configuration. + Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices. + ZFS has been updated to a modern version and seen many bugfixes. + New hardware-accelerated virtualization via NVMM. + NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default. + NVMe performance improvements + Optional kernel ASLR support, and partial kernel ASLR for the default configuration. + Kernel sanitizers: + KLEAK, detecting memory leaks + KASAN, detecting memory overruns + KUBSAN, detecting undefined behaviour + These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed. + The removal of outdated networking components such as ISDN and all of its drivers + The installer is now capable of performing GPT UEFI installations. + Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding. + Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU. We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find. + Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/ xargs wtf (https://medium.com/@aarontharris/xargs-wtf-34d2618286b7) xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too. I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better. xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else. It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem: This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept. News Roundup PkgSrc: A Tale of Two Spellcheckers (https://bentsukun.ch/posts/pkgsrccon-2019/) This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study. The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole. Adapting TriforceAFL for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1) I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation. For work done during the first coding period, check out this post. Summary > So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation. > Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far! Exploiting a no-name freebsd kernel vulnerability (https://www.synacktiv.com/posts/exploit/exploiting-a-no-name-freebsd-kernel-vulnerability.html) A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions. > A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory. [Allan and Benedicts Conference Gear Breakdown] Benedict’s Gear: GlocalMe G3 Mobile Travel HotSpot and Powerbank (https://www.glocalme.com/CA/en-US/cloudsim/g3) Mogics Power Bagel (http://www.mogics.com/3824-2) Charby Sense Power Cable (https://charbycharge.com/charby-sense-worlds-smartest-auto-cutoff-cable/) Allan’s Gear: Huawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro (https://smile.amazon.com/gp/product/B013CEGGKI/) AOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries (https://smile.amazon.com/dp/B071HJFX27/) All my devices charge from USB-C, so that is great More USB thumb drives than strictly necessary My Lenovo X270 laptop running FreeBSD 13-current My 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work Beastie Bits Replacing the Unix tradition (Warning may be rage inducing) (https://www.youtube.com/watch?v=L9v4Mg8wi4U&feature=youtu.be) Installing OpenBSD over remote serial on the AtomicPI (https://www.thanassis.space/remoteserial.html#remoteserial) Zen 2 and DragonFly (https://www.dragonflydigest.com/2019/08/05/23294.html) Improve Docking on FreeBSD (https://blog.yukiisbo.red/posts/2019/05/improve-docking-on-freebsd/) Register for vBSDCon 2019, Sept 5-7 in Reston VA. Early bird ends August 15th. (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions JT - Congrats (http://dpaste.com/0D7Y31E#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
310: My New Free NAS
OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more. Headlines OPNsense 19.7.1 (https://opnsense.org/opnsense-19-7-1-released/) We do not wish to keep you from enjoying your summer time, but this is a recommended security update enriched with reliability fixes for the new 19.7 series. Of special note are performance improvements as well as a fix for a longstanding NAT before IPsec limitation. Full patch notes: system: do not create automatic copies of existing gateways system: do not translate empty tunables descriptions system: remove unwanted form action tags system: do not include Syslog-ng in rc.freebsd handler system: fix manual system log stop/start/restart system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead system: allow curl-based downloads to use both trusted and local authorities system: fix group privilege print and correctly redirect after edit system: use cached address list in referrer check system: fix Syslog-ng search stats firewall: HTML-escape dynamic entries to display aliases firewall: display correct IP version in automatic rules firewall: fix a warning while reading empty outbound rules configuration firewall: skip illegal log lines in live log interfaces: performance improvements for configurations with hundreds of interfaces reporting: performance improvements for Python 3 NetFlow aggregator rewrite dhcp: move advanced router advertisement options to correct config section ipsec: replace global array access with function to ensure side-effect free boot ipsec: change DPD action on start to "dpdaction = restart" ipsec: remove already default "dpdaction = none" if not set ipsec: use interface IP address in local ID when doing NAT before IPsec web proxy: fix database reset for Squid 4 by replacing use of sslcrtd with securityfile_certgen plugins: os-acme-client 1.24[1] plugins: os-bind 1.6[2] plugins: os-dnscrypt-proxy 1.5[3] plugins: os-frr now restricts characters BGP prefix-list and route-maps[4] plugins: os-google-cloud-sdk 1.0[5] ports: curl 7.65.3[6] ports: monit 5.26.0[7] ports: openssh 8.0p1[8] ports: php 7.2.20[9] ports: python 3.7.4[10] ports: sqlite 3.29.0[11] ports: squid 4.8[12] Stay safe and hydrated, Your OPNsense team ZFS on Linux still has annoying issues with ARC size (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxARCShrinkage) One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there. Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.) This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case). The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all). News Roundup Hammer2 is now default (http://lists.dragonflybsd.org/pipermail/commits/2019-June/718989.html) ``` commit a49112761c919d42d405ec10252eb0553662c824 Author: Matthew Dillon Date: Mon Jun 10 17:53:46 2019 -0700 installer - Default to HAMMER2 * Change the installer default from HAMMER1 to HAMMER2. * Adjust the nrelease build to print the location of the image files when it finishes. Summary of changes: nrelease/Makefile | 2 +- usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824 ``` NetBSD audio – an application perspective (https://netbsd.org/gallery/presentations/nia/netbsd-audio/) NetBSD audio – an application perspective ... or, "doing it natively, because we can" audio options for NetBSD in pkgsrc Use NetBSD native audio (sun audio/audioio.h) Or OSS emulation layer: Basically a wrapper around sun audio in the kernel. Incomplete and old version, but works for simple stuff Many many abstraction layers available: OpenAL-Soft alsa-lib (config file required) libao, GStreamer (plugins!) PortAudio, SDL PulseAudio, JACK ... lots more!? some obsolete stuff (esd, nas?) Advantages of using NetBSD audio directly Low latency, low CPU usage: Abstraction layers differ in latency (SDL2 vs ALSA/OpenAL) Query device information: Is /dev/audio1 a USB microphone or another sound card? Avoid bugs from excessive layering Nice API, well documented: [nia note: I had no idea how to write audio code. I read a man page and now I do.] Your code might work on illumos too [nia note: SDL2 seems very sensitive to the blk_ms sysctl being high or low, with other implementations there seems to be a less noticable difference. I don't know why.] New FreeNAS Mini (https://www.ixsystems.com/blog/new-freenas-mini-models-release-pr/) Two new FreeNAS Mini systems join the very popular FreeNAS Mini and Mini XL: FreeNAS Mini XL+: This powerful 10 Bay platform (8x 3.5” and 1x 2.5” hot-swap, 1x 2.5” internal) includes the latest, compact server technology and provides dual 10GbE ports, 8 CPU cores and 32 GB RAM for high performance workgroups. The Mini XL+ scales beyond 100TB and is ideal for very demanding applications, including hosting virtual machines and multimedia editing. Starting at $1499, the Mini XL+ configured with cache SSD and 80 TB capacity is $4299, and consumes about 100 Watts. FreeNAS Mini E: This cost-effective 4 Bay platform provides the resources required for SOHO use with quad GbE ports and 8 GB of RAM. The Mini E is ideal for file sharing, streaming and transcoding video at 1080p. Starting at $749, the Mini E configured with 8 TB capacity is $999, and consumes about 36 Watts. Beastie Bits Welcome to NetBSD 9.99.1! (https://mail-index.netbsd.org/source-changes/2019/07/30/msg107671.html) Berkeley smorgasbord — part II (http://blog.snailtext.com/posts/berkeley-smorgasbord-part-2.html) dtracing postgres (https://www.youtube.com/watch?v=Brt41xnMZqo&list=PLuJmmKtsV1dOTmlImlD9U5j1P1rLxS2V8&index=20&t=0s) Project Trident 19.07-U1 now available (https://project-trident.org/post/2019-07-30_19.07-u1_available/) Need a Secure Operating System? Take a Look at OpenBSD (https://www.devprojournal.com/technology-trends/operating-systems/need-a-secure-operating-system-take-a-look-at-openbsd/) Feedback/Questions Jeff - OpenZFS Port Testing Feedback (http://dpaste.com/2AT7JGP#wrap) Malcolm - Best Practices for Custom Ports (http://dpaste.com/1R170D7) Michael - Little Correction (http://dpaste.com/0CERP6R) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Episode 309: Get Your Telnet Fix
DragonFlyBSD Project Update - colo upgrade, future trends, resuming ZFS send, realtime bandwidth terminal graph visualization, fixing telnet fixes, a chapter from the FBI’s history with OpenBSD and an OpenSSH vuln, and more. Headlines DragonFlyBSD Project Update - colo upgrade, future trends (http://lists.dragonflybsd.org/pipermail/users/2019-July/358226.html) For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone. The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system. Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while. Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get. The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time. Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going. Resuming ZFS send (https://www.oshogbo.vexillium.org/blog/66/) One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot? For a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off. In this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice. News Roundup Realtime bandwidth terminal graph visualization (https://dataswamp.org/~solene/2019-07-19-ttyplot-netstat-openbsd.html) If for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal. The following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5. fixing telnet fixes (https://flak.tedunangst.com/post/fixing-telnet-fixes) There’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations. The first line is indented with spaces while the others use tabs. The correct type for string length is size_t not unsigned int. sizeof(char) is always one. There’s no need to multiply by it. If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.) Return value of malloc doesn’t need to be cast. In fact, should not be, lest you disguise a warning. Return value of malloc is not checked for NULL. No reason to cast cp to char * when passing to snprintf. It already is that type. And if it weren’t, what are you doing? The whole operation could be simplified by using asprintf. Although unlikely (probably impossible here, but more generally), adding the two source lengths together can overflow, resulting in truncation with an unchecked snprintf call. asprintf avoids this failure case. A Chapter from the FBI’s History with OpenBSD and an OpenSSH Vuln (https://twitter.com/RooneyMcNibNug/status/1152327783055601664) Earlier this year I FOIAed the FBI for details on allegations of backdoor installed in the IPSEC stack in 2010, originally discussed by OpenBSD devs (https://marc.info/?l=openbsd-tech&m=129236621626462 …) Today, I got an interesting but unexpected responsive record: Freedom of Information Act: FBI: OpenBSD (https://www.muckrock.com/foi/united-states-of-america-10/foia-fbi-openbsd-70084/) GitHub Repo (https://github.com/RooneyMcNibNug/FOIA/blob/master/Responsive%20Docs/OpenBSD/FBI_OpenBSD_response_OCRd.pdf) Beastie Bits “Sudo Mastery, 2nd Edition” open for tech review (https://mwl.io/archives/4378) FreeBSD Journal: FreeBSD for Makers (https://www.freebsdnews.com/2019/07/12/freebsd-journal-freebsd-for-makers/) OpenBSD and NetBSD machines at Open Source Conference 2019 Nagoya (http://mail-index.netbsd.org/netbsd-advocacy/2019/07/19/msg000808.html) FreeBSD 12.0: WINE Gaming (https://www.youtube.com/watch?v=zuj9pRNR2oM) Introduction to the Structure and Interpretation of TNF (The NetBSD Foundation) (https://www.netbsd.org/gallery/presentations/wiz/pkgsrccon2019/index.html#/) vBSDcon speakers announced (https://www.vbsdcon.com/) Feedback/Questions Pat - NYCBug Aug 7th (http://dpaste.com/21Y1PRM) Tyler - SSH keys vs password (http://dpaste.com/3JEVVEF#wrap) Lars - Tor-Talk (http://dpaste.com/0RAFMXZ) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
308: Mumbling with OpenBSD
Replacing a (silently) failing disk in a ZFS pool, OPNsense 19.7 RC1 released, implementing DRM ioctl support for NetBSD, High quality/low latency VOIP server with umurmur/Mumble on OpenBSD, the PDP-7 where Unix began, LLDB watchpoints, and more. Headlines Replacing a (silently) failing disk in a ZFS pool (https://imil.net/blog/2019/07/02/Replacing-a-silently-failing-disk-in-a-ZFS-pool/) Maybe I can’t read, but I have the feeling that official documentations explain every single corner case for a given tool, except the one you will actually need. My today’s struggle: replacing a disk within a FreeBSD ZFS pool. What? there’s a shitton of docs on this topic! Are you stupid? I don’t know, maybe. Yet none covered the process in a simple, straight and complete manner. OPNsense 19.7 RC1 released (https://opnsense.org/opnsense-19-7-rc1-released/) Hi there, For four and a half years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. We thank all of you for helping test, shape and contribute to the project! We know it would not be the same without you. Download links, an installation guide[1] and the checksums for the images can be found below as well. News Roundup Implementation of DRM ioctl Support for NetBSD kernel (https://blog.netbsd.org/tnf/entry/implementation_of_drm_ioctl_support) What is DRM ioctl ? Ioctls are input/output control system calls and DRM stands for direct rendering manager The DRM layer provides several services to graphics drivers, many of them driven by the application interfaces it provides through libdrm, the library that wraps most of the DRM ioctls. These include vblank event handling, memory management, output management, framebuffer management, command submission & fencing, suspend/resume support, and DMA services. Native DRM ioctl calls NetBSD was able to make native DRM ioctl calls with hardware rendering once xorg and proper mesa packages where installed. We used the glxinfo and glxgears applications to test this out. High quality / low latency VOIP server with umurmur/Mumble on OpenBSD (https://dataswamp.org/~solene/2019-07-04-umurmur.html) Discord users keep telling about their so called discord server, which is not dedicated to them at all. And Discord has a very bad quality and a lot of voice distorsion. Why not run your very own mumble server with high voice quality and low latency and privacy respect? This is very easy to setup on OpenBSD! Mumble is an open source voip client, it has a client named Mumble (available on various operating system) and at least Android, the server part is murmur but there is a lightweight server named umurmur. People authentication is done through certificate generated locally and automatically accepted on a server, and the certificate get associated with a nickname. Nobody can pick the same nickname as another person if it’s not the same certificate. TMWL June’19 — JS Fetch API, scheduling in Spring, thoughts on Unix (https://blog.softwaremill.com/tmwl-june19-js-fetch-api-scheduling-in-spring-thoughts-on-unix-fd54f50ecd64) Unix — going back to the roots From time to time, I like to review my knowledge in a certain area, even when I feel like I know a lot about it already. I go back to the basics and read tutorials, manuals, books or watch interesting videos. I’ve been using macOS for a couple of years now, previously being a linux user for some (relatively short) time. Both these operating systems have a common ancestor — Unix. While I’m definitely not an expert, I feel quite comfortable using linux & macOS — I understand the concepts behind the system architecture, know a lot of command line tools & navigate through the shell without a hassle. So-called unix philosophy is also close to my heart. I always feel like there’s more I could squeeze out of it. Recently, I found that book titled “Unix for dummies, 5th edition” which was published back in… 2004. Feels literally like AGES in the computer-related world. However, it was a great shot — the book starts with the basics, providing some brief history of Unix and how it came to life. It talks a lot about the structure of the system and where certain pieces fit (eg. “standard” set of tools), and how to understand permissions and work with files & directories. There’s even a whole chapter about shell-based text editors like Vi and Emacs! Despite the fact that I am familiar with most of these, I could still find some interesting pieces & tools that I either knew existed (but never had a chance to use), or even haven’t ever heard of. And almost all of these are still valid in the modern “incarnations” of Unix’s descendants: Linux and macOS. The book also talks about networking, surfing the web & working with email. It’s cute to see pictures of those old browsers rendering “ancient” Internet websites, but hey — this is how it looked like no more than fifteen years ago! I can really recommend this book to anyone working on modern macOS or Linux — you will certainly find some interesting pieces. Especially if you like to go back to the roots from time to time as I do! ThePDP-7 Where Unix Began (https://bsdimp.blogspot.com/2019/07/the-pdp-7-where-unix-began.html) In preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine. V0 Unix could run on only one of the PDP-7s. Of the 99 PDP-7s produced, only two had disks. Serial number 14 had an RA01 listed, presumably a disk, though of a different type. In addition to the PDP-7 being obsolete in 1970, no other PDP-7 could run Unix, limiting its appeal outside of Bell Labs. By porting Unix to the PDP-11 in 1970, the group ensured Unix would live on into the future. The PDP-9 and PDP-15 were both upgrades of the PDP-7, so to be fair, PDP-7 Unix did have a natural upgrade path (the PDP-11 out sold the 18 bit systems though ~600,000 to ~1000). Ken Thompson reports in a private email that there were 2 PDP-9s and 1 PDP-15 at Bell Labs that could run a version of the PDP-7 Unix, though those machines were viewed as born obsolete. LLDB: watchpoints, XSTATE in ptrace() and core dumps (https://blog.netbsd.org/tnf/entry/lldb_watchpoints_xstate_in_ptrace) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types and fix compat32 issues. You can read more about that in my May 2019 report. In June, I have finally finished the remaining ptrace() work for xstate and got it merged both on NetBSD and LLDB end (meaning it's going to make it into NetBSD 9). I have also worked on debug register support in LLDB, effectively fixing watchpoint support. Once again I had to fight some upstream regressions. Beastie Bits Project Trident 19.07 Available (https://project-trident.org/post/2019-07-12_19.07_available/) A list of names from "Cold Blood" -- Any familiar? (https://www.montanalinux.org/cold-blood-list-of-numbers-201907.html) fern: a curses-based mastodon client modeled off usenet news readers & pine, with an emphasis on getting to 'timeline zero' (https://github.com/enkiv2/fern) OpenBSD Community goes Platinum for 2019! (https://undeadly.org/cgi?action=article;sid=20190707065226) tcp keepalive and dports on DragonFly (https://www.dragonflydigest.com/2019/07/15/23199.html) Feedback/Questions Patrick - OpenZFS/ZoL Module from Ports (http://dpaste.com/1W2HJ04) Brad - Services not starting (http://dpaste.com/345VM9Y#wrap) Simon - Feedback (http://dpaste.com/1B4ZKC8#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.