Just some Internet guy

He/him/them 🏳️‍🌈

  • 13 Posts
  • 1.48K Comments
Joined 1 year ago
cake
Cake day: June 25th, 2023

help-circle


  • auto rollbacks and easy switching between states.

    That’s the beauty of snapshots, you can boot them. So you just need GRUB to generate the correct menu and you can boot any arbitrary version of your system. On the ZFS side of things there’s zfsbootmenu, but I’m pretty sure I’ve seen it for btrfs too. You don’t even need rsync, you can use ssh $server btrfs send | btrfs recv and it should in theory be faster too (btrfs knows if you only modified one block of a big file).

    and the current r/w system as the part that gets updated.

    That kind of goes against the immutable thing. What I’d do is make a script that mounts a fork of the current snapshot readwrite into a temporary directory, chroot into it, install packages, exit chroot, unmount and then commit those changes as a snapshot. That’s the closest I can think of that’s easy to DIY that’s basically what rpm-ostree install does. It does it differently (daemon that manages hardlinks), but filesystem snapshots basically do the same thing without the extra work.

    However, I think it would be good to use OStree

    I found this, maybe it’ll help: https://ostreedev.github.io/ostree/adapting-existing/

    It looks like the fundamental is the same, temporary directory you run the package manager into and then you commit the changes. So you can probably make it work with Debian if you want to spend the time.


  • All you really have to do for that is mount the partition readonly, and have a designated writable data partition for the rest. That can be as simple as setting it ro in your fstab.

    How you ship updates can take many forms. If you don’t need your distro atomic, you can temporarily remount readwrite, rsync the new version over and make it readonly again. If you want it atomic, there’s the classic A/B scheme (Android, SteamOS), where you just download the image to the inactive partition and then just switch over when it’s ready to boot into. You can also do btrfs/ZFS snapshots, where the current system is forked off a snapshot. On your builder you just make your changes, then take a snapshot, then zfs/btrfs send it as a snapshot to all your other machines and you just boot off that new snapshot (readonly). It’s really not that magic: even Docker, if you dig deep enough, it’s just essentially tarballs being downloaded then extracted each in their own folder, and the layering actually comes from stacking them with overlayfs. What rpm-ostree does, from a quick glance at the docs, is they leverage the immutability and just build a new version of the filesystem using hardlinks and you just switch root to it. If you’ve ever opened an rpm or deb file, it’s just a regular tarball and the contents pretty much maps directly to the filesytem.

    Here’s an Arch package example, but rpm/deb are about the same:

    max-p@desktop /v/c/p/aur> tar -tvf zfs-utils-2.2.6-3-x86_64.pkg.tar.zst 
    -rw-r--r-- root/root    114771 2024-10-13 01:43 .BUILDINFO
    drwxr-xr-x root/root         0 2024-10-13 01:43 etc/
    drwxr-xr-x root/root         0 2024-10-13 01:43 etc/bash_completion.d/
    -rw-r--r-- root/root     15136 2024-10-13 01:43 etc/bash_completion.d/zfs
    -rw-r--r-- root/root     15136 2024-10-13 01:43 etc/bash_completion.d/zpool
    drwxr-xr-x root/root         0 2024-10-13 01:43 etc/default/
    -rw-r--r-- root/root      4392 2024-10-13 01:43 etc/default/zfs
    drwxr-xr-x root/root         0 2024-10-13 01:43 etc/zfs/
    -rw-r--r-- root/root       165 2024-10-13 01:43 etc/zfs/vdev_id.conf.alias.example
    -rw-r--r-- root/root       166 2024-10-13 01:43 etc/zfs/vdev_id.conf.multipath.example
    -rw-r--r-- root/root       616 2024-10-13 01:43 etc/zfs/vdev_id.conf.sas_direct.example
    -rw-r--r-- root/root       152 2024-10-13 01:43 etc/zfs/vdev_id.conf.sas_switch.example
    -rw-r--r-- root/root       254 2024-10-13 01:43 etc/zfs/vdev_id.conf.scsi.example
    drwxr-xr-x root/root         0 2024-10-13 01:43 etc/zfs/zed.d/
    ...
    

    It’s beautifully simple. You could for example install ArchLinux without pacman, by mostly just tar -x the individual package files directly to /. All the package manager does is track which file is owned by which package (so it’s easier to remove), and dependency solving so it knows to go pull more stuff or it won’t work, and mirror/download management.

    How you get that set up is all up to you. Packer+Ansible can make you disk images and you can maybe just throw them on a web server and download them and dd them to the inactive partition of an A/B scheme, and that’d be quite distro-agnostic too. You could build the image as a Docker container and export it as a tarball. You can build a chroot. Or a systemd-nspawn instance. You can also just install a VM yourself and set it up to your liking and then just dd the disk image to your computers.

    If you want some information on how SteamOS does it, https://iliana.fyi/blog/build-your-own-steamos-updates/




  • Pop_OS! is about to drop a whole new desktop environment (COSMIC) made from scratch that’s not just a fork of Gnome. Canonical tried that as well a while back with Unity although it was mostly still Gnome with extra Compiz plugins.

    A lot of cool stuff is also either for enterprise uses, or generally under the hood stuff. Simple packages updates can mean someone’s GPU is finally usable. Even that LibreOffice update might mean someone’s annoying bug is finally fixed.

    But yes otherwise distros are mostly there to bundle up and configure the software for you. It’s really just a bunch of software, you can get the exact same experience making your own with LFS. Distros also make some choices like what are the best versions to bundle up as a release, what software and features they’re gonna use. Distros make choices for you like glibc/musl, will it use PulseAudio or PipeWire, and so on. Some distros like Bazzite are all about a specific use case (gamers), and all they do is ship all the latest tweaks and patches so all the handhelds behave correctly and just run the damn games out of the box. You can use regular Fedora but they just have it all good to go for you out of the box. That’s valuable to some people.

    Sometimes not much is going on in open-source so it just makes for boring releases. Also means likely more focus on bug fixes and stability.



  • You have to keep in mind, when you write JavaScript, there’s an entire runtime written in C++ to run it under the hood, with some crazy optimizations to make it reasonably performant. What type of languages do you use to write that runtime? A systems programming language like Rust and C++.

    You don’t have to use Rust if you don’t like it. Not everything must be written in Rust. The whole pick a language also involves a lot of picking your tradeoffs. Picking a interpreted/JIT language for speed of development is a perfectly valid tradeoff, but not one you can universally make. Sometimes the performance cost becomes really expensive currency-wise, where you can save thousands of dollars on server costs by simply having a more efficient application that only needs a fraction of the hardware to run it. Even in JavaScript, a fair chunk of libraries you use end up calling to C++ native code because it would be too slow in pure JavaScript. Sometimes the tradeoff is pick the popular language so it’s easier to hire for cheaper.

    Even at the dawn of time, most computers shipped with a variant of BASIC so people could write simple applications easily. But if you wanted to squeeze out every bit of power in your Apple II or C64, you sure did reach for assembly. Assembly sucks so we made C, then C++. Rust is still a language that’s made to eventually compile to assembly/binary and have the same performance as if you wrote it in assembly.

    And low spec hardware still exists: the regular Pis have gotten pretty fast but if you run on an RP2040 then suddenly, you’re back in like 300MHz dual core land with pitiful amounts of memory, so you do need to write optimized and fast code for those.

    Rust’s type system is actually really, really good. Most of the time, if it compiles it runs. It eliminates a ton of errors other than memory safety: the system is so powerful you can straight up make invalid state unrepresentable. You can’t forget to close a connection, you can’t pass the wrong data, you can’t forget to unlock a lock. It does a lot more to enforce correctness of a program well beyond memory safety.



  • Perspective du Québec, je dirais même que le titre fait parti du problème.

    Ça fait une dizaine d’années que j’ai terminé mes études alors ça peut avoir changé pas mal, mais ce qui me surprenait pas mal à l’époque en parlant avec mes amis français, c’est à quel point les études sont compétitives en France. Faut toujours être meilleur que l’autre, faut toujours plus de points, la recherche éternelle de l’école prestigieuse et tout, un peu comme aux États-Unis. Au final ça fait des gens qui se croient le meilleur au monde et qui se la pête constamment.

    Au Québec ils évitent que les élèves puissent se comparer, de sorte à ce que l’attention soit pleinement sur l’apprentissage. L’important c’est de bien apprendre et surtout comprendre les concepts, plutôt que de mesurer à quelle vitesse tu peux régurgiter une formule mathématique. L’école c’est l’école, laquelle tu va n’a pas vraiment d’importance non plus. Avoir 2 points de plus que l’autre ne veut rien dire non plus en entrevue, la personne dans son ensemble est évaluée, parce que le type avec le score parfait il est souvent très chiant de travailler avec.

    Et puis, au final, on m’a jamais demandé mes diplomes ou même si j’ai passé, du coup l’école c’était une sacré perte de temps au final.






  • I had to block ByteSpider at work because it can’t even parse HTML correctly and just hammers the same page and accounts to sometimes 80% of the traffic hitting a customer’s site and taking it down.

    The big problem with AI scrapers is unlike Google and traditional search engines, they just scrape so aggressively. Even if it’s all GETs, they hit years old content that’s not cached and use up the majority of the CPU time on the web servers.

    Scraping is okay, using up a whole 8 vCPU instance for days to feed AI models is not. They even actively use dozens of IPs to bypass the rate limits too, so theyre basically DDoS’ing whoever they scrape with no fucks given. I’ve been woken up by the pager way too often due to ByteSpider.

    My next step is rewriting all the content with GPT-2 and serving it to bots so their models collapse.


  • That’s pretty much why I made my own instance: nobody can take it away from me. I can ban whichever instance I deem hostile or don’t want content from. Nobody’s taking away my API anymore or shoving ads in my face.

    Nobody can pull a Reddit or Twitter on the fediverse, there will always be alternative instances to use putting pressure on the big ones to not drive away people.