I’ve been working on a “Linux Inside Out” series and wrote a post that might interest folks here who like low(ish)-level / OS internals.
The idea is to dissect the components of a Linux OS, layer by layer, and build a mental model of how everything fits together through experiments.
The first part is about the kernel, in the post I:
- take the same kernel image my distro boots from /boot
- boot it directly with QEMU (no distro, no init system)
- watch it panic
- write a tiny Go program and use it as PID 1
- build a minimal initramfs around it so the kernel can actually start our process
The goal isn’t to build a real distro, just to give a concrete mental model of:
- that the Linux kernel is just a compressed file, you can boot it
- without anything else
- what the kernel actually does at boot
- how it hands control to userspace
- what PID 1 / init is in practice
- what is kernel space vs user space
Link: https://serversfor.dev/linux-inside-out/the-linux-kernel-is-just-a-program/
I’m the author, would be happy to hear from other devs whether this way of explaining things makes sense, and what you’d add or change for future posts in the series.
Hope you find it useful.
You must log in or # to comment.
I’m old enough to remember UML (User Mode Linux). I don’t know if it’s still around, but it was a port of the Linux kernel that you would run as a standard user binary.
Nitpicking but a line is missing IMHO namely
The code of the program:should also suggest which file to edit, e.gpotato.go. It might be obviously to anybody working with Go but for others it’s not.I took kernel device driver programming as a course in uni. They were still teaching the device driver course but not the operating system course you were supposed to take before that. Always felt like I have been systematically lacking that knowledge because of that. I’m interested!
Oh this is very cool. Never read a tutorial like this!
It’s pretty cool and a nice way to discover the kernel!
I like the “take things appart, recombine, look what breaks” approach, same line of thought.
Btw, i think the current tech stack (CPU, initram, X/Wayland server, input handling, pam/polkit) is just shoehorned Desktop on tech made for server and a lot of historical baggage.
Bookmarked 😜 Mind adding a dark mode?
Dark mode added.
Thank you! :) Sure, working on it. :)
Ah, please utilize prefers-color-scheme and just place your --color-variables in it, if possible. It’s the simplest and most compatible way.
This is amazing. Thank you!
Very cool, reminds me of https://jsandler18.github.io/tutorial/boot.html
FWIW makes me wonder how much work would be required to have this as a Web container, e.g. Dockerfile with
FROM debian:13 RUN apt update && apt install -y qemu-system-x86 qemu-utils WORKDIR /linux-inside-outthen https://github.com/container2wasm/container2wasm#container-on-browser
Edit: FWIW the image of Debian 13 with QEMU and its utils is ~1.1Gb
Cool! Looking forward to the next post. I’d love to learn about how the kernel and initramfs work together though. For now the magic seems to be in how qemu strings things together.
Very cool guide, got the kernel running! Looking forward to your next post!
You got a subscriber! Looking forward to more.
Are you just reverse-engineering this for fun, or are you trying to learn how qemu builds on a bootstrap?
I am writing a blog post series to make this topic more approachable for others.
Nice, and good job.
With respect:
- You are describing the init process, after the boot stuff is done. Not sure if you meant that or not.
- vmlinuz is the bootable kernel image distinct from the kernel itself. It contains the compressed bootbale ELF kernel image (which itself contains bvmlinuz) and gzip decompress function, bsetup code, bbootsect system.map, and a handful of commands for the kernel image to bootstrap itself.
- Kernel space doesn’t stop once user space is allocated memory and initialized, they both run at the same time. Maybe I read that wrong, but it didn’t seem right to me.
I want to be careful here not to discourage you, this is great exploration!
I realize I’m handing out unsolicited advice here, but when I was first learning about Unix/Linux kernels in the Solaris and HP/UX days, the thing that helped the process “click” for me was compiling a kernel and building an ELF. And if you’re going to continue on this journey (which I hope you do), you should probably read a bit on memory segmentation and broadly about assembly instructions.
Good luck!
Thank you for the feedback!
My goal with this series is to make Linux internals more approachable, so I’m intentionally keeping each post narrow and avoiding details that would overwhelm readers at this stage.
The primary target audience of these blog posts are developers, and I have a progression plan in mind how to build up layer by layer the necessary knowledge they need to know to understand how they program work/interact with the other parts of the OS. The boot process or the exact composition of the kernel image is irrelevant here in my opinion, they wont need to touch them on a VPS for example or on their dev machine.
Regarding the kernel space/user space I did not write that the kernel space stopped, I will come back to it in the next post that will be about system calls, we will discuss there that when our programs interact with the kernel we are switching between user and kernel mode back and forth, with the syscall instruction (on x86-64 systems).
Anyway, I really appreciate your feedback and that you put effort into it, and thank you for reading the post! :)
