qemu is a good way to experience with kernel hacking
Hopefully someone can update the LDD(linux device driver) and Linux kernel books. In fact Linux Foundation should sponsor such efforts since technical book like this is hard to make any profit.
deivid [3 hidden]5 mins ago
I've written a little bit about writing a driver & using QEMU to create a custom device for it at [0] & [1]
Are you the David V from Meta, who had bytelab.codes? I recently discovered that blog, and was very excited by the content, only to find he last updated in 2022. Either way, I’m excited to see your site, too! I love finding well-written kernel-level stuff.
I use qemu extensively especially for early-stage kernel debugging when no console is available; one such was just this week with v6.8 where, on arm64, any kernel command-line parameter >= 146 characters hangs the kernel instantly and silently.
Here's how I used qemu + gdb (on Debian 12 Bookworm amd64 host) to emulate and execute the arm64 kernel build to single-step the problematic code to identify the cause.
1. In a prepared kernel build system (i.e; all build dependencies and cross-compile tools installed) build the kernel image. I do this in an unprivileged systemd-nspawn amd64 container to avoid messy -dev package installs on the host. Nspawn bind-mounts the host's source-code tree which includes a separate build directory:
cd "${SRC_DIR}"
# copy/install/configure a suitable ${BUILD_DIR}/.config; review/edit with:
make V=1 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- O=${BUILD_DIR} -j 4 menuconfig
# build the kernel
export KBUILD_BUILD_USER=linux; export KBUILD_BUILD_HOST=iam.tj; time make V=1 LOCALVERSION="" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- O=${BUILD_DIR} -j 12 Image
# build gdb helper (Python) scripts
export KBUILD_BUILD_USER=linux; export KBUILD_BUILD_HOST=iam.tj; time make V=1 LOCALVERSION="" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- O=${BUILD_DIR} scripts_gdb
This will create the debug symbols needed by gdb in ${BUILD_DIR}/vmlinux and the executable kernel in ${BUILD_DIR}/arch/arm64/boot/Image
2. Install "gdb" (and if doing foreign architecture debugging "gdb-multiarch") on the host as well as "qemu-system-arm"
3. Execute the kernel but -S[uspend] it and have QEMU listen for a connection from gdb:
qemu-system-aarch64 -machine virt,gic-version=3 -cpu max,pauth-impdef=on -smp 2 -m 4096 -nographic -kernel ${BUILD_DIR}/arch/arm64/boot/Image -append "debug $( for l in {144..157}; do echo -n param$l=$(pwgen $((l-9)) 1)' '; done )" -initrd rootfs/boot/initrd.img-6.8.12-arm64-debug -S -gdb tcp::1234
The -append and -initrd shown here are optional; in my case no -initrd is actually needed since the (silent) panic occurs in the first few instructions the kernel executes. If debugging loadable modules however they would be in the initrd and loaded in the usual way. If the problem being diagnosed occurs after the root file-system and userspace proper are active then one would need to add the appropriate qemu options for the emulated storage device where the root file-system lives.
4. In another terminal shell (I use "tmux" and create a new tmux window) start the debugger:
cd ${BUILD_DIR}
# this cd is important - gdb needs to be in the base of the BUILD directory
gdb-multiarch ./vmlinux
The wireguard test suite that’s now in the kernel is an excellent way to experiment with using qemu to develop kernel modules and also do automated tests.
Greg KH said pretty explicitly there won’t be a 4th edition LDD
j33zusjuice [3 hidden]5 mins ago
Did he give any context for why? ROI for him, or?
sthuck [3 hidden]5 mins ago
I'm purely guessing here, but also considering I read him and Linus both say "we have enough kernel developers", I think it's likely they don't want to encourage low quality contributions from new developers.
mardifoufs [3 hidden]5 mins ago
Wouldn't it be helpful then to put out more information on how to be a good contributor? I'm not sure how a technical book about the kernel would lead to worse contributions, you'd think a lack of readily available information and educational material would do that.
saagarjha [3 hidden]5 mins ago
I wonder what a good way to help developers improve the quality of their contributions would be
heavyset_go [3 hidden]5 mins ago
Seems short-sighted. People retire, get new jobs, and move on from projects all the time.
tdiff [3 hidden]5 mins ago
Some examples seem hard to play with, unfortunately. For instance, "Detecting button presses" assumes one is able to build modules for RPi, which probably is not trivial by itself (e.g., requires cross-compilation).
yjftsjthsd-h [3 hidden]5 mins ago
I'll grant that it's a bit of friction, but you can just run a compiler on the pi?
See also The Linux Memory Manager: https://linuxmemory.org/chapters Last update the author sent out was in early July noting that the book is now in editing:
> I am happy to report that I have completed the first draft of the book [...]
> I am now in an editing phase, which may well take some time. Sadly I can't give a reasonable estimate as this will be done in concert with my publisher.
ephaeton [3 hidden]5 mins ago
looks like a great TOC, sadly no preorder to support its creation :(
simonz05 [3 hidden]5 mins ago
I cannot remember (or find) where I signed up for updates, but I get an email every 6 months (or so) from Lorenzo Stoakes personal email. Probably just send him an e-mail and he'll add you to his list.
anta40 [3 hidden]5 mins ago
What about Linux kernel programming in general, e.g hacking the filesystem or memory management parts?
Many years ago there was "Linux Kernel Development" by Robert Love, probably not updated anymore.
donpdonp [3 hidden]5 mins ago
A detailed, hands-on, build a kernel module right away kind of tutorial. Bravo.
Did the authors use an LLM to write or improve the text? I have no problem with that but I feel I'd like to know how much work is LLM based before reading.
BossingAround [3 hidden]5 mins ago
Why would "Before delving into code..." be a red flag that marks the text as LLM-generated?
SPascareli13 [3 hidden]5 mins ago
Someone said that the word "delve" is a favourite of AI and a sign that something was AI written.
cloudwalk9 [3 hidden]5 mins ago
I don't usually suspect AI unless I see in a closing paragraph "However, it is important to note..."
ugh123 [3 hidden]5 mins ago
I wouldn't think it matters as long as the [human] authors review it for accuracy.
LLM likes to use "delve" doesn't mean every usages of "delve" imply LLM
vbezhenar [3 hidden]5 mins ago
Why does it matter? My English is poor, so when I write long articles or posts, I ask GPT to fix errors. I do this because I respect my readers and don't want their eyes to bleed from reading my text.
tczMUFlmoNk [3 hidden]5 mins ago
AI-generated text doesn't just make my eyes bleed; it makes my blood boil. I haven't read much of your English specifically, so I can't say for sure, but generally non-native speakers get a ton of leeway in my book. I do not speak your language anywhere near as well as you speak mine, and your words will not make me feel frustrated even if I occasionally have to pause to figure out the intended meaning.
(Also, IMHO, your comment history is perfectly readable without being distracting.)
stevenhuang [3 hidden]5 mins ago
The proclivity to suggest something is LLM generated when it isn't is such a fun one. Almost like a Rorschach test for literary exposure.
The answer in this context is no (you've might not been exposed to enough fiction).
ashconnor [3 hidden]5 mins ago
Perfectly valid synonym for 'dive' in this context.
HN.zip
Hopefully someone can update the LDD(linux device driver) and Linux kernel books. In fact Linux Foundation should sponsor such efforts since technical book like this is hard to make any profit.
[0]: https://blog.davidv.dev/posts/learning-pcie/
[1]: https://blog.davidv.dev/posts/pcie-driver-dma/
Here's how I used qemu + gdb (on Debian 12 Bookworm amd64 host) to emulate and execute the arm64 kernel build to single-step the problematic code to identify the cause.
1. In a prepared kernel build system (i.e; all build dependencies and cross-compile tools installed) build the kernel image. I do this in an unprivileged systemd-nspawn amd64 container to avoid messy -dev package installs on the host. Nspawn bind-mounts the host's source-code tree which includes a separate build directory:
This will create the debug symbols needed by gdb in ${BUILD_DIR}/vmlinux and the executable kernel in ${BUILD_DIR}/arch/arm64/boot/Image2. Install "gdb" (and if doing foreign architecture debugging "gdb-multiarch") on the host as well as "qemu-system-arm"
3. Execute the kernel but -S[uspend] it and have QEMU listen for a connection from gdb:
The -append and -initrd shown here are optional; in my case no -initrd is actually needed since the (silent) panic occurs in the first few instructions the kernel executes. If debugging loadable modules however they would be in the initrd and loaded in the usual way. If the problem being diagnosed occurs after the root file-system and userspace proper are active then one would need to add the appropriate qemu options for the emulated storage device where the root file-system lives.4. In another terminal shell (I use "tmux" and create a new tmux window) start the debugger:
5. In the gdb shell: At this point the usual gdb functionality is available to examine memory, variables, single-step, view the stack and so on.For more details on debugging kernel using gdb and the gdb scripts lx-* see
https://www.kernel.org/doc/html/latest/dev-tools/gdb-kernel-...
Edit: Forgot to note that for gdb to be able to use the lx-* Python scripts it usually needs the path authorising:
I’d link but cumbersome to find on phone.
there are only 3 files under drivers/net/wireguard/selftest and no qemu there in linux kernel git
https://github.com/torvalds/linux/tree/5437f30d3458ad36e83ab...
https://news.ycombinator.com/item?id=28283030
> I am happy to report that I have completed the first draft of the book [...] > I am now in an editing phase, which may well take some time. Sadly I can't give a reasonable estimate as this will be done in concert with my publisher.
Many years ago there was "Linux Kernel Development" by Robert Love, probably not updated anymore.
Did the authors use an LLM to write or improve the text? I have no problem with that but I feel I'd like to know how much work is LLM based before reading.
(Also, IMHO, your comment history is perfectly readable without being distracting.)
The answer in this context is no (you've might not been exposed to enough fiction).