Minimal Linux Bootloader debugging story 🐞

I keep two builds of the Linux kernel, a linux/arm64 construct for gokrazy,
my Go appliance platform, which began out on the
Raspberry Pi, after which a linux/amd64 one for router7,
which runs on PCs.

The replace course of for each of those builds is totally automated, that means new
Linux kernel releases are mechanically examined and merged, however lately the
steady integration testing failed to automatically merge Linux
6․7 — this text is about monitoring
down the foundation reason for that failure.

Background information on the bootloader

gokrazy began out focusing on solely the Raspberry Pi, the place you configure the
bootloader with a plain textual content file on a FAT partition, so we didn’t must
embody our personal UEFI/MBR bootloader.

After I ported gokrazy to work on PCs in BIOS mode, I made a decision towards sophisticated
options like GRUB — I actually wasn’t seeking to keep a GRUB bundle. Simply
preserving GRUB installations engaged on my machines is sufficient work. The truth that
GRUB consists of many alternative recordsdata (modules) that may exit of sync actually
doesn’t attraction to me.

As an alternative, I went with Sebastian Plotz’s Minimal Linux
Bootloader as a result of it matches
totally into the Master Boot Record
(MBR) and doesn’t require
any recordsdata. In bootloader lingo, it is a stage1-only bootloader. You don’t even
want a C compiler to compile its (Meeting) code. It appeared easy sufficient to
combine: simply write the bootloader code into the primary sector of the gokrazy
disk picture; performed. The bootloader had its final launch in 2012, so no want for
updates or upkeep.

You’ll be able to’t actually implement booting a kernel and parsing textual content configuration
recordsdata in 446
bytes of 16-bit
8086 meeting directions, so to inform the bootloader the place on disk to load the
kernel code and kernel command line from, gokrazy writes the disk offset
(LBA) of vmlinuz and
cmdline.txt to the final bytes of the bootloader code. As a result of gokrazy
generates the FAT partition, we all know there’s by no means any fragmentation, so the
bootloader doesn’t want to know the FAT file system.

Symptom

The symptom was that the rtr7/kernel pull request
#434 for updating to Linux 6.7 failed.

My steady integration exams run in two environments: a bodily embedded PC
from PC Engines (apu2c4) in my lounge, and a
digital QEMU PC. Solely the QEMU take a look at failed.

On the bodily PC Engines apu2c4, the pull request really handed the boot
take a look at. It might be improper to attract conclusions like “the difficulty solely impacts QEMU”
from this, although, as later makes an attempt to energy on the apu2c4 confirmed the gadget
boot-looping. I made a psychological notice that one thing is completely different about how the
downside impacts the 2 environments, however each are affected, and determined to
handle the failure in QEMU first, then take into consideration the PC Engines failure some
extra.

In QEMU, the output I see is:

SeaBIOS (model Arch Linux 1.16.3-1-1)

iPXE (http://ipxe.org) 00:03.0 C900 PCI2.10 PnP PMM+06FD3360+06F33360 C900

Booting from Onerous Disk...

Notably, the kernel doesn’t even appear to begin — no “Decompressing linux”
message is printed, the boot simply hangs. I attempted enabling debug output in
SeaBIOS and eventually succeeded, but only with an older QEMU
version:

Booting from Onerous Disk...
Booting from 0000:7c00
In resume (standing=0)
In 32bit resume
Making an attempt a tough reboot

This doesn’t inform me something sadly.

Okay, so one thing about introducing Linux 6.7 into my setup breaks MBR boot.

I figured utilizing Git Bisection
ought to determine the problematic change inside just a few iterations, so I cloned the
at present working Linux 6.6 supply code, utilized the router7 config and compiled
it.

To my shock, even my self-built Linux 6.6 kernel wouldn’t boot! 😲

Why does the router7 construct work when constructed contained in the Docker container, however not
when constructed on my Linux set up? I made a decision to rebase the Docker container
from Debian 10 (buster, from 2019) to Debian 12 (bookworm, from 2023) and that
resulted in a non-booting kernel, too!

We’ve got two triggers: constructing Linux 6.7 or constructing older Linux, however in newer
environments.

% mkdir -p go/src/github.com/rtr7/
% cd go/src/github.com/rtr7/
% git clone --depth=1 https://github.com/rtr7/kernel
% cd kernel
% sed -i 's,CONFIG_KERNEL_ZSTD,#CONFIG_KERNEL_ZSTD,g' cmd/rtr7-build-kernel/config.addendum.txt
% go run ./cmd/rtr7-rebuild-kernel
# takes a couple of minutes to compile Linux
% ls -l vmlinuz
-rw-r--r-- 1 michael michael 15885312 2024-01-28 16:18 vmlinuz

% gok -i mbr new
% gok -i mbr add .
% gok -i mbr edit
# Alter to comprise:
    "KernelPackage": "github.com/rtr7/kernel",
    "FirmwarePackage": "github.com/rtr7/kernel",
    "EEPROMPackage": "",

% GOARCH=amd64 gok -i mbr overwrite 
  --full /tmp/gokr-boot.img 
  --target_storage_bytes=1258299392
% qemu-system-i386 
  -nographic 
  -drive file=/tmp/gokr-boot.img,format=uncooked

Up/Downgrade Variations

Not like utility packages, the Linux kernel doesn’t rely on shared libraries
at runtime, so the dependency footprint is somewhat smaller than standard. Essentially the most
important dependencies are the elements of the construct surroundings, just like the C
compiler or the linker.

So let’s take a look at the software program variations of the known-working (Debian 10)
surroundings and the smallest change we are able to make to that (upgrading to Debian
11):

• Debian 10 (buster) accommodates gcc-8 (8.3.0-6) and binutils 2.31.1-16.
• Debian 11 (bullseye) accommodates gcc-10 (10.2.1-6) and binutils 2.35.2-2.

To determine if the issue is triggered by GCC, binutils, or one thing else
totally, I checked:

Debian 10 (buster) with its gcc-8, however with binutils 2.35 from bullseye
nonetheless works. (Checked by updating /and so forth/apt/sources.checklist, then upgrading solely
the binutils bundle.)

Debian 10 (buster), however with gcc-10 and binutils 2.35 ends in a
non-booting kernel.

So it looks like upgrading from GCC 8 to GCC 10 triggers the difficulty.

As an alternative of working with a Docker container and Debian’s packages, you could possibly additionally
use Nix. The directions
aren’t simple, however I used
nix-shell
to shortly check out GCC 8 (works), GCC 9 (works) and GCC 10 (kernel doesn’t boot)
on my machine.

New Speculation

To recap, we’ve got two triggers: constructing Linux 6.7 or constructing older Linux, however
with GCC 10.

Two theories appeared most believable to me at this level: Both a change in GCC
10 (presumably enabled by one other change in Linux 6.7) is the issue, or the dimensions
of the kernel is the issue.

To confirm the file dimension speculation, I padded a known-working vmlinuz file to
the dimensions of a known-broken vmlinuz:

% ls -l vmlinuz
% dd if=/dev/zero bs=108352 rely=1 >> vmlinuz

However, though it had the identical file dimension because the known-broken kernel, the
padded kernel booted!

So I dominated out kernel dimension as an issue and began researching important
adjustments in GCC 10.

I learn that one of many primary adjustments in GCC 10 is to allow stack safety by default.

Certainly, constructing the kernel with Debian 11 (bullseye), however with
CONFIG_STACKPROTECTOR=n makes it boot. So, I suspected that our bootloader
doesn’t arrange the stack appropriately, or comparable.

I despatched an e mail to Sebastian Plotz, the creator of the Minimal Linux Bootloader,
to ask if he knew about any points along with his bootloader, or if stack safety
looks like a probable situation along with his bootloader to him.

To my shock (it has been over 10 years since he printed the bootloader!) he
really replied: He hadn’t obtained any downside reviews relating to his
bootloader, however didn’t actually perceive how stack safety can be associated.

Debugging with QEMU

At this level, we’ve got remoted no less than one set off for the issue, and
exhausted the straightforward methods of upgrading/downgrading surrounding software program
variations and asking upstream.

It’s time for a Tooling Stage Up! With out a debugger you possibly can solely poke into
the darkish, which takes time and doesn’t lead to thorough
explanations. Significantly on this case, I feel it is extremely probably that any
supply modifications may have launched delicate points. So let’s attain for a
debugger!

Fortunately, QEMU comes with built-in assist for the GDB debugger. Simply add the -s -S flags to your QEMU command to make QEMU cease execution (-s) and arrange a
GDB stub (-S) listening on localhost:1234.

For those who wished to debug the Linux kernel, you could possibly join GDB to QEMU proper
away, however for debugging a boot loader we want an additional step, as a result of the boot
loader runs in Real Mode, however QEMU’s
GDB integration rightfully defaults to the extra fashionable Protected Mode.

When GDB is just not configured appropriately, it decodes addresses and registers with
the improper dimension, which throws off the whole disassembly — examine GDB’s
output with our meeting supply:

(gdb) b *0x7c00
(gdb) c
(gdb) x/20i $computer                         ; [expected (bootloader.asm)]
=> 0x7c00: cli                          ; => 0x7c00: cli
   0x7c01: xor    %eax,%eax             ;    0x7c01: xor %ax,%ax
   0x7c03: mov    %eax,%ds              ;    0x7c03: mov %ax,%ds
   0x7c05: mov    %eax,%ss              ;    0x7c05: mov %ax,%ss
   0x7c07: mov    $0xb87c00,%esp        ;    0x7c07: mov $0x7c00,%sp
   0x7c0c: adc    %cl,-0x47990440(%esi) ;    0x7c0a: mov $0x1000,%ax
   0x7c12: add    %eax,(%eax)           ;    0x7c0d: mov %ax,%es
   0x7c14: add    %al,(%eax)            ;    0x7c0f: sti
   0x7c16: xor    %ebx,%ebx

So we have to guarantee we use qemu-system-i386 (qemu-system-x86_64 prints
Distant 'g' packet reply is just too lengthy) and configure the GDB target
architecture to 16-bit
8086:

(gdb) set structure i8086
(gdb) goal distant localhost:1234

Sadly, the above doesn’t really work in QEMU 2.9 and newer:
https://gitlab.com/qemu-project/qemu/-/issues/141.

On the internet, persons are working round this bug by using a modified target.xml
file. I
tried this, however should have made a mistake — I believed modifying goal.xml
didn’t assist, however once I wrote this text, I discovered that it does really appear
to work. Perhaps I didn’t use qemu-system-i386 however the x86_64 variant or
one thing like that.

Utilizing an older QEMU

It’s sometimes an train in frustration to become old software program to compile in newer environments.

It’s a lot simpler to make use of an older surroundings to run previous software program.

By querying packages.debian.org, we are able to see the QEMU versions included in
current and previous Debian
versions.

Sadly, the oldest listed model (QEMU 3.1 in Debian 10 (buster)) isn’t
sufficiently old. By querying snapshot.debian.org, we are able to see that Debian 9
(stretch) contained QEMU
2.8.

So let’s run Debian 9 — the best manner I do know is to make use of Docker:

% docker run --net=host -v /tmp:/tmp -ti debian:stretch

Sadly, the debian:stretch Docker container doesn’t work out of the
field anymore, as a result of its /and so forth/apt/sources.checklist factors to the deb.debian.org
CDN, which solely serves present variations and now not serves stretch.

So we have to replace the sources.checklist file to level to
archive.debian.org. To appropriately set up QEMU you want each entries, the
debian line and the debian-security line, as a result of the Docker container has
packages from debian-security put in and will get confused when these are
lacking from the bundle checklist:

root@650a2157f663:/# cat > /and so forth/apt/sources.checklist <<'EOT'
deb http://archive.debian.org/debian/ stretch contrib primary non-free
deb http://archive.debian.org/debian-security/ stretch/updates primary
EOT
root@650a2157f663:/# apt replace

Now we are able to simply set up QEMU as standard and begin it to debug our boot course of:

root@650a2157f663:/# apt set up qemu-system-x86
root@650a2157f663:/# qemu-system-i386 
  -nographic 
  -drive file=/tmp/gokr-boot.img,format=uncooked 
  -s -S

Now let’s begin GDB and set a breakpoint on handle 0x7c00, which is the
address to which the BIOS loades the MBR
code and begins execution:

% gdb
(gdb) set structure i8086
The goal structure is about to "i8086".
(gdb) goal distant localhost:1234
Distant debugging utilizing localhost:1234
0x0000fff0 in ?? ()
(gdb) break *0x7c00
Breakpoint 1 at 0x7c00
(gdb) proceed
Persevering with.

Breakpoint 1, 0x00007c00 in ?? ()
(gdb)

Debug symbols

Okay, so we’ve got GDB hooked up to QEMU and may step by meeting
directions. Let’s begin debugging!?

Not so quick. There may be one other Tooling Stage Up we want first: debug
symbols. Sure, even for a Minimal Linux Bootloader, which doesn’t use any
libraries or native variables. Having correct names for features, in addition to line
numbers, will probably be massively useful in only a second.

Earlier than debug symbols, I’d instantly construct the bootloader utilizing nasm bootloader.asm, however to finish up with an emblem file for GDB, we have to instruct
nasm to generate an ELF file with debug symbols, then use ld to hyperlink it and
lastly use objcopy to repeat the code out of the ELF file once more.

After commit
d29c615
in gokrazy/inside/mbr, I’ve bootloader.elf.

Again in GDB, we are able to load the symbols utilizing the symbol-file command:

(gdb) set structure i8086
The goal structure is about to "i8086".
(gdb) goal distant localhost:1234
Distant debugging utilizing localhost:1234
0x0000fff0 in ?? ()
(gdb) symbol-file bootloader.elf
Studying symbols from bootloader.elf...
(gdb) break *0x7c00
Breakpoint 1 at 0x7c00: file bootloader.asm, line 48.
(gdb) proceed
Persevering with.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)

Automation with .gdbinit

At this level, we want 4 instructions every time we begin GDB. We will automate these
by writing them to a .gdbinit file:

% cat > .gdbinit <<'EOT'
set structure i8086
goal distant localhost:1234
symbol-file bootloader.elf
break *0x7c00
EOT

% gdb
The goal structure is about to "i8086".
0x0000fff0 in ?? ()
Breakpoint 1 at 0x7c00: file bootloader.asm, line 48.
(gdb) 

Understanding program movement

The best strategy to perceive program movement appears to be to step by this system.

However Minimal Linux Bootloader (MLB) accommodates loops that run by 1000’s of
iterations. You’ll be able to’t use gdb’s stepi command with that.

As a result of MLB solely accommodates just a few features, I ultimately realized that inserting a
breakpoint on every perform can be the quickest strategy to perceive the
high-level program movement:

(gdb) b read_kernel_setup
Breakpoint 2 at 0x7c38: file bootloader.asm, line 75.
(gdb) b check_version
Breakpoint 3 at 0x7c56: file bootloader.asm, line 88.
(gdb) b read_protected_mode_kernel
Breakpoint 4 at 0x7c8f: file bootloader.asm, line 105.
(gdb) b read_protected_mode_kernel_2
Breakpoint 5 at 0x7cd6: file bootloader.asm, line 126.
(gdb) b run_kernel
Breakpoint 6 at 0x7cff: file bootloader.asm, line 142.
(gdb) b error
Breakpoint 7 at 0x7d51: file bootloader.asm, line 190.
(gdb) b reboot
Breakpoint 8 at 0x7d62: file bootloader.asm, line 204.

With the working kernel, we get the next transcript:

(gdb)
Persevering with.

Breakpoint 2, read_kernel_setup () at bootloader.asm:75
75		xor	eax, eax
(gdb)
Persevering with.

Breakpoint 3, check_version () at bootloader.asm:88
88		cmp	phrase [es:0x206], 0x204		; we want protocol model >= 2.04
(gdb)
Persevering with.

Breakpoint 4, read_protected_mode_kernel () at bootloader.asm:105
105		mov	edx, [es:0x1f4]			; edx shops the variety of bytes to load
(gdb)
Persevering with.

Breakpoint 5, read_protected_mode_kernel_2 () at bootloader.asm:126
126		mov	eax, edx
(gdb)
Persevering with.

Breakpoint 6, run_kernel () at bootloader.asm:142
142		cli
(gdb)

With the non-booting kernel, we get:

(gdb) c
Persevering with.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)
Persevering with.

Breakpoint 2, read_kernel_setup () at bootloader.asm:75
75		xor	eax, eax
(gdb)
Persevering with.

Breakpoint 3, check_version () at bootloader.asm:88
88		cmp	phrase [es:0x206], 0x204		; we want protocol model >= 2.04
(gdb)
Persevering with.

Breakpoint 4, read_protected_mode_kernel () at bootloader.asm:105
105		mov	edx, [es:0x1f4]			; edx shops the variety of bytes to load
(gdb)
Persevering with.

Breakpoint 1, ?? () at bootloader.asm:48
48		cli
(gdb)

Okay! Now we see that the bootloader begins loading the kernel from disk into
RAM, however doesn’t really get far sufficient to name run_kernel, that means the
downside isn’t with stack safety, with loading a working command line or with
something inside the Linux kernel.

This lets us rule out a big a part of the issue area. We now know that we are able to
focus totally on the bootloader and why it can not load the Linux kernel into
reminiscence.

Let’s take a more in-depth look…

Wait, this isn’t GDB!

Within the instance above, utilizing breakpoints was adequate to slender down the issue.

You would possibly suppose we used GDB, and it appeared like this:

However that’s not GDB! It’s a simple mistake to make. In spite of everything, GDB begins up with
only a textual content immediate, and as you possibly can see from the instance above, we are able to simply enter
textual content and obtain an excellent consequence.

To see the true GDB, you’ll want to begin it up totally, that means together with its consumer
interface.

You’ll be able to both use GDB’s textual content consumer interface (TUI), or a graphical consumer
interface for gdb, such because the one out there in Emacs.

The GDB text-mode consumer interface (TUI)

You’re already conversant in the structure, goal and breakpoint
instructions from above. To additionally arrange the text-mode consumer interface, we run just a few
structure instructions:

(gdb) set structure i8086
(gdb) goal distant localhost:1234
(gdb) symbol-file bootloader.elf
(gdb) structure cut up
(gdb) structure src
(gdb) structure regs
(gdb) break *0x7c00
(gdb) proceed

The structure cut up command hundreds the text-mode consumer interface and splits the
display right into a register window, disassembly window and command window.

With structure src we disregard the disassembly window in favor of a supply
itemizing window. Each are in meeting language in our case, however the supply
itemizing accommodates feedback as effectively.

The structure src command additionally removed the register window, which we’ll get
again utilizing structure regs. I’m undecided if there’s a better manner.

The consequence seems like this:

The supply window will spotlight the subsequent line of code that will probably be executed. On
the left, the B+ marker signifies an enabled breakpoint, which can turn out to be
useful with a number of breakpoints. At any time when a register worth adjustments, the
register and its new worth will probably be highlighted.

The up and down arrow keys scroll the supply window.

Use C-x o to change between the home windows.

For those who’re conversant in Emacs, you’ll acknowledge the keyboard shortcut. However as an
Emacs consumer, you would possibly desire the GDB Emacs consumer interface:

The GDB Emacs consumer interface (M-x gdb)

That is M-x gdb with gdb-many-windows
enabled:

Debugging the failing loop

Let’s check out the loop that we all know the bootloader is getting into, however not
leaving (neither read_protected_mode_kernel_2 nor run_kernel are ever known as):

read_protected_mode_kernel:
    mov  edx, [es:0x1f4]              ; edx shops the variety of bytes to load
    shl  edx, 4

.loop:
    cmp  edx, 0
    je   run_kernel

    cmp  edx, 0xfe00                  ; lower than 127*512 bytes remaining?
    jb   read_protected_mode_kernel_2

    mov  eax, 0x7f                    ; load 127 sectors (most)
    xor  bx, bx                       ; no offset
    mov  cx, 0x2000                   ; load short-term to 0x20000
    mov  esi, current_lba
    name read_from_hdd

    mov  cx, 0x7f00                   ; transfer 65024 bytes (127*512 byte)
    name do_move

    sub  edx, 0xfe00                  ; replace the variety of bytes to load
    add  phrase [gdt.dest], 0xfe00
    adc  byte [gdt.dest+2], 0
    jmp  quick read_protected_mode_kernel.loop

The feedback clarify that the code hundreds chunks of FE00h == 65024 (127*512)
bytes at a time.

Loading means calling read_from_hdd, then do_move. Let’s check out do_move:

do_move:
    push edx
    push es
    xor  ax, ax
    mov  es, ax
    mov  ah, 0x87
    mov  si, gdt
    int  0x15     ; line 182
    jc   error
    pop  es
    pop  edx
    ret

int 0x15 is a name to the BIOS Service Interrupt, which can dispatch the decision
primarily based on AH == 87H to the Move Memory Block
(techhelpmanual.com)
perform.

This perform strikes the desired quantity of reminiscence (65024 bytes in our case)
from supply/vacation spot addresses laid out in a World Descriptor Desk (GDT)
file.

We will use GDB to point out the addresses of every of do_move’s reminiscence transfer calls by
telling it to cease at line 182 (the int 0x15 instruction) and print the GDT
file’s vacation spot descriptor:

(gdb) break 182
Breakpoint 2 at 0x7d49: file bootloader.asm, line 176.

(gdb) command 2
Kind instructions for breakpoint(s) 2, one per line.
Finish with a line saying simply "finish".
>x/8bx gdt+24
>finish

(gdb) proceed
Persevering with.

Breakpoint 1, ?? () at bootloader.asm:48
42		cli

(gdb)
Persevering with.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x00	0x10	0x93	0x00	0x00

(gdb)
Persevering with.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0xfe	0x10	0x93	0x00	0x00

(gdb)

The vacation spot handle is saved in byte 2..4. Bear in mind to learn these little
endian entries “again to entrance”.

• Handle #1 is 0x100000.

• Handle #2 is 0x10fe00.

If we press Return lengthy sufficient, we finally find yourself right here:

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x1e	0xff	0x93	0x00	0x00
(gdb)
Persevering with.

Breakpoint 2, do_move () at bootloader.asm:182
182		int	0x15
0x7d85:	0xff	0xff	0x00	0x1c	0x00	0x93	0x00	0x00

(gdb)
Persevering with.

Breakpoint 1, ?? () at bootloader.asm:48
42		cli
(gdb)

Program obtained sign SIGTRAP, Hint/breakpoint lure.
0x000079b0 in ?? ()
(gdb)

Now that execution left the bootloader, let’s check out the final do_move
name parameters: We discover that the vacation spot handle overflowed its 24 byte
information sort:

• Handle #y is 0xff1e00
• Handle #z is 0x001c00

Root trigger

At this level I reached out to Sebastian once more to ask him if there was an
(undocumented) elementary architectural restrict to his Minimal Linux Bootloader —
with 24 bit addresses, you possibly can handle at most 16 MB of reminiscence.

He replied explaining that he didn’t know of this restrict both! He then linked
to Move Memory Block
(techhelpmanual.com)
as proof for the 24 bit restrict.

Hypothesis

So, is it unattainable to load bigger kernels into reminiscence from Actual Mode? I’m not
certain.

The present bootloader code prepares a GDT by which addresses are 24 bits lengthy
at most. However notice that the techhelpmanual.com documentation that Sebastian
referenced is outwardly for the Intel
286 (a 16 bit CPU), and a few of the
GDT bytes are declared reserved.

At this time’s CPUs are Intel 386-compatible (a
32 bit CPU), which appears to make use of one of many previously reserved bytes to signify
bit 24..31 of the handle, that means we’d have the ability to cross 32 bit addresses
to BIOS features in a GDT in any case!

I wasn’t capable of finding clear authoritative documentation on the Transfer Reminiscence Block
API on 386+, or whether or not BIOS features generally are simply anticipated to work with 32 bit addresses.

However Microsoft’s 1989 HIMEM.SYS source contains a
struct
that paperwork this 32-bit descriptor utilization. A extra fashionable reference is that this
Operating Systems Class from FAU
2023 (web page
71/72).

Therefore I’m pondering that the majority BIOS implementations ought to really assist 32
bit addresses for his or her Transfer Reminiscence Block implementation — supplied you fill the
descriptor accordingly.

If that doesn’t work out, there’s additionally “Unreal
Mode”, which
permits utilizing as much as 4 GB in Actual Mode, however is a change that’s much more
sophisticated. See additionally Julio Merino’s “Beyond the 1 MB barrier in DOS”
post to get
an concept of the quantity of code wanted.

Are you aware if both of those two adjustments is workable? Would you have an interest
in tinkering? Ship me a mail! I’d like to be taught extra.

Bonus: studying BIOS supply

There are literally a few BIOS implementations that we are able to look into to get
a greater understanding of how Transfer Reminiscence Block works.

We will take a look at DOSBox, an open supply
DOS emulator. Its Move Memory Block
implementation
does appear to assist 32 bit addresses:

PhysPt dest	= (mem_readd(information+0x1A) & 0x00FFFFFF) +
              (mem_readb(information+0x1E)<<24);

One other implementation is SeaBIOS. Opposite
to DOSBox, SeaBIOS isn’t just utilized in emulation: The PC Engines apu makes use of
coreboot with SeaBIOS. QEMU additionally makes use of SeaBIOS.

The SeaBIOS handle_1587 source
code
is somewhat tougher to comply with, as a result of it requires information of Actual Mode
meeting. The best way I learn it, SeaBIOS doesn’t truncate or in any other case modify the
descriptors and simply passes them to the CPU. On 386 or newer, 32 bit addresses
ought to work.

Mitigation

Whereas it’s nice to know the limitation we’re operating into, I wished to
unblock the pull request as shortly as potential, so I wanted a fast mitigation
as a substitute of investigating if my speculation will be developed into
a correct repair.

After I began router7, we didn’t assist loadable kernel modules, so every part
needed to be compiled into the kernel. We now do assist loadable kernel modules,
so I may have moved performance into modules.

As an alternative, I discovered a good simpler fast repair: switching from gzip to zstd
compression. This
saved about 1.8 MB and can purchase us a while to implement a correct repair whereas
unblocking automated new Linux kernel model merges.

Conclusion

I wished to share this debugging story as a result of it exhibits a few fascinating classes:

1. Having the ability to run older variations of varied elements of your software program stack is a
   very worthwhile debugging instrument. It helped us isolate a set off for the bug
   (utilizing an older GCC) and it helped us arrange a debugging surroundings (utilizing
   an older QEMU).

2. Establishing a debugger will be annoying (image recordsdata, studying the UI) however
   it’s so value it.

3. Be looking out for improper turns throughout debugging. Write down each
   conclusion and problem it.

4. The BIOS can appear mysterious and “too low stage” however there are a lot of weblog
   posts, lectures and tutorials. You can too simply learn open-source BIOS code
   to know it a lot better.

Take pleasure in poking at your BIOS!

Appendix: Sources

I discovered the next sources useful: