GSoC report: Physical memory management

[Justus Winter]

Hello :)

this is my report on my GSoC project, "Physical memory management".

Tldr: There are some remaining issues to fix, but here is a flashy
screenshot of a patched kernel not quite booting Debian/Hurd:

GNU Mach 1.5
biosmem: physical memory map:
biosmem: 000000000000000000:00000000000009fc00, available
biosmem: 00000000000009fc00:0000000000000a0000, reserved
biosmem: 0000000000000f0000:000000000000100000, reserved
biosmem: 000000000000100000:00000000007fffe000, available
biosmem: 00000000007fffe000:000000000080000000, reserved
biosmem: 0000000000feffc000:0000000000ff000000, reserved
biosmem: 0000000000fffc0000:000000000100000000, reserved
biosmem: heap: 406000-38000000
vm_page: page table size: 524270 entries (26624k)
vm_page: DMA: pages: 4080 (15M), free: 3066 (11M)
vm_page: DIRECTMAP: pages: 225280 (880M), free: 218367 (852M)
vm_page: HIGHMEM: pages: 294910 (1151M), free: 294910 (1151M)
pcibios_init : BIOS32 Service Directory structure at 0xf6ac0
pcibios_init : BIOS32 Service Directory entry at 0xfd4be
pcibios_init : PCI BIOS revision 2.10 entry at 0xfd456
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE 
    Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
    ide0: BM-DMA at 0xc100-0xc107
    ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
eth0: RealTek RTL-8029 found at 0xc000, IRQ 11, 52:54: 0:12:34:56.
ne2k-pci.c:v1.05 6/13/2002 D. Becker/P. Gortmaker
  http://www.scyld.com/network/ne2k-pci.html
rtl8139.c:v1.23a 8/24/2003 Donald Becker, becker@scyld.com.
 http://www.scyld.com/network/rtl8139.html
via-rhine.c:v1.16 7/22/2003  Written by Donald Becker <becker@scyld.com>
  http://www.scyld.com/network/via-rhine.html
0 3c515 cards found.
eth1: D-Link DE-600 pocket adapter: not at I/O 0x378.
D-Link DE-620 pocket adapter not identified in the printer port
Partition check (DOS partitions):
 hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
module 0: ext2fs.static --multiboot-command-line=${kernel-command-line} 
--host-priv-port=${host-port} --device-master-port=${device-port} 
--exec-server-task=${exec-task} -T typed device:hd0s1 $(task-create) 
$(task-resume)
module 1: ld.so.1 /hurd/exec $(exec-task=task-create)
2 multiboot modules
task loaded: ext2fs.static --multiboot-command-line=gnumach console=com0 
init=/bin/bash --host-priv-port=1 --device-master-port=2 --exec-server-task=3 
-T typed device:hd0s1
task loaded: ld.so.1 /hurd/exec

start ext2fs.static: Hurd server bootstrap: ext2fs.static[device:hd0s1] exec 
startup proc auth.

wired MEMORY_OBJECT_LOCK_RESULT_MUST_BLOCK

At the top you see the "biosmem" module.  It is imported from Richard
Brauns x15 kernel.  It interprets the memory layout as provided by the
multiboot-compliant bootloader.

Then the vm_page module is initialized.  It is also from x15, slightly
adapted and glued into the environment.  This module contains the
buddy allocator which provides efficient allocation of continuous
chunks of memory at the cost of high internal fragmentation.  To avoid
the fragmentation overhead, we do not use it directly, but through the
slab allocator that is already used for most of the allocations inside
the kernel.

The module provides three different segments, DMA, DIRECTMAP, and
HIGHMEM.  The DMA segment provides memory at low addresses for drivers
that need buffers that devices can read from and write into using DMA.

The DIRECTMAP segment is directly mapped into the kernels address
space.  We can use this segment for kernel objects and can directly
access it in inter-task memory copies.  You can see its size capped at
880M because the kernel is using a 1G kernel / 3G userspace split (the
stock Debian kernel uses a 2/2 split to maximize the amount of
mappable physical memory).

We cannot use the HIGHMEM segment for kernel objects, but we can give
it out to userspace tasks.

You can then see the drivers probing for devices, modules being loaded
and the Hurd server bootstrap running.  The system then deadlocks,
with a custom message being emitted right before that.

And this is the current state of affairs.  It is basically done,
modulo finding all the remaining issues.  I know I've been saying that
a lot lately...

From a high-level point of view what I did was:

* Replace the per-task IPC tables with a per-task radix tree, and also
  the reverse lookup (hash) table with a radix tree.

* Inserting nodes into the radix tree sometimes requires memory
  allocations, which may block when memory is tight.  Therefore, I had
  to convert a lot of simple locks with sleep locks, a manual process
  that required fixes to the locking here and there.

* Replace the current memory allocator with a buddy allocator.

Upon completion of the project, we will be a) able to allocate kernel
objects from the direct map as opposed to having to use the scarce
kernel virtual memory, and b) use more physical memory.  And while
there are still some bugs to be found and the patch series still needs
to be cleaned up considerably, we are already enjoying some benefits.
The patches replacing the IPC tables improved the overall system
performance, and I'm happy to report that swapping works well
nowadays.

I have pushed snapshots of my development branch to darnassus from
time to time, and will do so in the future.  I'll keep you posted on
my progress and answer any questions you might have.

Thanks,
Justus

signature.asc
Description: signature