Re: [Qemu-devel] Can't switch to 64bit mode at boot, how to debug?

[Goswin von Brederlow]

On Sat, Nov 17, 2012 at 10:58:17PM +0300, Max Filippov wrote:
> On Sat, Nov 17, 2012 at 9:50 PM, Goswin von Brederlow <address@hidden> wrote:
> > Hi,
> >
> > I'm having problems with the startup code for my toy kernel for amd64 that
> > switches from 32bit mode to 64bit mode and I'm at a loss how to debug this.
> > The code used to work last year (with qemu-kvm 0.14.0) but fails now with
> > QEMU emulator version 1.1.0 (Debian 1.1.0+dfsg-1).
> >
> > Has something been changed in the boot environment for the multiboot?
> >
> > When I start qemu I get the following:
> >
> > % qemu -s -m 64 -kernel loader -initrd "../../kernel/moose arg=arg"
> > Could not open option rom 'kvmvapic.bin': No such file or directory
> > Trying to execute code outside RAM or ROM at 0x001000c1
> >
> > EAX=80000013 EBX=0010fc90 ECX=c0000080 EDX=00000000
> > ESI=00009500 EDI=00107000 EBP=0010b000 ESP=0010afb4
> > EIP=001000c1 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
> > ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> > CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
> > SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> > DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> > FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> > GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> > LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
> > TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
> > GDT=     00100dc8 00000028
> > IDT=     00000000 000003ff
> > CR0=80000013 CR2=00000000 CR3=00101000 CR4=00000020
> > DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000
> > DR6=ffff0ff0 DR7=00000400
> > CCS=00000000 CCD=00000000 CCO=SARL
> > EFER=0000000000000000
> > FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
> > FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
> > FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
> > FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
> > FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
> > XMM00=00000000000000000000000000000000 
> > XMM01=00000000000000000000000000000000
> > XMM02=00000000000000000000000000000000 
> > XMM03=00000000000000000000000000000000
> > XMM04=00000000000000000000000000000000 
> > XMM05=00000000000000000000000000000000
> > XMM06=00000000000000000000000000000000 
> > XMM07=00000000000000000000000000000000
> > zsh: abort      qemu -s -m 64 -kernel loader -initrd "../../kernel/moose 
> > arg=arg"
> >
> > 0x001000c1 is the next instruction after the "movl %eax, %cr0" instruction
> > that enables protected mode and paging.
> >
> > So how do I debug this? Can I tell qemu to go into monitor mode instead of
> > quiting?
> 
> Try adding -S to your qemu command line and then connecting to it with gdb:
> 
> $ qemu-system-x86_64 -s -S -nographic -serial none -monitor stdio -kernel test
> QEMU 1.2.50 monitor - type 'help' for more information
> (qemu)
> 
> $ gdb
> ...
> (gdb) target remote 127.0.0.1:1234
> Remote debugging using 127.0.0.1:1234
> 0x0000fff0 in ?? ()

I tried that. But qemu still simply quits when it hits the above problem
instead of jumping into gdb.

> (gdb) b *0x100000
> Breakpoint 1 at 0x100000
> (gdb) c
> Continuing.

That works, but only for addresses BEFORE where it crashes. What works, but
in no way helps (see below), it setting the breakpoint before the last
instruction and then "stepi".

Only way that would help would be entering the monitor mode on such an error.
 
> Breakpoint 1, 0x00100000 in ?? ()
> (gdb) display/10i $pc
> 1: x/10i $pc
> => 0x100000:    jmp    0x100010
>    0x100002:    xchg   %ax,%ax
>    0x100004:    add    0x31bad(%eax),%dh
>    0x10000a:    add    %al,(%eax)
>    0x10000c:    sti
>    0x10000d:    dec    %edi
>    0x10000e:    push   %edx
>    0x10000f:    in     $0xbc,%al
>    0x100011:    add    %dh,0x6a0010(%eax)
>    0x100017:    popf
> 
> ...
> 
> > Or can someone spot the problem from the source?
> >
> > MfG
> >         Goswin
> > --
> > ----------------------------------------------------------------------
> > #define ASM     1
> > #include <multiboot.h>
> >
> >         .text
> >
> >         .globl  start, _start
> >         .code32
> > start:
> > _start:
> >         jmp     multiboot_entry
> >
> >         /* Align 32 bits boundary.  */
> >         .align  4
> >
> >         /* Multiboot header.  */
> > multiboot_header:
> >         /* magic */
> >         .long   MULTIBOOT_HEADER_MAGIC
> >         /* flags */
> >         .long   MULTIBOOT_HEADER_FLAGS
> >         /* checksum */
> >         .long   -(MULTIBOOT_HEADER_MAGIC + MULTIBOOT_HEADER_FLAGS)
> > #ifndef __ELF__
> >         /* header_addr */
> >         .long   multiboot_header
> >         /* load_addr */
> >         .long   _start
> >         /* load_end_addr */
> >         .long   _edata
> >         /* bss_end_addr */
> >         .long   _end
> >         /* entry_addr */
> >         .long   multiboot_entry
> > #endif /* ! __ELF__ */
> >
> > multiboot_entry:
> >         /* Initialize the stack pointer.  */
> >         movl    $(stack + STACK_SIZE), %esp
> >
> >         /* Reset EFLAGS.  */
> >         pushl   $0
> >         popf
> >
> >         /* Push the pointer to the Multiboot information structure.  */
> >         pushl   %ebx
> >         /* Push the magic value.  */
> >         pushl   %eax
> >
> >         /* Enable PAE */
> >         movl    %cr4, %eax
> >         btsl    $5, %eax
> >         movl    %eax, %cr4
> 
> Interestingly with the current HEAD the code disappears after this 
> instruction:
> 
> (qemu) p/x $eip
> 0x100024
> (qemu) x/10i 0x00100021
> 0x0000000000100021:  mov    %eax,%cr4
> 0x0000000000100024:  lea    0x101000,%edi
> 0x000000000010002a:  xor    %eax,%eax
> 0x000000000010002c:  mov    $0x1800,%ecx
> 0x0000000000100031:  rep stos %eax,%es:(%edi)
> 0x0000000000100033:  lea    0x101000,%edi
> 0x0000000000100039:  lea    0x1007(%edi),%eax
> 0x000000000010003f:  mov    %eax,(%edi)
> 0x0000000000100041:  lea    0x102000,%edi
> 0x0000000000100047:  lea    0x1007(%edi),%eax
> 
> <<<single step in the debugger>>>
> 
> (qemu) p/x $eip
> 0x100024
> (qemu) x/10i 0x00100021
> 0x0000000000100021:  add    %dl,0x33(%ebp)
> 0x0000000000100024:  add    %dl,0x33(%ebp)
> 0x0000000000100027:  add    %dl,0x33(%ebp)
> 0x000000000010002a:  add    %dl,0x33(%ebp)
> 0x000000000010002d:  add    %dl,0x33(%ebp)
> 0x0000000000100030:  add    %dl,0x33(%ebp)
> 0x0000000000100033:  add    %dl,0x33(%ebp)
> 0x0000000000100036:  add    %dl,0x33(%ebp)
> 0x0000000000100039:  add    %dl,0x33(%ebp)
> 0x000000000010003c:  add    %dl,0x33(%ebp)
> (qemu) info registers
> EAX=00000020 EBX=00009500 ECX=00100000 EDX=00000511
> ESI=00000000 EDI=0010c000 EBP=00000000 ESP=0010aff8
> EIP=00100024 EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
> SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
> TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
> GDT=     000ca210 00000027
> IDT=     00000000 000003ff
> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000020
> ...
> 
> Looks like PAE support is broken.

Yeah, any memory read just returns all zeroes. That threw me too. I had
to debug this by adding "mov $0x101000,%ebx" type instructions and then
check the contents of ebx on crash. I wagely remember this always being
broken.

My assumption would be that I'm setting up the gdt, paging, the page tables
and such and the debug mode already tries to use them while the cpu is
still in the original mode. So the two don't agree on what is where.

MfG
        Goswin