QEMU PPC guest debugging

[BALATON Zoltan]

Hello,

Sending this to the mailng list in case there are somebody who has some 
idea about these. While trying to debug an issue with g3beige emulation we 
came across some problems. One is about gdb:

On Mon, 13 Jul 2020, Max Poliakovski wrote:
> Side question: I noticed that GDB attached to QEMU doesn't support the "ni"
> (next instruction) command. It's very useful for stepping over subroutine
> calls. For me, that command acts just like "si", i.e. it steps *into* calls.
>
> Do you have any idea why it doesn't work?
>
> I'm using GDB 7.9 configured as "--host=x86_64-apple-darwin17.7.0
> --target=powerpc-linux"

Another question is that during executing guest code (a Mac ROM known to 
work on real g3beige and other emulators) I see unexpectedly getting 0 
bytes instead of opcodes previously written there by guest. I thought it 
may have been overwritten by something but can't find the cause. This is 
what I get with -singlestep -d mmu,unimp,guest_errors

[This is where it's written,
after this point memory contains valid opcodes, not 0:]
----------------
IN:
0xfff20550:  4081ffa4  ble      0xfff204f4

htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f806
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f806
found PTE at offset 00000180
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
found PTE at offset 00000140
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f804
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f804
found PTE at offset 00000100
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f803
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f803
found PTE at offset 000000c0
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f802
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f802
found PTE at offset 00000080
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f801
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f801
found PTE at offset 00000040
PTE access granted !
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f800
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f800
found PTE at offset 00000000
PTE access granted !
----------------
IN:
0xfff20554:  7c0027ac  icbi     0, r4

[Then later when it is executed (the 0s should be valid code here):]

----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a81c:  60008010  ori      r0, r0, 0x8010
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a820:  68008010  xori     r0, r0, 0x8010
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a824:  7c000124  mtmsr    r0
----------------
IN:
Unassigned mem read 00000000ff80a828
0xff80a828:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a82c
0xff80a82c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a758
0xff80a758:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a75c
0xff80a75c:  00000000  .byte    0x00, 0x00, 0x00, 0x00

htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
found PTE at offset 00000140
PTE access granted !
----------------
IN:
Unassigned mem read 00000000ff80a760
0xff80a760:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a764
0xff80a764:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a768
0xff80a768:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a76c
0xff80a76c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a770
0xff80a770:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a774
0xff80a774:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a648
0xff80a648:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a64c
0xff80a64c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a650
0xff80a650:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a654
0xff80a654:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a658
0xff80a658:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a65c
0xff80a65c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a660
0xff80a660:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a664
0xff80a664:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a668
0xff80a668:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a66c
0xff80a66c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a670
0xff80a670:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a674
0xff80a674:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a678
0xff80a678:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a67c
0xff80a67c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a778
0xff80a778:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a77c
0xff80a77c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a720
0xff80a720:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a724
0xff80a724:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a728
0xff80a728:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a72c
0xff80a72c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a730
0xff80a730:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a734
0xff80a734:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a738
0xff80a738:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a73c
0xff80a73c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a740
0xff80a740:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a744
0xff80a744:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a780
0xff80a780:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a798
0xff80a798:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a79c
0xff80a79c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a680
0xff80a680:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a684
0xff80a684:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a688
0xff80a688:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a68c
0xff80a68c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6b0
0xff80a6b0:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6b4
0xff80a6b4:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6b8
0xff80a6b8:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6bc
0xff80a6bc:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a690
0xff80a690:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a694
0xff80a694:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a698
0xff80a698:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a69c
0xff80a69c:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6a0
0xff80a6a0:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6a4
0xff80a6a4:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6a8
0xff80a6a8:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a6ac
0xff80a6ac:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
Unassigned mem read 00000000ff80a830
0xff80a830:  00000000  .byte    0x00, 0x00, 0x00, 0x00
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a834:  4c00012c  isync
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a838:  7e6903a6  mtctr    r19
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a83c:  827e0000  lwz      r19, 0(r30)

htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f80f
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f80f
found PTE at offset 000003c0
PTE access granted !
----------------
IN:
htab_base 00000000004e0000 htab_mask 000000000000ffff hash 000000000000f805
0 htab=00000000004e0000/000000000000ffff vsid=f ptem=7be hash=000000000000f805
0xff80a840:  3bde0004  addi     r30, r30, 4

[Then back to normal after this point.]

I've attached gdb and set a watch point to 0xff80a828 but it's only 
written once with valid opcode value. We also know that this virtual 
address corresponds to phys address 0x0040a828 which still contains the 
valid value after it fails so it does not seem to be overwritten. Maybe 
it's an MMU problem? (But it happens mid page not at page boundary.) I 
thought maybe the PTEs were overwritten so I've set watchpoint for 
0x004e0140 and 0x004e0180 but those also fired only once. Also thought 
some device emulation may overwrite it from host so I've tried the same 
attaching gdb to QEMU itself and watching addresses returned by gpa2hva 
monitor command but that did not reveal anything either. Any other idea 
what could be wrong and how to debug this further?

Regards,
BALATON Zoltan