Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/29/14 14:01, Laszlo Ersek wrote:
> On 01/29/14 12:49, Dietmar Maurer wrote:

>> A bisect points to the following patch:
>>
>> # git bisect bad
>> c45e5b5b30ac1f5505725a7b36e68cedfce4f01f is the first bad commit
>> commit c45e5b5b30ac1f5505725a7b36e68cedfce4f01f
>> Author: Gerd Hoffmann <address@hidden>
>> Date:   Tue Feb 26 17:46:11 2013 +0100
>>
>>     Switch to efi-enabled nic roms by default
>>
>>     All PCI nics are switched to EFI-enabled roms by default.  They are
>>     composed from three images (legacy, efi ia32 & efi x86), so classic
>>     pxe booting will continue to work.
>>
>>     Exception: eepro100 is not switched, it uses a single rom for all
>>     emulated eepro100 variants, then goes patch the rom header on the
>>     fly with the correct PCI IDs.  I doubt that will work as-is with
>>     the efi roms.
>>
>>     Keep old roms for 1.4+older machine types via compat properties,
>>     needed because the efi-enabled roms are larger so the pci rom bar
>>     size would change.
>>
>>     Signed-off-by: Gerd Hoffmann <address@hidden>
>>
>>
>>
>>> pxe boot does not work with qemu 1.7 (also tested with latest code
>>> from master).
>>>
>>> # kvm -m 1024 -net nic -net tap
>>>
>>> simply hangs at:
>>>
>>> iPXE (PCI 00:03.0) starting execution.
>>>
>>> and I get the following output:
>>>
>>> # kvm -m 1024 -net nic -net tap
>>> KVM: unknown exit, hardware reason 80000021
>>> EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
>>> ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
>>> EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
>>> ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
>>> SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
>>> DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
>>> TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
>>> GDT=     0009cf40 00000037
>>> IDT=     00000000 0000ffff
>>> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
>>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>>> DR3=0000000000000000
>>> DR6=00000000ffff0ff0 DR7=0000000000000400
>>> EFER=0000000000000000
>>> Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 
>>> 00
>>> 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e 
>>> e0
>>>
>>> this is using RHEL6 kernel 358.23.2.el6
>>>
>>> any ideas?
>
> Does it work with TCG? If so, what host CPU do you have?
>
> Also, can you try with a NIC model different from the default e1000?

I can reproduce this.

- Qemu: 0169c511554cb0014a00290b0d3d26c31a49818f
- Host kernel: 2.6.32-431.3.1.el6.x86_64 (RHEL-6.5)
- Host CPU:
  - Intel(R) Xeon(R) CPU W3550
  - /sys/module/kvm_intel/parameters/unrestricted_guest == N

TCG works fine.

With KVM enabled, I tested all NIC models listed by
"-net nic,model=help":
- fails: ne2k_pci rtl8139 e1000 pcnet virtio
- works: i82551 i82557b i82559er

I rebuilt ipxe (albeit a different version from what Gerd checked in,
probably...) and tried to look for the offending insn stream in the
disassembly:

$ file bin/8086100e.mrom.tmp

  bin/8086100e.mrom.tmp: ELF 32-bit LSB executable, Intel 80386, version
  1 (SYSV), statically linked, not stripped

$ objdump -S bin/8086100e.mrom.tmp

>  3c8:   66 0f 01 1e             lidtw  (%esi)
>  * Note that the broadcast address is also a multicast address.
>  */
> static inline int is_multicast_ether_addr ( const void *addr ) {
>         const uint8_t *addr_bytes = addr;
>
>         return ( addr_bytes[0] & 0x01 );
>  3cc:   48                      dec    %eax
>  3cd:   00 0f                   add    %cl,(%edi)
>  *
>  * Check that the Ethernet address (MAC) is not 00:00:00:00:00:00, is
>  * not a multicast address, and is not ff:ff:ff:ff:ff:ff.
>  */
> static inline int is_valid_ether_addr ( const void *addr ) {
>         return ( ( ! is_multicast_ether_addr ( addr ) ) &&
>  3cf:   20 c0                   and    %al,%al
>  3d1:   0c 01                   or     $0x1,%al
>                 memcpy ( hw_addr, mac.raw, ETH_ALEN );
>                 return 0;
>         }
>
>         DBGC ( intel, "INTEL %p has no MAC address to use\n", intel );
>         return -ENOENT;
>  3d3:   0f 22 c0                mov    %eax,%cr0
>  3d6:   66 ea a4 00 00 00       ljmpw  $0x0,$0xa4

Here: ^^^^

>         return 0;
>
>         unregister_netdev ( netdev );
>  err_register_netdev:
>  err_fetch_mac:
>         intel_reset ( intel );
>  3dc:   08 00                   or     %al,(%eax)
>  3de:   0f 20 c0                mov    %cr0,%eax
>  *
>  * Drivers should call this method immediately before the final call
>  * to netdev_put().
>  */
> static inline void netdev_nullify ( struct net_device *netdev ) {
>         netdev->op = &null_netdev_operations;
>  3e1:   24 fe                   and    $0xfe,%al
>  3e3:   0f 22 c0                mov    %eax,%cr0
>  *
>  * @v netdev            Network device
>  */
> static inline __attribute__ (( always_inline )) void
> netdev_put ( struct net_device *netdev ) {
>         ref_put ( &netdev->refcnt );
>  3e6:   ff 2e                   ljmp   *(%esi)
>  3e8:   4e                      dec    %esi

KVM chokes on the LJMPW instruction. (It needs to emulate it on this
host CPU, but the emulation code fails to decode the instruction.)

The code seems to belong to intel_probe() in "src/drivers/net/intel.c":

intel_probe()
  intel_fetch_mac()
    is_valid_ether_addr()
      is_multicast_ether_addr()

I *guess* upstream Linux commit

    commit 414e6277fd148f6470261cef50a7fed0d88a2825
    Author: Gleb Natapov <address@hidden>
    Date:   Wed Apr 28 19:15:26 2010 +0300

        KVM: x86 emulator: handle "far address" source operand

        ljmp/lcall instruction operand contains address and segment.
        It can be 10 bytes long. Currently we decode it as two different
        operands. Fix it by introducing new kind of operand that can hold
        entire far address.

        Signed-off-by: Gleb Natapov <address@hidden>
        Signed-off-by: Avi Kivity <address@hidden>

which had been first released in v2.6.36, should be ported to the RHEL-6
kernel.

... The pxe-e1000.rom binary seems to work. However, the last update to
*that* file is much older than the one to the efi-1000.rom binary.
Compare:

    commit 0099cd43ecf07710a608db5ca0945758514a14c2
    Author: Gerd Hoffmann <address@hidden>
    Date:   Mon Mar 25 09:13:18 2013 +0100

        ipxe: update binaries

        Signed-off-by: Gerd Hoffmann <address@hidden>

     pc-bios/efi-e1000.rom    | Bin 174080 -> 173568 bytes
     pc-bios/efi-eepro100.rom | Bin 175104 -> 174592 bytes
     pc-bios/efi-ne2k_pci.rom | Bin 173568 -> 173056 bytes
     pc-bios/efi-pcnet.rom    | Bin 173568 -> 173056 bytes
     pc-bios/efi-rtl8139.rom  | Bin 177152 -> 176640 bytes
     pc-bios/efi-virtio.rom   | Bin 171008 -> 171008 bytes
     6 files changed, 0 insertions(+), 0 deletions(-)

vs.

    commit 36d8d02dc8c45780cae74e2ba7a6135b95c16f81
    Author: Alex Williamson <address@hidden>
    Date:   Mon Apr 18 11:46:41 2011 -0600

        PXE: Refresh all PXE ROMs from the ipxe submodule

        Add script to make this easy to repeat later.

        Signed-off-by: Alex Williamson <address@hidden>

     pc-bios/README              |  17 ++++----
     pc-bios/pxe-e1000.rom       | Bin 72192 -> 67072 bytes
     pc-bios/pxe-eepro100.rom    | Bin 56832 -> 61440 bytes
     pc-bios/pxe-ne2k_pci.rom    | Bin 56320 -> 61440 bytes
     pc-bios/pxe-pcnet.rom       | Bin 56832 -> 61440 bytes
     pc-bios/pxe-rtl8139.rom     | Bin 56320 -> 61440 bytes
     pc-bios/pxe-virtio.rom      | Bin 56320 -> 60416 bytes
     scripts/refresh-pxe-roms.sh |  99 
++++++++++++++++++++++++++++++++++++++++++++
     8 files changed, 107 insertions(+), 9 deletions(-)

SeaBIOS has seen such problems before, I believe.

Laszlo