Re: Relocation overflow on RISC-V with multi-range memory layout

[Wu, Fei]

On 10/15/2023 5:16 AM, Vladimir 'phcoder' Serbinenko wrote:
> I looked into it and found out that current code misses both got and
> trampolines. I have a template for solution but I didn't test it yet.
> Can you upload your EFI images (RISCV_VIRT_CODE.fd and
> RISCV_VIRT_VARS.fd) somewhere so I don't have to compile myself?
> https://github.com/phcoder/GRUB/tree/riscv
> 
Do you have any update on this? Please let me know if you want me to
have a try. Currently this branch doesn't work for me.

Loading driver at 0x000BDCCD000 EntryPoint=0x000BDCCE000 ed yet
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF
BED98A18(0x0000000000004000)
ProtectUefiImageCommon - 0xBED60040attributes not supported yet
  - 0x00000000BDCCD000 - 0x00000000004DD000C-A22A-E5F46812F4CA 83FFF720
SetUefiImageMemoryAttributes - 0x00000000BDCCD000 - 0x0000000000001000
(0x0000000000004000)
CpuSetMemoryAttributes: Set memory attributes not supported yet
SetUefiImageMemoryAttributes - 0x00000000BDCCE000 - 0x000000000000C000
(0x0000000000020000)
CpuSetMemoryAttributes: Set memory attributes not supported yet
SetUefiImageMemoryAttributes - 0x00000000BDCDA000 - 0x00000000004D0000
(0x0000000000004000)
CpuSetMemoryAttributes: Set memory attributes not supported yet
InstallProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA 83FFF720
!!!! RISCV64 Exception Type -
0000000000000005(EXCEPT_RISCV_LOAD_ACCESS_FAULT) !!!!
     t0 = 0x00000000000000047        t1 = 0x00000000083FFF370
     t2 = 0x00000000000000043        t3 = 0x000000000752F3136
     t4 = 0x00000000000004E16        t5 = 0x00000000000004FDC
     t6 = 0x00025100300200001        s0 = 0x00000000083FFF5F0
     s1 = 0x00000000000000013        s2 = 0x00000000000000000
     s3 = 0x00000000020000000        s4 = 0x00000000000000000
     s5 = 0x000000000BDCDB0C0        s6 = 0x08000000A00006800
     s7 = 0x0000000000000007F        s8 = 0x0000000008001A038
     s9 = 0x0000000008002AAB0       s10 = 0x00000000000000000
    s11 = 0x00000000000000000        a0 = 0x000000000BDCDB0C0
     a1 = 0x000000000BFFFE018        a2 = 0x000000000BDCD4F12
     a3 = 0x000000000BED60E98        a4 = 0x000000000BFFFE018
     a5 = 0x0AFAFAFAFAFAFAFAF        a6 = 0x00000000000004FDC
     a7 = 0x000000000000000A2      zero = 0x00000000000000000
     ra = 0x000000000BDCD2FA4        sp = 0x00000000000000010
     gp = 0x00000000000000000        tp = 0x00000000080036000
   sepc = 0x000000000BDCD1D9A   sstatus = 0x08000000200006120
  stval = 0x0AFAFAFAFAFAFAFAF

Thanks,
Fei.

> On Thu, Oct 12, 2023 at 2:30 AM Wu, Fei <fei2.wu@intel.com> wrote:
>>
>> On 10/11/2023 9:50 PM, Vladimir 'phcoder' Serbinenko wrote:
>>>
>>>
>>> Le mer. 11 oct. 2023, 12:20, Wu, Fei <fei2.wu@intel.com
>>> <mailto:fei2.wu@intel.com>> a écrit :
>>>
>>>     On 10/9/2023 11:14 AM, Wu, Fei wrote:
>>>     > On 9/27/2023 11:23 PM, Vladimir 'phcoder' Serbinenko wrote:
>>>     >> That is not the correct solution. Correct solution is to use
>>>     trampoline
>>>     >> facility like e.g. ppc does. Can you post the full reproduction
>>>     >> instructions?
>>>     >>
>>>     Hi Vladimir,
>>>
>>>     Could you please explain a little more about why this is not good, and
>>>     how does the trampoline address that? IIUC, x86_64 sets
>>>     GRUB_EFI_MAX_USABLE_ADDRESS to 0xffffffff for the same reason.
>>>
>>> x86_64 reason is different. It's because some EFI implementations don't
>>> map RAM above 4GiB contrary to the spec.
>>> Trampolines are small piece of code that is inserted by linker to
>>> provide absolute jump. Trampolines are always allocated together with
>>> the module and so they are reachable by relative jump. Basically what
>>> they do then is:
>>> ptr=&symbol;
>>> return (*ptr) (...);
>>> On most platforms that translates to a load+register jump. Let me see if
>>> I can do it quickly. Will you be able to test?
>>>
>> Thank you Vladimir. Sure, I can test it, let me know when your code is
>> ready.
>>
>> Best Regards,
>> Fei.
>>
>>>
>>>     Thanks,
>>>     Fei.
>>>
>>>     > 1. prepare the host for pcie passthrough, e.g. on ubuntu, put
>>>     something
>>>     > like the following to kernel cmd, the pci ids are the pci devices to
>>>     > passthrough:
>>>     >       intel_iommu=on iommu=pt vfio-pci.ids=10de:0f02,10de:0e08
>>>     >
>>>     > 2. apply the patch mentioned below to qemu commit ccb86f079a9e4
>>>     > 3. run qemu as follows:
>>>     >
>>>     > qemu-system-riscv64
>>>     >  -nographic \
>>>     >  -M virt,pflash0=pflash0,pflash1=pflash1,acpi=off \
>>>     >  -m 3G -smp 2 \
>>>     >  -blockdev
>>>     >
>>>     node-name=pflash0,driver=file,read-only=on,filename=RISCV_VIRT_CODE.fd \
>>>     >  -blockdev node-name=pflash1,driver=file,filename=RISCV_VIRT_VARS.fd \
>>>     >  -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
>>>     >  -device virtio-blk-device,drive=hd0 \
>>>     >  -drive file=fat:rw:/home/wufei/src/fat,id=hd0
>>>     >
>>>     > 4. build and put grub.efi to the directory 'fat'
>>>     > 5. on uefi shell, run grub.efi
>>>     >
>>>     > Thanks,
>>>     > Fei.
>>>     >
>>>     >> Le lun. 25 sept. 2023, 10:53, Wu, Fei <fei2.wu@intel.com
>>>     <mailto:fei2.wu@intel.com>
>>>     >> <mailto:fei2.wu@intel.com <mailto:fei2.wu@intel.com>>> a écrit :
>>>     >>
>>>     >>     Hi All,
>>>     >>
>>>     >>     I'm enabling PCIe passthrough on qemu riscv, the physical memory
>>>     >>     range between 3GB and 4GB is reserved. Therefore if guest has
>>>     4GB ram,
>>>     >>     two ranges are created as [2G, 3G) and [4G, 7G). More details
>>>     can be
>>>     >>     found here:
>>>     >>
>>>      
>>> https://lore.kernel.org/all/CAKmqyKMtAzt5saCUMd4vXYfgAQibpzQJAhtTSuSb+yeKhcYpfw@mail.gmail.com/T/
>>>  
>>> <https://lore.kernel.org/all/CAKmqyKMtAzt5saCUMd4vXYfgAQibpzQJAhtTSuSb+yeKhcYpfw@mail.gmail.com/T/>
>>>  
>>> <https://lore.kernel.org/all/CAKmqyKMtAzt5saCUMd4vXYfgAQibpzQJAhtTSuSb+yeKhcYpfw@mail.gmail.com/T/
>>>  
>>> <https://lore.kernel.org/all/CAKmqyKMtAzt5saCUMd4vXYfgAQibpzQJAhtTSuSb+yeKhcYpfw@mail.gmail.com/T/>>
>>>     >>
>>>     >>     When run grub.efi from uefi shell, a relocation problem
>>>     happened in
>>>     >>     grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
>>>     >>
>>>     >>             case R_RISCV_CALL:
>>>     >>             case R_RISCV_CALL_PLT:
>>>     >>               {
>>>     >>                 grub_uint32_t *abs_place = place;
>>>     >>                 grub_ssize_t off = sym_addr - (grub_addr_t) place;
>>>     >>                 grub_uint32_t hi20, lo12;
>>>     >>
>>>     >>                 if (off != (grub_int32_t) off)
>>>     >>                   return grub_error (GRUB_ERR_BAD_MODULE, "relocation
>>>     >>     overflow");
>>>     >>
>>>     >>     It requires `off' in the range of int32, but it's not
>>>     enforced since the
>>>     >>     >4GB memory can be used. I'm not familiar with grub, but this
>>>     patch does
>>>     >>     work for me:
>>>     >>
>>>     >>     --- a/include/grub/riscv64/efi/memory.h
>>>     >>     +++ b/include/grub/riscv64/efi/memory.h
>>>     >>     @@ -1,6 +1,6 @@
>>>     >>      #ifndef GRUB_MEMORY_CPU_HEADER
>>>     >>      #include <grub/efi/memory.h>
>>>     >>
>>>     >>     -#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffffffffULL
>>>     >>     +#define GRUB_EFI_MAX_USABLE_ADDRESS 0xffffffffULL
>>>     >>
>>>     >>     Any comments?
>>>     >>
>>>     >>     Thanks,
>>>     >>     Fei.
>>>     >>
>>>     >>     _______________________________________________
>>>     >>     Grub-devel mailing list
>>>     >>     Grub-devel@gnu.org <mailto:Grub-devel@gnu.org>
>>>     <mailto:Grub-devel@gnu.org <mailto:Grub-devel@gnu.org>>
>>>     >>     https://lists.gnu.org/mailman/listinfo/grub-devel
>>>     <https://lists.gnu.org/mailman/listinfo/grub-devel>
>>>     >>     <https://lists.gnu.org/mailman/listinfo/grub-devel
>>>     <https://lists.gnu.org/mailman/listinfo/grub-devel>>
>>>     >>
>>>     >>
>>>     >> _______________________________________________
>>>     >> Grub-devel mailing list
>>>     >> Grub-devel@gnu.org <mailto:Grub-devel@gnu.org>
>>>     >> https://lists.gnu.org/mailman/listinfo/grub-devel
>>>     <https://lists.gnu.org/mailman/listinfo/grub-devel>
>>>     >
>>>
>>
> 
>