Re: [RFC PATCH] kern/dl: Add module version check

[Pete Batard]

Hi Zhang,

On 2022.12.21 11:38, Zhang Boyang wrote:
> Hi,
>
> On 2022/12/21 17:43, Thomas Schmitt wrote:
> > Hi,
> >
> > Pete Batard wrote:
> > > > unlike what is the case for UEFI, one can not expect
> > > > to be able to pick all the GRUB core files needed to convert a GRUB
> > > > based ISO bootable media to a GRUB based USB bootable media [...]
> > > > Typically, one of the
> > > > missing files will be a 'core.img' that can work for USB BIOS boot of a
> > > > MBR partitioned FAT or NTFS media, and that even a GRUB based ISOHybrid
> > > > image will not readily provide.
> >
> > Zhang Boyang wrote:
> > > The situation is, for BIOS boot, there is no grub core file in ISO
> > > because it's written to sectors instead of files? Did I understand
> > > correctly?
> >
> > I think it is rather about the fact that GRUB equipped hybrid ISOs boot
> > on legacy BIOS from USB stick via the MBR code (~440 byte) to the
> > El Torito boot image, which is in charge for booting from optical media.
> > In grub-mkrescue ISOs it is called
> >    /boot/grub/i386-pc/eltorito.img
> > The MBR code is taken by grub-mkrescue.c from the file "boot_hybrid.img"
> > in source_dirs[GRUB_INSTALL_PLATFORM_I386_PC].
> >
> > So no core.img is needed and the filesystem module set can be sparse,
> > because the filesystem type is known from which GRUB will read its 
> > further
> > software.
>
> Thanks for explaining! So I think the best solution for Rufus is to 
> overwrite distro's .mod files with its owns (if mismatched modules is a 
> problem).

I'm sorry but that is not an option because, whereas when distro 
maintainers apply patches to GRUB to be able to boot their distribution, 
core.img is usually "safe" from major alterations, meaning that we are 
able to use a core.img that wasn't specifically compiled alongside the 
modules, the modules are usually the elements that are modified.

So, we can NOT rely on vanilla GRUB modules working at all for booting a 
distro that has had GRUB patches applied. Unlike what is the case for 
core.img, where we very seldom observe breakage, and as you actually 
observed below, this will break boot in a lot of cases.

Plus, since we cannot embed GRUB modules in a 1 MB application that we 
purposefully designed to have a small download footprint, you would now 
be forcing users to download ~10 MB of data, which introduces other 
issues (it's already slightly problematic to have users download a 
core.img since we only embed the one from the latest GRUB release in Rufus).

I do understand that what I am reporting is problematic for the solution 
you are proposing, and that you would like to be able to brush it away 
with a simple "Just do it this other way then", but I can assure you, it 
is not that simple, and, to avoid users running into issues, core.img 
replacement is the much better solution.

> > Pete Batard wrote:
> > > > Rufus usually recommends to write such images in what it calls
> > > > "ISO mode" (shorthand for "ISO extraction mode") with the ISO content
> > > > extracted to a FAT or NTFS file system and with a GRUB core.img
> > > > bootloader
> >
> > Maybe one should not only put a copy of the EFI boot image's /EFI/BOOT
> > tree into grub-mkrescue ISOs but also an outdoor pack with the GRUB
> > equipment that is needed to make bootable a USB stick with FAT or NTFS.
> > (The filesystem modules are included in Pete Batard's proposal for the
> >   /EFI/BOOT tree. But the disk MBR image and core.img are not.)

Yeah, that was really the only other *remotely viable* alternative I 
considered at the time.

This however requires a few factors:
- ISOHybrid to be used (not guaranteed. One great plus of Rufus is that, 
in most case, it can make bootable media of non ISOHybrids, including 
GRUB based ones)
- grub-mkrescue to be used to generate the ISOHybrid (The maintainers 
may choose to use a different method)
- Distro maintainers to be cooperative with the idea of supporting an 
alternate mode of creation of bootable USB media compared to DD (which, 
sadly, is not a given).

> > Zhang Boyang wrote:
> > > test whether that commit (052e6068be62) breaks Rufus
> > > (e.g. Rufus with latest git GRUB, to boot Debian ISO).
> >
> > Debian ISOs still boot on legacy BIOS via ISOLINUX.
> > Archlinux, Ubuntu, Fedora, and others meanwhile use GRUB in their ISOs 
> > for
> > both, legacy BIOS and EFI.
>
> Oh, yes, Debian seems using isolinux instead of grub.... So please use 
> Ubuntu ISOs for testing, which uses grub in BIOS mode. By the way, I 
> also tried Arch Linux, it also uses isolinux (but it has a theme looks 
> like grub).
>
> I did reversed tests (i.e. old GRUB core + new GRUB modules) just now:
>
> 1) I used Rufus 3.21 and ubuntu-22.04.1-desktop-amd64.iso (which uses 
> GRUB 2.06) to create a USB stick. To simulate mismatched modules, I 
> overwrited "\boot\grub\i386-pc\relocator.mod" (on USB stick) with my 
> latest git build. Then that USB stick failed to boot in BIOS mode, 
> showing "452: out of range pointer: 0x100010" error.
>
> 2) I tested supergrub2-2.06s1-beta2-multiarch-CD.iso with latest Arch 
> Linux (BIOS mode, grub 2:2.06.r403.g7259d55ff-1). It doesn't have 
> problems booting Arch Linux because it has $prefix properly set to its 
> own GRUB path in CDROM, so it is using its own modules. If I manually 
> run "set prefix=(hd0,msdos1)/boot/grub", it will failed to boot, showing 
> "error: symbol `grub_debug_malloc' not found."
>
> By the way, I googled "452: out of range pointer" and there seems a lot 
> of them. Probably the root cause of many of them is mismatched modules, 
> so I think giving a clear message to user should be helpful (if similar 
> things happens in future).

Zhang, you said:

> The main purpose is to implement truly loadable modules in secure 
boot environment

and I said:

> I have no issue with the module version check proposal if it is to be 
restricted to UEFI boot only

So I think we can have a compromise that works for the both of us if you 
do restrict your proposal for UEFI only (which, apparently and 
realistically is all we should care about, as trying to secure BIOS boot 
is a fool's errand anyway).

Is there any reason why you couldn't submit a proposal that only adds 
module version check for UEFI boot?

Regards,

/Pete

> Best Regards,
> Zhang Boyang
>
> > I think that in any case an ISO made by grub-mkrescue should be tested.
> > Maybe a distro developer is here who uses it for making a full fledged
> > installation ISO or a live ISO.
> >
> >
> > Have a nice day :)
> >
> > Thomas