Re: [Bug-xorriso] ISO installer image: GPT versus MBR partitions

[Thomas Schmitt]

Hi,

after the adventure of diagnosing Macbook EFI and mformat, i want to
return to the original topic of this thread:
I deem it worthwhile to offer the opportunity to create the Guix ISO
with MBR partition table rather than with GPT.

If you just want to see my technical proposal, then hop to the next
section titled "How to do it".

The advantages of my proposal are:

- No GPT backup which needs to be relocated after the ISO was put
  onto USB stick.

- Mountable ISO 9660 partition. (The GPT of a pure grub-mkrescue ISO
  offers a HFS+ partition for mounting. But afaik, Linux hates the
  block size 2048.)

A disadvantage is that for best overall properties, this layout should
have the ISO partition start at 512-byte block 64. This needs a second
superblock and directory tree. About 17 MiB with Guix 0.16.0.

If the size increase matters, then i propose to sacrifice the HFS+
filesystem tree as compensation.
It is for Macs even older than Florian's.

Without that non-zero partition start, the ISO superblock (PVD) of the
overall image cannot claim the appended EFI partition 2 as part of its
filesystem. That's because the superblock is mountable as partition 1
and a partition filesystem should not be larger than its partition.

But with partition start at LBA 64, the overall superblock is not
mountable via a partition and thus free to claim as much space as it
wants.
So programs like /sbin/isosize can tell the image size even when the
ISO has already been copied to USB stick.

Partition editors, too, will love a partition start after the partition
table rather than at block 0.

----------------------------------------------------------------------

How to do it:

We can achieve the MBR partition table based layout by a script which
is part of libisoburn. It acts as pseudo-xorriso between grub-mkrescue
and real xorriso.
Yesterday i improved it by a partition table killer for the EFI image:
  MKRESCUE_SED_IN_EFI_NO_PT=yes.

For experiments it is best to download it from upstream.
Currently the https certificate is not valid. So for now unsafe:

  wget --no-check-certificate \
       
https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/frontend/grub-mkrescue-sed.sh

Size of today's version is 11077 bytes.
SHA512 is
  
23cbc25aac8e03de82bd6e76b00a6ad249629c19822728616d6144f075dbb78f69811648559846ba33319c38baa21515e4a3a0f7bb3ca86f2427e887592101cf
(Ask if you get a different checksum. Maybe i'll make more changes.)

------------------------------------------------------------------------

In a shell script it would be used like this:

  export MKRESCUE_SED_MODE=mbr_hfs
  export MKRESCUE_SED_IN_EFI_NO_PT=yes

  grub-mkrescue --xorriso=./grub-mkrescue-sed.sh \
                ... the usual arguments for grub-mkrescue ... \
                ... with or without a -- switch to native command mode ... \
                -- \
                -boot_image any partition_offset=16 \
                -boot_image any iso_mbr_part_type=0x83

The result (with nearly no payload) looks like

  $ /sbin/fdisk -l output.iso
  ...
  Disklabel type: dos
  ...
  Device      Boot Start   End Sectors  Size Id Type
  output.iso1 *       64 28695   28632   14M 83 Linux
  output.iso2      28696 34455    5760  2.8M ef EFI (FAT-12/16/32)

  $ expr $(/sbin/isosize output.iso) / 512
  34456

The HFS+ superblock and directory tree will be marked by a partition
in an Apple Partition Map. From xorriso -report_system_area :

  APM                :   N  Info
  APM block size     :      2048
  APM gap fillers    :      1
  APM partition name :   1  Gap0
  APM partition type :   1  ISO9660_data
  APM start and size :   1  16  150
  APM partition name :   2  HFSPLUS_Hybrid
  APM partition type :   2  Apple_HFS
  APM start and size :   2  166  7008

------------------------------------------------------------------------

If HFS+ shall be omitted for size reduction or other reasons, use
a different mode of grub-mkrescue-sed.sh :

  export MKRESCUE_SED_MODE="mbr_only"

rather than MKRESCUE_SED_MODE="mbr_hfs".
The result will be smaller by the size of the additional HFS+ tree.

  $ expr $(/sbin/isosize output.iso) / 512
  33864

(The true cost of HFS+ and second ISO directory tree will show up only
 with much more files in the trees.)

------------------------------------------------------------------------

The original GPT layout with additionally zeroed inner EFI partition
table can be achieved by:

  export MKRESCUE_SED_MODE=original
  export MKRESCUE_SED_IN_EFI_NO_PT=yes

  grub-mkrescue --xorriso=./grub-mkrescue-sed.sh \
                ... the usual arguments for grub-mkrescue ...

(Partition offset and MBR partition type are of no use with "original".)

The result is

  $ /sbin/fdisk -l output.iso
  ...
  Disklabel type: gpt
  ...
  Device      Start   End Sectors  Size Type
  output.iso1    64   339     276  138K Microsoft basic data
  output.iso2   340  6099    5760  2.8M EFI System
  output.iso3  6100 34131   28032 13.7M Apple HFS/HFS+
  output.iso4 34132 34731     600  300K Microsoft basic data

  $ expr $(/sbin/isosize output.iso) / 512
  34780

------------------------------------------------------------------------

Another interesting layout is "gpt_appended". It moves the EFI partition
out of the ISO filesystem, as does "mbr_only". But it marks the partitions
in GPT. The ISO partition is mountable. Two directory trees are created
automatically.

  export MKRESCUE_SED_MODE=gpt_appended
  export MKRESCUE_SED_IN_EFI_NO_PT=yes

  grub-mkrescue --xorriso=./grub-mkrescue-sed.sh \
                ... the usual arguments for grub-mkrescue ...

Result:

  $ /sbin/fdisk -l output.iso
  ...
  Disklabel type: gpt
  ...
  Device      Start   End Sectors  Size Type
  output.iso1    64 28103   28040 13.7M Microsoft basic data
  output.iso2 28104 33863    5760  2.8M EFI System

  $ expr $(/sbin/isosize output.iso) / 512
  33928

(With the next version of xorriso it will be possible to choose
   -boot_image any iso_mbr_part_type=0FC63DAF-8483-4772-8E79-3D69D8477DE4
 for "Linux filesystem data".)

------------------------------------------------------------------------

As can be seen with MKRESCUE_SED_IN_EFI_NO_PT, such a pseudo-xorriso
program between grub-mkrescue and xorriso gives the caller control
over the stage between GRUB file preparation and ISO production run.

For example, it would be possible to make a smaller FAT image, copy
the files from grub-mkrescue's EFI images over to the small one,
and then use that as EFI partition.

It would of course be possible to create such a program in Guile.
Be invited to pick from grub-mkrescue-sed.sh what you need.

--------------------------------------------------------------------
Somewhat off topic:

The script also has a killer for the MBR signature in the EFI image
file. It gets activated together with partition table erasure by
  export MKRESCUE_SED_IN_EFI_NO_PT=extra
It is meant for those users who are curious whether block 0 of an EFI
partition really must look like an MBR to be accepted by EFI firmware.

It would be interesting to learn about EFIs which fail without MBR
signature.

The specs say that there may be "compatibility code" in a EFI System
Partition. They talk of "MBR" and understanding partitions in the EFI
partition. I understand it as "may be there, but does not have to".

Minimal test outside of Guix ISO production:

- Install GRUB and configure it for EFI for 32 or 64 bit as your firmware
  needs. (Both is combinable, as Debian GRUB packages demonstrate.)

- Create a minimal bootable ISO. It will not offer more than a GRUB
  command prompt after it was successfully booted:

    mkdir minimal
    touch minimal/empty-file.txt

    export MKRESCUE_SED_MODE=original
    export MKRESCUE_SED_IN_EFI_NO_PT=extra

    grub-mkrescue -o output.iso minimal \
                  --xorriso=./grub-mkrescue-sed.sh

- Put the resulting output.iso on USB stick and offer it to your firmware
  at boot time.
  Is it not found by the EFI boot manager ?
  If you let it boot, does the machine not show a GRUB prompt in the end ?

- Negative results with other mode settings in MKRESCUE_SED_MODE would be
  interesting, too.

--------------------------------------------------------------------

Have a nice day :)

Thomas