Re: [Libcdio-devel] [PATCH v2 0/4] Add El Torito virtual image support

[Pete Batard]

On 2024.01.30 08:49, Thomas Schmitt via Libcdio-devel wrote:
> The specs do not prescribe a particular sequence of boot images or their
> position within the overall body of an ISO 9660 filesystem.

The specs do not prescribe that an image should contain extra garbage. 
It is implied.

You can of course always twist specs to make them do something that they 
were *implicitly* never meant to allow, and pat yourself on the back for 
having found a loophole. And then, you'll probably find that, once that 
loophole becomes troublesome, the specs get altered to close it.

If I wanted to have a little fun, I could propose a UEFI specs update 
(since I've gone through that process before), to close this specific 
and obvious loophole...

I'm not going to do that, but I would strongly urge applications not to 
rely on specs loopholes, that could easily be closed.

> Sure, it would be better to have the oversized EFI image near the end of
> the filesystem data range. This would reduce the trailing backpack that is
> caused by the fallback prescription in UEFI specs.
> But achieving that position would be up to the ISO producer. The reader
> must not flatly assume it.

The reader will always assume underlying properties from specs. That's 
how specs have always worked.

> > Especially, as opposed to what Thomas stated in his other reply, nowhere
> > does it states that: "The real size is the size of the FAT filesystem in
> > such an image". Instead, UEFI is quite explicit that the real size is the
> > size from the LSN to the end of the volume.
>
> The size that matters is indeed the size of the FAT filesystem which
> starts at the LBA which is given in the boot catalog.

This is not what the specs say, or even imply. There's no talk of the 
type of file system that MUST or SHOULD be used by the ESP, and for good 
reason, because any file system is acceptable as long as UEFI has a 
driver for it.

The only thing the UEFI specs is that, at the very least, UEFI should 
have support for the FAT file system. But it doesn't imply that FAT must 
be used for the ESP, and I covered that in my previous reply.

> The size which is spedified in UEFI for values 0 or 1 is just a coarse
> workaround for a situation where the catalog format is insufficient and
> where any EFI firmware can get a better size by looking into the FAT
> filesystem.

...provided that it is FAT, which it may not be.

> Actually the exact partition size does not matter for a filesystem driver
> as long as the partition is not smaller than the file system.

This I tend agree with (to the extent of my knowledge of most modern 
file systems), though I'm sure one see a file system that pokes at the 
end of the partition size it is provided with to make this assumption lie.

Because the problem we are being faced with is that the file system used 
*will* be presented by the (UEFI image emulation) system with a 
partition size that is not the actual size of the partition it was 
formatted to.

And if that file system happens to want to place stuff in the last 
superblocks (similar to what one does for partition scheme with the 
backup GPT) and does not read the partition size from a 
beginning-of-the-media based superblock (or does not assume that the 
partition size there can never become corrupted), then it will get a 
wrong size and may start to try to interpret garbage data.

> The immediate reason for the sequence of file data content is in the
> alphabetic order of the paths in the ISO:
> /efi.img comes before /isolinux.bin .

I see.

> You just have to drop the assumption about the convenience of the position
> of the EFI System Partition.

Yes, I am going to have to disregard what the specs implicitly tell, and 
contend with images that may very well (but unlikely) end up not booting 
because the virtual partition size is now a lie (compared to the size it 
was when it was formatted).

> I wonder why you are so opposed to expanding the if-construct

Because I want to handle more than EFI for size 0, and reduce the amount 
of extra code (so that we don't have that many paths to validate in the 
end).

Considering that xorriso always uses 0 as the size for > 32MB 
partitions, I'm planning to craft a v3 that adds a special case for size 
0 to auto-expand to the next image LSN (or max volume if we're already 
on the highest image LSN). Which will allow for all platform IDs that 
want to use the *out of El Torito specs and therefore free for grab* 
size 0 for > 32MB images.

> New objection:
> This reading by struct will probably give a byte-swapped result in the
> .num_sectors fields if the program is run on a big-endian processor.

Good point. I agree this warrants fixing, since I'm going to have to do 
a v3 anyway.

> (Avoidable by reading the records as character array and interpreting
> them byte by byte as described in the El Torito specs.)

Well, I'm sure we already have means to swap bytes when needed in 
lbcdio, so this shouldn't be a big deal.

> Possibly (7-zip's) El Torito code was not renewed since EFI appeared and only
> expects legacy BIOS boot images.

Yes, I am now assuming that too, which should give us some hints on how 
frequent >32 MB El Torito images really are, if nobody has complained to 
the 7-zip folks that it can't handle them.

And I am planning to report this to 7-zip once I can finally get this 
lbcdio stuff out of the way.

No idea when I'll be able to submit a v3 at this stage though...

Regards,

/Pete