Re: About firmware facilities

[Brendan Trotter]

Hi,

On Tue, Sep 15, 2009 at 6:19 AM, Vladimir 'phcoder' Serbinenko
<address@hidden> wrote:
> On Mon, Sep 14, 2009 at 10:12 PM, Brendan Trotter <address@hidden> wrote:
>> On Tue, Sep 15, 2009 at 4:41 AM, Pavel Roskin <address@hidden> wrote:
>>> On Tue, 2009-09-15 at 04:27 +0930, Brendan Trotter wrote:
>>> We are already doing it.  There is functional ATA support, USB support
>>> is under development.
>>
>> But, are you doing it for valid technical reasons?
>>
> Yes. I have a borked firmware right on this laptop.

Sounds like you need better firmware - Coreboot might be a good
alternative. I assume GRUB is meant to be a boot loader though (rather
than a project to rewrite every piece of dodgy software ever written).

>>> GRUB can serve as BIOS together with Coreboot.
>>
>> I know. It'll break my code.
>>
>> The multi-boot specification says "However, other machine state should
>> be left by the boot loader in normal working order, i.e. as
>> initialized by the bios (or DOS, if that's what the boot loader runs
>> from)."; although I seem to remember it saying words to the effect of
>> "the firmware should be left in a usable state".
>>
> Firmware if present is left in usable state. However it may simply not
> be present.

So if the firmware is present, GRUB won't alter the state that the
firmware left the hardware in (or if it does it'll restore all
hardware to the default firmware state before starting the OS);
including all devices that GRUB uses via. it's own device drivers
(except for specific things mentioned in the multi-boot specification,
like the A20 gate)?

>> Due to limitations in the original multi-boot specification my code
>> switches back to real mode and uses the BIOS to do memory detection,
>> do video mode detection, switch video modes and gather other
>> information.
>
> Have you actually read the multiboot specification? Booter passes info
> about memory and video mode in mbi (video for multiboot isn't
> implemented yet). If you need firmware for basic bootup you're clearly
> doing something wrong and are firmware-dependent. Of course it's your
> freedom to make suboptimal software.

I've read the multi-boot specification. I've also read the code in
GRUB-legacy that does memory detection, and I'm unwilling to allow my
code to rely on it for "quality control" reasons. Without going into
details, GRUB-legacy tends to do a "minimal" job and then expects the
user to fix the problem if/when it goes wrong (but even then it only
offers a "uppermem" command without providing a way for the user to
specify a complete system memory map).

For video, I believe there's patches to make GRUB-legacy comply with
the original multi-boot specification (but for dual boot setups I
can't expect people to replace their main OS's existing GRUB with a
patched version). Even if all versions of GRUB did support the video
stuff in multi-boot, the multi-boot specification is still
ridiculously inadequate.

My code gives the user a choice of any video modes that are supported
by the video card and my code (and doesn't allow them to choose video
modes that aren't supported by my code), including double scanned
modes (if supported by the video card) and a variety of refresh rates
(if supported by the video card). It also uses the monitor's EDID
information to try to prevent video modes that are supported by the
video card but not supported by the monitor from being used. This is
mostly automated, so that (for e.g.) the user can replace the monitor
with another monitor that doesn't support the video mode that was
previously used, and the software will find the best video mode that
the new monitor does support without the user changing anything.

>> If GRUB is running on coreboot or UEFI I can't do this
>> (and can't work around the limitations any other way). Unless the
>> multi-boot specification is changed significantly (such that a
>> multi-boot compliant code needs no work-arounds, or at least so that
>> multi-boot compliant code can determine what sort of firmware there
>> was and use the firmware) then GRUB as a whole becomes useless to me.

> Just say in clean text what changes you need so we can discuss it.
> Don't expect us to understand the statement "your standard is borked,
> change it".

Fair enough...

The video support is already mentioned above.

It'd be nice if other output methods were exposed to the OS too. For
example, if GRUB is configured to use a serial terminal it'd be good
if it told the OS details for the serial port, so that the OS can
continue using the same serial terminal without resetting/restarting
an established connection or requiring the user to configure it in 2
separate places (in GRUB and in the OS).

The same would apply to internationalization - GRUB should tell the OS
which language/locale it used, so that the OS can automatically use
the same language as a default (especially for OS installation CDs).

For memory detection, ACPI 3.0 allows the BIOS (" INT 15H, E820H") to
return extended attributes - mostly only a volatile/non-volatile flag.
This isn't in GRUB's information. ACPI 3.0 also allows the BIOS to
return areas of the type "AddressRangeUnusable" (e.g. faulty RAM). If
the BIOS reports that the area from 0x00100000 to 0x001FFFFF is
faulty, then how is GRUB planning to handle that? My own code (if
using it's own boot loaders rather than GRUB) can tolerate RAM
failures anywhere in RAM except for about 64 KiB in low memory.

While I'm on the subject, I also want a list of RAM areas that the
boot loader has relied on. That way if a 24/7 server detects that any
RAM that the boot loader relies on has become faulty it can warn the
user that the computer may not be able to boot. Currently I assume
that all multi-boot compliant boot loaders (including GRUB) rely on
all RAM below 0x00100000, but it's a dodgy hack I'd rather avoid.

In the memory map, GRUB should mark areas that it has used to pass
information to the OS (e.g. the multi-boot information structure) as
"multi-boot reclaimable" (in a similar way that ACPI uses the "ACPI
reclaimable" type). This would make it easier for the OS to avoid
overwriting this data before it attempts to read it. The area types
need to be "architecture independent" types too (e.g. GRUB converts
ACPI area types, UEFI area types, etc into "standard multi-boot area
types").

The "Boot device" field in the multi-boot information structure should
be improved to handle a variety of conditions; including if the disk
was an emulated disk (e.g. "El Torito" emulating a hard drive). The
BIOS drive number isn't much use (especially if the firmware is
coreboot, UEFI, OpenFirmware, etc), and should be replaced with
something that identifies the corresponding drive structure (this
includes USB). If the boot device was ethernet then the "Boot device"
field should contain a reference to a "network card information
structure" that includes the ethernet card's MAC address (and any IP
addresses, etc from DHCP, if applicable). If the boot device was
firmware (e.g. GRUB and OS embedded in the firmware's flash) then the
"Boot device" field should indicate this. Under no circumstances
should "Boot device" field be invalid (bit 1 in the `flags' word
should always be set).

The drive structures. Here, the "BIOS device number" should be a
"firmware specific identifier". The OS must be able to uniquely
identify each device without using the "firmware specific identifier".
For an example, something like the information returned by "INT 0x13,
ah = 0x48" (see http://www.ctyme.com/intr/rb-0715.htm) would be a good
place to start (e.g. "host bus, interface type, interface path, device
path"). The boot loader (GRUB) should list as many storage devices as
possible (but the boot loader shouldn't be required to guarantee that
all storage devices are listed, or that complete information is
present for all devices that are listed). Note: "storage devices"
means hard drives, floppy drives, CD-ROMs (with and without CDs
inserted), Zip drives, USB flash, USB floppy/CD, tape backup units,
etc. Note: I don't care how obsolete floppy, zip and/or tapes are, the
specification should cover it (even if boot loaders don't).

The "boot loader name" field is nice, but it needs a "boot loader
version" field to go with it. A "firmware type" field is also needed.

The OS image also needs a different magic number to indicate that the
OS image is designed for future versions of the multi-boot
specification (rather than the old/current version). If the OS image
uses the new magic number, then the OS image must also include an
"version of the multi-boot specification that this image complies
with" field. If the OS image indicates that it's intended for a newer
version of the multi-boot specification than the boot loader complies
with, then the boot loader refuses to boot and displays a "this boot
loader needs to be upgraded" error. If the OS image has the old magic
number, and if the firmware is "PC BIOS" then the boot loader should
boot the old OS image. If the OS image has the old magic number, and
if the firmware is not "PC BIOS" then the boot loader refuses to boot
and displays a "this OS requires a PC BIOS" error message.

In a similar way, if the OS image has the old magic number and the
boot loader is not running on 80x86 then the boot loader refuses to
boot and displays an error message. If the OS image has the new magic
number then it must also include a field that indicates which
architecture the OS is intended for, and the boot loader must check
this field and display a "this OS is intended for a different
architecture" error message (and refuse to boot) if it's wrong.

>> The current draft (http://grub.enbug.org/MultibootDraft) is a small
>> step in the right direction; but it has changed much in 3 years.
>>
> It mainly introduces multi-CPU and another MBI format compared to multiboot1

By "multi-CPU" you mean different architectures, rather than
SMT/SMP/NUMA support?

>>>> Why are you worrying about such silly things when the multi-boot
>>>> specification (which actually is relevant) is still severely borked?
>>>
>>> Patches are welcome.
>>
>> I'm not sure it's possible to write patches for a multi-boot compliant
>> boot loader without a specification that defines "multi-boot
>> compliant".
>>
> Read it here: http://www.gnu.org/software/grub/manual/multiboot/multiboot.html

That's the old version of the multi-boot specification (for
GRUB_legacy). I'm looking forward to a new version of the multi-boot
specification (for GRUB 2) that is designed to support different
architectures, different types of firmware on the same architecture,
etc; and hopefully other improvements.


Cheers,

Brendan