[Qemu-devel] PC keyboard emulation (was: Regression: more 0.12 regression (SeaBIOS related?))

[Jamie Lokier]

Kevin O'Connor wrote:
> On Thu, Mar 11, 2010 at 03:42:28PM +0800, Roy Tam wrote:
> > Sorry I can't find documentation on this usage. But instead I have
> > lots of old programs written with this usage.
> > Using undocumented features from BIOS/DOS is very usual in that time.
> 
> Can you confirm these other programs fail in the same way (no keyboard
> input, and "ps2 irq but no data." messages in log when using
> bios.bin-0.5.1-debug-20100228)?
> 
> > > It's broken because it causes key presses to be lost and corrupted.
> > > The ps2 port hardware just doesn't work the way that software is
> > > trying to use it.
> > >
> > 
> > You said that "it causes key presses to be lost and corrupted" but I
> > haven't heard any complain about this.
> > Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
> > very well and no key press are lost or corrupted.
> 
> Under qemu-0.11 normal typing lead to lots of keyboard errors for me.
> It's possible real hardware would be less susceptible to this error,
> but there is nothing that a BIOS inside qemu can do to stop the
> corruption.

The DOS coding method brought up in this thread, resulting
in two reads of port 0x60, is quite common.  It works on all real PCs,
and correct emulation must handle it.

I'm not sure if reading port 0x60 is supposed to clear the status bit
immediately or not.  The problem might be QEMU's hardware emulation
clearing the 8042 status bit too quickly, or it might be SeaBIOS
should not check the status bit - in which case it probably fails on
_real_ hardware too, when running these old DOS TSRs and similar
programs.

It would be good if someone can check the behaviour of real hardware.
Roy, are you able to run one of the DOS drivers on a real PC with a
modified keyboard TSR and read port 0x64 in between the two reads of
port 0x60?

The same keyboard problem was brought up on qemu-devel 6 months ago,
as an emulation problem (with hacky patch that didn't fix all uses),
and I thought it had been addressed in QEMU, but maybe not:

>> To: Jamie Lokier <address@hidden>
>> Cc: address@hidden
>> Date: Sun, 23 Aug 2009 19:20:31 +0200
>> Subject: Re: [Qemu-devel] [PATCH 0 of 1] Fix for DOS keyboard problems
>> From: Stefan Ring <address@hidden>
>> 
>> On Sun, Aug 23, 2009 at 2:44 PM, Jamie Lokier<address@hidden> wrote:
>> > Stefan Ring wrote:
>> >> The keyboard is still very unreliable when a keyboard driver is
>> >> loaded inside DOS (I assume that the keyboard driver completely
>> >> disables the BIOS handler).  This behavior is also present without
>> >> my patch, and I personally don't care about it, so this should not
>> >> be an obstacle.
>> >
>> > Can you say a bit more about what your patch actually fixes?
>> >
>> > I've been using QEMU with several versions of MS-DOS and haven't
>> > noticed any keyboard problems.
>> >
>> > I even use various "halt-on-idle" idle programs, and haven't noticed
>> > any keyboard problems.
>> >
>> > So what behaviour does your patch fix?
>> 
>> The particular problem that I noticed and that is 100% reproducible
>> and also very understandable when you look at what the DOS programs
>> do, is that in Borland's Text-Mode DOS IDEs (most likely BC++ 3.1, 4,
>> Turbo Pascal 6 but definitely Turbo Pascal 7) and IIRC also in Turbo
>> Vision programs generated by them, every cursor key press is
>> interpreted twice. So when you open up the leftmost menu and press the
>> down cursor key once, the third entry gets selected instead of the
>> second one.
>> 
>> The reason for this is that said programs install an IRQ 1 handler
>> which does little more than read from port 60h and pass control to the
>> underlying BIOS (or keyboard driver) handler. The BIOS handler reads
>> port 60h again, and should apparently see the same value as the first
>> handler. The problem with the cursor keys is that they generate two
>> scan codes in succession on port 60h that should be read by two
>> separate IRQ handler activations but because QEMU's original behavior
>> is to consume one data byte per read from 60h, the data gets used up
>> too fast.

There was a patch accompanying that thread.  However, this is what I
wrote about the patch:

[Jamie Lokier wrote:]
>>> Let's see if I understand your explanation.
>>> 
>>>    1. Cursor key is pressed.  The key press is represented as two scan 
>>> codes.
>>>    2. IRQ 1 is entered.
>>>    3. Borland's code reads port 60h - gets the first scan code.
>>>    4. BIOS's code reads port 60h - gets the second scan code.
>>>    5. Return from IRQ 1.
>>> 
>>>    6. Cursor key is released.  The key release is represented as two
>>>       scan codes.
>>>    7. IRQ 1 is entered.
>>>    8. Borland's code reads port 60h - gets the first scan code.
>>>    9. BIOS's code reads port 60h - gets the second scan code.
>>>   10. Return from IRQ 1.
>>> 
>>> So both Borland's code and the BIOS are *missing* scan codes.
>>> 
>>> How does that result in Borland seeing *multiple* cursor key
>>> press/release sequences?
>>> 
>>> 
>>> Apparently when port 0x60 is read, that de-asserts IRQ1, resets the
>>> IBF flag ("input buffer full"), and another byte could be received
>>> from the keyboard.  However, reading port 0x60 quickly, before another
>>> byte can be received over the keyboard cable, should return the same byte.
>>> 
>>> So I agree that the emulated port 0x60 should return the same value if
>>> there has not been enough time for the (emulated) keyboard cable to
>>> transmit another scan code.
>>> 
>>> But detecting the particular sequence used by Borland code and the
>>> BIOS together is a hack.  I'm not surprised that, as you say, other
>>> DOS keyboard drivers remain broken after the patch.
>>> 
>>> If it goes in, the patch should include a very clear comment that the
>>> "held" value and detecting the disable/read/enable sequence is only a
>>> workaround for what Borland does and also depends on the BIOS
>>> sequence, and is not a correct emulation in general.

All the DOS code which transfers control to the BIOS after reading
port 0x60 assumes that the whole sequence is quite fast - faster than
the 8042 can change the byte.

That should always be true on a real PC.

Unfortunately that means "correct" hardware emulation must behave
differently if a short guest time elapses between two port 0x60 reads
versus a longer one.

Here are some excerpts from DOSEMU technical documentation:

    http://www.dosemu.org/docs/README-tech/0.99/README-tech-8.html

#

      if a dos application or TSR has redirected the keyboard
      interrupt, its handler might read from port 60h to get raw
      scancodes. Port 60h is of course virtualized, and the read
      returns the value from raw_buffer.

      Note that a mix between the two cases is also possible, e.g. a
      TSR's int9 handler first reads port 60h to check if a particular
      key was pressed, then gives over to the default int9
      handler. Even these cases should be (and are, I think) handled
      properly.

      Note also that in any case, int9 is called once for each raw
      scancode byte. Eg., suppose the user pressed the PgDn key, whose
      raw scancode is E0 51:

      - first call to int9:

          read port 60h        = 0xe0
          read port 60h        = 0xe0   (**)
          call get_bios_key()  = 0
          iret

        do_irq1() reschedules IRQ1 because further scancodes are in the queue

      - second call to int9

          read port 60h        = 0x51
          call get_bios_key()  = 0x5100    (bios scancode of PgDn)
          iret

      (** multiple port 60h reads during the same interrupt yield the
      same result.)

Note that (**) ^^ - DOSEMU authors knew about this behaviour.  And
they specifically note that DOS applications and TSRs redirect the
BIOS keyboard handling in this way, and require multiple reads to work.

It is even possible to have multiple TSRs installed, so that port 0x60
is read more than 2 times.

Something else is revealed from another DOSEMU document:

    http://www.dosemu.org/docs/README-tech/1.4/x1173.html

      5.3. Known bugs & incompatibilites

      If the interrupt is not acknowledged and the keyboard port is
      read we don't eventually give up like a real keyboard and
      deliver the next byte in the keyboard buffer.

In other words, 8042 emulation _should_ permit port 0x60 to be read
multiple times, but if sufficient time elapses without the irq being
acknowledged, it should drop the byte and deliver the next one.

Unfortunately, in QEMU _guaranteeing_ that the guest has had some
guest time to process the irq before the timeout is a bit tricky, if
QEMU is delayed by host scheduling.

I think that timeout is necessary for some code which polls the ports
with irqs disabled (it sounds familiar), and for just generally
unwedging :-) but I don't have an example in mind.

As I said earlier, I'm not sure if reading port 0x60 is supposed to
clear the port 0x64 status bit immediately, in which cause SeaBIOS
must be fixed, or if SeaBIOS is fine and the emulation must be fixed.

It would be good if someone can check the behaviour of real hardware
by reading port 0x64 between the two port 0x60 reads in rapid
succession, and/or check if SeaBIOS works fine on real hardware with
these DOS apps/TSRs.

-- Jamie