Re: [Qemu-devel] [RFC PATCH v1 7/9] cpus: move icount preparation out of tcg_exec_cpu

[Pavel Dovgalyuk]

> From: Alex Bennée [mailto:address@hidden
> Paolo Bonzini <address@hidden> writes:
> 
> > On 04/04/2017 14:31, Alex Bennée wrote:
> >>
> >> Paolo Bonzini <address@hidden> writes:
> >>
> >>> On 04/04/2017 12:46, Alex Bennée wrote:
> >>>>> In theory the main-loop should be sequenced before or after vCPU events
> >>>>> because of the BQL. I'm not sure why this is not currently the case.
> >>>>
> >>>> It seems cpu_handle_exception doesn't take the BQL until
> >>>> replay_exception() has done its thing. This is fixable but the function
> >>>> is a mess so I'm trying to neaten that up first.
> >>>
> >>> Long term neither cpu_handle_exception nor cpu_handle_interrupt need the
> >>> BQL at all.
> >>
> >> Well for record/replay they might. Otherwise we end up moving the record
> >> stream on even though a checkpoint might be being written by the
> >> main-loop.
> >>
> >> As far as the cc->do_interrupt() stuff is concerned it will be guest
> >> dependant because you could end up in device emulation code down this
> >> path which must be protected by the BQL - the arm_gic code being a good
> >> example.
> >
> > I think recording an event could be split in two parts:
> >
> > - recording the (icount, event) tuple and getting back a unique event id
> >
> > - waiting for all events with lower event id to be complete before
> > starting to process this one
> >
> > This doesn't require the BQL, you can use a condition variable on
> > replay_lock (but you do need to unlock/lock the BQL around it if
> > currently taken).
> 
> Would you then leave the recording to the stream to the main-loop
> thread? I guess it would marshal all events that occurred before the
> checkpoint first and then finish draining the queue after recording its
> checkpoint?
> 
> Wrapping the exception stuff in the BQL does improve the repeat-ability
> but of course it breaks if I take away the graceful handling of time
> differences because there is a race between recording the exception
> event (with current_step+insns so far) and getting back to the main loop
> where insns is finally credited to timers_state.qemu_icount.
> 
> I guess we could improve the situation by updating
> timers_state.qemu_icount (under BQL) as we record events. I don't know
> how clunky that would get.

Does io instructions make some lock to prevent races in virtual hardware?

vCPU thread updates icount in the beginning of the TB execution.
It means that checkpoints in the replay log will appear only at the boundaries
of TBs. However, the same log may be generated by different scenarios.
Consider the following cases:

1. Sequence: vCPU-block-begin vCPU-update-icount iothread-io vCPU-io 
vCPU-block-end
2. Sequence: vCPU-block-begin vCPU-update-icount vCPU-io iothread-io 
vCPU-block-end

These sequences will generate the same order of replay events, but different
states of virtual hardware.

Therefore we need some lock for the time while vCPU executes translation block
(or the whole sequence of blocks as in old times).

> > The complicated part is ensuring that there are no deadlocks where the
> > I/O thread needs the VCPU thread to proceed, but the VCPU thread is
> > waiting on the I/O thread's event processing.
> 
> This sort of update sounds more like 2.10 material though.

Pavel Dovgalyuk