Re: [Qemu-devel] virtqueue corruption in emulation mode?

[Sinha, Ani]

On Sep 27, 2011, at 12:17 AM, Stefan Hajnoczi wrote:

> On Mon, Sep 26, 2011 at 07:16:56PM -0500, Sinha, Ani wrote:
>> I am using the virtqueue (virtqueue_pop, virtqueue_push etc) in the emulated 
>> mode (non-kvm mode) from an IO thread (a separate thread different from main 
>> QEMU thread). What I am observing is that the virtqueue memory seems to get 
>> corrupt. Either qemu crashes while performing virtqueue_push() 
>> (virtqueue_push() -> virtqueue_fill() 
>> ->bring_used_idx()->lduw_phys()->qemu_get_ram_ptr()->"bad ram offset") or 
>> crashes when the guest accesses a bad memory while using virtqueue. Now this 
>> never ever happens when I run QEMU in KVM mode (/dev/kvm present) OR when I 
>> use my functions from within the main qemu thread. I am unable to figure out 
>> why this is happening. I have looked into my code over and over again and I 
>> can't seem to explain this behavior. Can any of you guys give me any inkling?
>
> QEMU is not thread-safe in general.  It uses a big lock to protect most
> of its internal state.


I see. So may be I should do something like qemu_set_fd_handler(fd, …) where fd 
is a pipe and the handler does the virtqueue_push() etc?
Now my question is, is it safe to do elem = virtqueue_pop(vq) from main event 
loop, then so some work on the elem popped out in an worker thread and then at 
some later point do a virtqueue_push(vq, elem) from that handler (which is 
called by main_loop() ->main_loop_wait())?  In other words, the vq reference is 
being used from the main event loop at two different points from two different 
functions but not in a contiguous fashion within the same function.


>
> When you say "an IO thread" it sounds like you spawn a new thread
> outside the big lock (qemu_global_mutex).  You cannot call the existing
> virtqueue functions outside the big lock because they traverse (and
> modify!) the memory management data structures.


Thanks for this info.


>
> Please call new threads "helper threads" or something other than "IO
> thread" because I/O thread has a specific meaning in QEMU.  It's the
> event loop thread that execute main_loop_wait() and dispatches fd
> handlers when select(2) returns.  This will prevent confusion.

understood.

>
> If you follow the way that existing virtio devices are implemented there
> should be no problem.
>


yeah, perhaps. It will require a little more thinking.


Ani


============================================================
The information contained in this message may be privileged
and confidential and protected from disclosure. If the reader
of this message is not the intended recipient, or an employee
or agent responsible for delivering this message to the
intended recipient, you are hereby notified that any reproduction,
dissemination or distribution of this communication is strictly
prohibited. If you have received this communication in error,
please notify us immediately by replying to the message and
deleting it from your computer. Thank you. Tellabs
============================================================