Re: [Qemu-devel] hotplug: VM got stuck when attaching a pass-through device to the non-pass-through VM for the first time

[Zhanghaoyu (A)]

>> Hi, all
>> 
>> The VM will get stuck for a while(about 6s for a VM with 20GB memory) when 
>> attaching a pass-through PCI card to the non-pass-through VM for the first 
>> time. 
>> The reason is that the host will build the whole VT-d GPA->HPA DMAR 
>> page-table, which needs a lot of time, and during this time, the 
>> qemu_global_mutex lock is hold by the main-thread, if the vcpu thread IOCTL 
>> return, it will be blocked to waiting main-thread to release the 
>> qemu_global_mutex lock, so the VM got stuck.
>> The race between qemu-main-thread and vcpu-thread is shown as below,
>> 
>>               QEMU-main-thread                                vcpu-thread    
>>        
>>                      |                                             |
>>       qemu_mutex_lock_iothread                     
>> qemu_mutex_lock(&qemu_global_mutex)
>>                      |                                             |
>>         +----loop- ->+                               +----loop---->+         
>>       
>>         |            |                               |             |
>>         |  qemu_mutex_unlock_iothread                | 
>> qemu_mutex_unlock_iothread 
>>         |            |                               |             |         
>>      
>>         |           poll                             |    
>> kvm_vcpu_ioctl(KVM_RUN) 
>>         |            |                               |             |         
>>      
>>         | qemu_mutex_lock_iothread                   |             |
>>         |            |                               |             | 
>>  
>> --------------------------------------------------------------------------------------
>>         |            |                               |  
>> qemu_mutex_lock_iothread
>>         |   kvm_device_pci_assign                    |             |         
>>      
>>         |            |                               |   blocked to waiting 
>> main-thread to release the qemu lock
>>         |      about 6 sec for 20GB memory           |             |         
>>      
>>         |            |                               |             |         
>>             
>>         +------------+                               +-------------+         
>>      
>> 
>> 
>> Any advises?
>> 
>> Thanks,
>> Zhang Haoyu
>
>What if you detach and re-attach?
>Is it fast then?
Yes, because the VT-d GPA->HPA DMAR page-table has been built, no need to 
re-build it.

>If yes this means the issue is COW breaking that occurs with get_user_pages, 
>not translation as such.
>Try hugepages with prealloc - does it help?
Yes, a bit help gained, but it cannot resolve the problem completely, the stuck 
still happened.

Thanks,
Zhang Haoyu