Re: [Qemu-devel] [PATCH] fix the memory leak for share hugepage

[Linhaifeng]

On 2014/10/20 13:32, Wen Congyang wrote:
> On 10/20/2014 12:48 PM, Linhaifeng wrote:
>>
>>
>> On 2014/10/20 10:12, Wen Congyang wrote:
>>> On 10/18/2014 11:20 AM, Linhaifeng wrote:
>>>>
>>>>
>>>> On 2014/10/17 21:26, Daniel P. Berrange wrote:
>>>>> On Fri, Oct 17, 2014 at 04:57:27PM +0800, Linhaifeng wrote:
>>>>>>
>>>>>>
>>>>>> On 2014/10/17 16:33, Daniel P. Berrange wrote:
>>>>>>> On Fri, Oct 17, 2014 at 04:27:17PM +0800, address@hidden wrote:
>>>>>>>> From: linhaifeng <address@hidden>
>>>>>>>>
>>>>>>>> The VM start with share hugepage should close the hugefile fd
>>>>>>>> when exit.Because the hugepage fd may be send to other process
>>>>>>>> e.g vhost-user If qemu not close the fd the other process can
>>>>>>>> not free the hugepage otherwise exit process,this is ugly,so
>>>>>>>> qemu should close all shared fd when exit.
>>>>>>>>
>>>>>>>> Signed-off-by: linhaifeng <address@hidden>
>>>>>>>
>>>>>>> Err, all file descriptors are closed automatically when a process
>>>>>>> exits. So manually calling close(fd) before exit can't have any
>>>>>>> functional effect on a resource leak.
>>>>>>>
>>>>>>> If QEMU has sent the FD to another process, that process has a
>>>>>>> completely separate copy of the FD. Closing the FD in QEMU will
>>>>>>> not close the FD in the other process. You need the other process
>>>>>>> to exit for the copy to be closed.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Daniel
>>>>>>>
>>>>>> Hi,daniel
>>>>>>
>>>>>> QEMU send the fd by unix domain socket.unix domain socket just install 
>>>>>> the fd to
>>>>>> other process and inc the f_count,if qemu not close the fd the f_count 
>>>>>> is not dec.
>>>>>> Then the other process even close the fd the hugepage would not freed 
>>>>>> whise the other process exit.
>>>>>
>>>>> The kernel always closes all FDs when a process exits. So if this FD is
>>>>> not being correctly closed then it is a kernel bug. There should never
>>>>> be any reason for an application to do close(fd) before exiting.
>>>>>
>>>>> Regards,
>>>>> Daniel
>>>>>
>>>> Hi,daniel
>>>>
>>>> I don't think this is kernel's bug.May be this a problem about usage.
>>>> If you open a file you should close it too.
>>>
>>> If you don't close it, the kernel will help you when the program exits.
>>>
>> Yes,when the hugepage is only used for qemu,the kernel will free the file 
>> object.If the hugepage shared for other process,when qemu exit the kernel 
>> will not free the file.
> 
> Even if the hugepage is shared with the other process, the kernel will auto 
> close the fd when qemu
> exits. If the kernel doesn't do it, it is a kernel bug.
> 
Kernel supply close to fix this bug.If you call open you must call close.
If not, the result is unpredictability.
>>>>
>>>> This is <<linux man page>>about how to free resource of file.
>>>> http://linux.die.net/man/2/close
>>>>
>>>>
>>>> I'm trying to describe my problem.
>>>>
>>>> For example, there are 2 VMs run with hugepage and the hugepage only for 
>>>> QEMU to use.
>>>>
>>>> Before run VM the meminfo is :
>>>> HugePages_Total:    4096
>>>> HugePages_Free:     4096
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>>
>>>> Run the two VMs.QEMU deal with hugepage as follow steps:
>>>> 1.open
>>>> 2.unlink
>>>> 3.mmap
>>>> 4.use memory of hugepage.After this step the meminfo is :
>>>> HugePages_Total:    4096
>>>> HugePages_Free:        0
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>> 5.shutdown VM with signal 15 without close(fd).After this step the meminfo 
>>>> is :
>>>> HugePages_Total:    4096
>>>> HugePages_Free:     4096
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>>
>>>> Yes,it works well,like you said the kernel recycle all resources.
>>>>
>>>> For another example,there are 2 VMs run with hugepage and share the 
>>>> hugepage with vapp(a vhost-user application).
>>>
>>> The vapp is your internal application?
>>>
>> Yes vapp is a application to share the QEMU's hugepage.So threr are two 
>> process use the hugepage.
>>
>>>>
>>>> Before run VM the meminfo is :
>>>> HugePages_Total:    4096
>>>> HugePages_Free:     4096
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>>
>>>> Run the first VM.QEMU deal with hugepage as follow steps:
>>>> 1.open
>>>> 2.unlink
>>>> 3.mmap
>>>> 4.use memory of hugepage and send the fd to vapp with unix domain 
>>>> socket.After this step the meminfo is:
> 
> Do you modify qemu?
> 
>>>> HugePages_Total:    4096
>>>> HugePages_Free:     2048
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>>
>>>> Run the second VM.After this step the meminfo is:
>>>> HugePages_Total:    4096
>>>> HugePages_Free:        0
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>>
>>>> Then I want to close the first VM and run another VM.After close the first 
>>>> VM and close the fd in vapp the meminfo is :
>>>> HugePages_Total:    4096
>>>> HugePages_Free:        0
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>
>>> Does the qemu still run after you close the first VM? If the qemu exits, 
>>> the fd will be closed by the kernel, so this
>>> bug is very strange.
>>>
>> qemu is not run when close the first VM.If other process used the file will 
>> be closed by kernel too?
> 
> If qeum doesn't run after the first vm is closed, the fd should be closed 
> even if another process uses the file.
> 
>>
>>>>
>>>> So failed to run the third VM because the first VM have not free the 
>>>> hugepage.After apply this patch the meminfo is:
>>>> HugePages_Total:    4096
>>>> HugePages_Free:     2048
>>>> HugePages_Rsvd:        0
>>>> HugePages_Surp:        0
>>>> Hugepagesize:       2048 kB
>>>> So i can run the third VM success.
>>>>
>>>
>>>
>>> .
>>>
>>
> 
> 
> .
> 

-- 
Regards,
Haifeng