Re: [Dazuko-devel] Dazuko driver takes lots of cpu time, bottleneck discovered

[John Ogness]

Sami Tikka wrote:
> What is the reason for keeping this list? Is it used only for
> determining if a file is 'dirty', i.e. has been written to after open()?

The list is necessary because the sys_close() function does not provide any 
information except a file descriptor. This list is for looking up files from 
file descriptors. As long as ON_CLOSE events should report file names, this 
list is necessary.

And with the new improvements in the FreeBSD extension of Dazuko, this list 
becomes even more important for ON_OPEN. In FreeBSD the dup() and dup2() 
functions are also hooked, which generate additional ON_OPEN events.


> If that's the case, then how about abandoning the list altogether and
> using brute force: send DAZUKO_CLOSE_MODIFIED event for every close() if
> the file was opened with flags O_WRONLY|O_RDWR and do not send the event
> if the file was opened O_RDONLY. I believe the Linux 'struct file'
> carries the open flags in its f_flags field.

The dirty flag is a secondary use of the list to also mark when open files 
have been written to. Since this happens with each sys_write(), it would be 
wise for us to optimize this lookup scheme further (although "Swade" already 
did some excellent work for optimizing ON_CLOSE_MODIFIED).


> I guess there could be situations where a process forgets to call
> close, either on purpose or otherwise...

This usually doesn't matter because the system will close the descriptor, 
even if the program itself did not. I think leaks are occurring because 
certain types of "files" don't require a close from the system and so are 
never closed. /dev/null might be an example of this. It definately needs to 
be investigated further.


> (Lots of references to /var/run/utmp. I believe it is the 'top' utility
> that is responsible for those.)

Yes, this is exactly the problem. The question is, why is dazuko not getting 
a corresponding close for each of those opens? I have thought about if 
dazuko should have an expire time on the list entries so that older items 
can just be removed. Of course, understanding the cause of the problem would 
make me happier.

John Ogness

--
Dazuko Maintainer