Re: [PATCH 1/2] fts: introduce the FTS_NOLEAF flag

[Kamil Dudka]

On Thursday, December 10, 2015 2:26:38 AM CEST Pádraig Brady wrote:
> On 09/12/15 10:35, Pádraig Brady wrote:
> > On 09/12/15 06:34, Kamil Dudka wrote:
> >> The flag is needed to implement the -noleaf option of find.
> >> * lib/fts.c (link_count_optimize_ok): Implement the FTS_NOLEAF flag.
> >> * lib/fts_.h (FTS_NOLEAF): New macro, shifted conflicting constants.
> > 
> > Is this exposed to fix issues with certain file systems,
> > or just in case there may be issues, or support easily
> > testing find without the leaf optimization?
> > 
> > I see Jim said the current FTS implementation
> > would make -noleaf a no-op there:
> > https://lists.gnu.org/archive/html/bug-gnulib/2008-12/msg00280.html
> > 
> > cheers,
> > Pádraig.
> > 
> > p.s. I see that find does a stat per file on XFS,
> > while d_type can be used to distinguish dirs there.
> > On XFS DT_DIR is set for dirs and DT_UNKNOWN otherwise.
> > I wonder is there some optimization we could do for that case.
> 
> I did a quick check on XFS which suggests the leaf optimization
> based on st_nlink is valid:
>   test $(($(find . -maxdepth 1 -type d | wc -l) + 1)) = $(stat -c %h .) &&
> echo leaf_ok
> 
> Applying this diff:
> @@ -717,6 +718,7 @@ leaf_optimization_applies (int dir_fd)
>      {
>        /* List here the file system types that lack usable dirent.d_type
>           info, yet for which the optimization does apply.  */
> +    case S_MAGIC_XFS:
>      case S_MAGIC_REISERFS:
>        return true;
> 
> Gives this significant speedup:
>   $ time find/find-before /usr/share >/dev/null
>   real    0m0.410s
>   user    0m0.145s
>   sys     0m0.266s
> 
>   $ time find/find-after /usr/share >/dev/null
>   real    0m0.278s
>   user    0m0.147s
>   sys     0m0.131s

Pádraig, do you remember which kernel you tested it with?

I am currently not able to get any speedup on XFS with Fedora/el7 kernels...

Kamil

> I also noticed a lot of fcntl calls on XFS
> (basically one per file), which I need to look further into:
>   $ strace -c find/find /usr/share >/dev/null
>   % time     seconds  usecs/call     calls    errors syscall
>   ------ ----------- ----------- --------- --------- ----------------
>    40.03    0.147809           1    151710           fcntl
>    17.62    0.065069           1     63154           close
>    14.52    0.053608           1     40071           newfstatat
>    13.15    0.048547           1     35400           getdents
> 
> cheers,
> Pádraig