Re: What tricks used in readlink to make it faster than realpath bash loadable?

[L A Walsh]

On 12/13/2018 2:02 PM, Eric Blake wrote:
> On 12/13/18 3:37 PM, Peng Yu wrote:
>   
> > Hi,
> >
> > `readlink` is faster than `realpath` for a large number of input
> > arguments. Note that the former starts slower than the latter. What
> > tricks is used in readlink to make it faster? Thanks.
> >     
>
> Why don't you use strace and/or read the code to compare the two 
> implementations, instead of asking someone else to do it for you?
>   
It could be that someone might already know without anyone having
to do it. 

I see that usually being what people want when they ask a question
like that -- most are not asking someone else to do it Eric, they
are asking if anyone might already know the answer.  If not THEN
they might follow your advice and track it down further.

The way you phrase your question implies that they are too lazy
to do it themselves, and that, like you, they are assuming no one
on this list would be knowledgable enough to know the answer off
the top of their head, which may be true, but does sorta hint
at your expectations of other list members.  Personally,
I'd think someone might already have been curious about that
and checked it out, maybe not, but you don't know until you've
asked. 

You did remind me of a script I wrote sometime back to
show the most used calls in an strace of something running.
(script attached -- combination of shell+perl)

It seems to echo some code unnecessarily, but at the end still gives
the 10 top time users in strace (10 was arbitrarily chosen)

Never bother to do much more with it but it hints at the reason in
this case:

wheretime realpath "-e $(printf '. %.0s' {1..50000}) >/dev/null"
realpath:
    0.000345    brk(0)                    = 0x60b000
    0.000089    close(3)                  = 0
    0.000029    mprotect(0x609000, 4096, PROT_READ) = 0
    0.000029    read(3, "", 8192)         = 0
    0.000029    write(2, ""..., 10realpath: )       = 10
    0.000026    access("/etc/ld.so.preload", R_OK) = -1
    0.000026    openat(AT_FDCWD, "/usr/lib/locale/en_US.
    0.000026    openat(AT_FDCWD, "/usr/share/locale/loca
    0.000025    openat(AT_FDCWD, "/lib64/libc.so.6", O_R
    0.000024    arch_prctl(ARCH_SET_FS, 0x7fc5d29c9500)

wheretime readlink "-e $(printf '. %.0s' {1..50000}) >/dev/null"
readlink:
    0.000319    brk(0)                    = 0x60a000
    0.000094    close(3)                  = 0
    0.000035    mprotect(0x608000, 4096, PROT_READ) = 0
    0.000031    access("/etc/ld.so.preload", R_OK) = -1
    0.000031    read(3, "", 8192)         = 0
    0.000029    openat(AT_FDCWD, "/etc/ld.so.cache", O_R
    0.000028    write(2, ""..., 10readlink: )       = 10
    0.000027    openat(AT_FDCWD, "/lib64/libc.so.6", O_R
    0.000027    openat(AT_FDCWD, "/usr/lib/locale/en_US.
    0.000025    arch_prctl(ARCH_SET_FS, 0x7fecd4f99500)

Looks like realpath makes more calls to malloc/alloc or similar
to allocate memory that creates a small overhead of 26 microseconds,
but more extensive testing might be required to get a better idea.

Of course I didn't do any work -- but I did happen to have a script
lying around since 2014 that did something in this area -- who knows
they might find it helpful, or not...but if they don't ask they don't
find out.

P.s. given its been over 4 years since I wrote this, I doubt I can
add any insight into what I did  except by reading it ...








#!/bin/bash -x

# by Linda A. Walsh (c) 2014 -- free for use to nice people!
#  (not guaranteed to do anything useful)...


p=$0
cmd="/bin/sleep 1.5"
lines=10

if [[ $# -lt 1 ]]; then
        echo "wheretime \"cmd [flags]\" [opt#lines]"
        echo "    -     w/cmd+flags in quotes -- to time cmd & flags"
        echo "using default test \"$cmd\" $lines"
else
        cmd="$1"
        shift
fi

if [[ $# ]]; then 
        if [[ $1 =~ ^[0-9]+$ ]]; then lines=$1; fi
fi

#testing
#if [[ $p -nt /tmp/timings.txt ||
#                       /tmp/timings.cmd -nt /tmp/timings.txt ]]; then
        rm -f /tmp/timings.{cmd,txt}
#fi

read last < <(if [[ -e /tmp/timings.cmd ]]; then cat /tmp/timings.cmd; else  
echo ""; fi)
TIMEFORMAT="%2Rsec %2Uusr %2Ssys (%P%% cpu)"
if [[ $last != $cmd ]]; then
  echo "generating timings for $cmd..."
  echo "$cmd" >/tmp/timings.cmd
  time strace  -s 0 -r -qq -f $cmd >& /tmp/timings.txt
fi

prog='
##!/usr/bin/perl;use warnings;use strict;use P;

my ($val, $calls)=(0,{});
my $lines=15;

if ($#ARGV>=0) {
        $lines=$ARGV[0] if $ARGV[0] =~ /^\d+$/;
        shift;
}

while(<>) { 
chomp;
        if (m{((?:\[pid\s\d+\])?)\s*([\d\.]+)\s(\S+.*)$}){
                my ($pid,$time,$call)=($1, $2, $3);
                $call=($pid||"").$call;
                if (ref $calls->{$call}) { 
                        $calls->{$call}[0]=$calls->{$call}[0]+$time; 
                        #P "cc1=%s", $calls->{$call}
                } else { 
                        $calls->{$call}=[$time, $call];
                }
        }
}
my @scalls = sort { my ($ca, $cb) = @{$calls}{$a,$b}; 
                                                                                
$cb->[0] <=> $ca->[0] || 
                                                                                
$ca->[1] cmp $cb->[1] } keys %$calls;
my $i;

for ($i=0; $i<$lines; ++$i) {
        my $sc=$scalls[$i];
        printf "  %11.6f    %.40s\n", @{$calls->{$sc}}[0,1];
}
# vim: ts=2 sw=2
'

perl -e "$prog" "$lines" </tmp/timings.txt
# vim: ts=2 sw=2