Re: How to sort and count efficiently?

[Assaf Gordon]

Hello,

On Sun, Jun 30, 2019 at 07:34:19AM -0500, Peng Yu wrote:
> Hi,
> 
> I have a long list of string (each string is in a line). I need to
> count the number of appearance for each string.
> 
> I currently use `sort` to sort the list and then use another program
> to do the count. The second program doing the count needs only a small
> amount of the memory as the input is sorted.
> 
> [...] Does anybody know any better way
> to make the sort and count run more efficiently?
> 

Using awk:

  awk 'a[$1]++ {} END { for (i in a) { print i, a[i] } }' INPUT \
    | sort -k1,1

Or using gnu awk:

  awk 'a[$1]++ {}
       END { n = asort(a) ;
             for (i = 1; i <= n; i++) {
                 print i, a[i]
             }
       }'


regards,
 - assaf