[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Zutils-bug] zgrep performance long line
From: |
Antonio Diaz Diaz |
Subject: |
Re: [Zutils-bug] zgrep performance long line |
Date: |
Fri, 17 Aug 2018 12:53:10 +0200 |
User-agent: |
Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.9.1.19) Gecko/20110420 SeaMonkey/2.0.14 |
Hi Walter,
Walter Anema wrote:
You made a nice package with z utilities.
Thanks!
I have a problem with the performance of a special file. It is a file with
logging in json format, without a \n.
I need to append an `echo` before `wc` shows up with a count.
(zcat
/logs/s3/2018/04/11/08/prod-kinesis-firehose-stream-1-2018-04-11-08-05-23-bcdf3841-52b5-47eb-bf85-c36dfa2d0d55;echo
) | wc
1 2145643 37786248
I have crafted a similar file, but I have found no differences between
zutils' zgrep and gzip's zgrep. Also I did not need to append a newline
for wc to show results:
$ zcat messages_nnl.gz | wc
0 7371268 44149261
$ time zgrep -o connect messages_nnl.gz | wc
104 104 832
real 0m24.714s
user 0m22.000s
sys 0m2.250s
$ time gzip-1.9/zgrep -o connect messages_nnl.gz | wc
104 104 832
real 0m24.896s
user 0m22.340s
sys 0m2.250s
What version of grep and wc are you using? Mine are (I have tested in
two machines):
GNU grep 2.5
GNU grep 3.1
wc (GNU coreutils) 6.9
wc (GNU coreutils) 8.11
Best regards,
Antonio.