Re: [Bug-datamash] Regarding 'transpose'

[Assaf Gordon]

Hello Sanjeev,

Thanks again for reporting this and providing easily reproducible examples.

I've added two bug fixes for issues relating to this issue.
Available in the git repository:
http://git.savannah.gnu.org/cgit/datamash.git/

And an non-official tarball here:
http://download-mirror.savannah.gnu.org/releases/datamash/src/datamash-1.1.0.3-3741.tar.gz

With these fixes, 'datamash transpose' (and also 'datamash check') works on 
your input file, without requiring "--no-strict".

However, I'd like to expand on the "--filler" option after these fixes.
The fixes accept lines that end in a delimiter at the end of the line, 
indicating an empty field (previously it was ignored).
This means that the field is no longer missing (it is there, just empty) - thus 
"filler" isn't used at all.

The will illustrate, using ':' as delimiter.

Using this input:

  $ printf "a::c\n1:2:\nX:Y:Z\n"
  a::c
  1:2:
  X:Y:Z

The middle field on the first line is empty, and the last field on the second 
line is empty.
The current datamash (1.1.0, before this bug fix) would reject the second line,
thus requiring "--no-strict":

  $ printf "a::c\n1:2:\nX:Y:Z\n" | datamash-1.1.0 -t: transpose
  datamash: transpose input error: line 2 has 2 fields (previous lines had 3);
  see --help to disable strict mode

  $ printf "a::c\n1:2:\nX:Y:Z\n" | datamash-1.1.0 -t: --no-strict --filler '*' 
transpose
  a:1:X
  :2:Y
  c:*:Z

With this bug fix, datamash treats the 3rd field of the second line as valid 
but empty,
and 'just works':

  $ printf "a::c\n1:2:\nX:Y:Z\n" | datamash -t: transpose
  a:1:X
  :2:Y
  c::Z

This works the same for TAB (i.e. two consecutive tab characters indicate an 
empty field between them,
and a tab at the end of the line indicate an empty last field).

I hope this addresses the issue.
If not, or if you spot other issues, please let me know.

regards,
- assaf