Re: [Chicken-janitors] #1200: file->byte-blob can only read files in text mode on Windows

[Chicken Trac]

#1200: file->byte-blob can only read files in text mode on Windows
-------------------------+-----------------------
  Reporter:  dthedens    |      Owner:
      Type:  defect      |     Status:  new
  Priority:  minor       |  Milestone:  someday
 Component:  extensions  |    Version:  4.9.x
Resolution:              |   Keywords:  byte-blob
-------------------------+-----------------------

Comment (by dthedens):

 Tests for my own use cases run fine with the new version. However, the
 tests revealed some other subtleties related to text versus binary mode on
 Windows. The default mode for Windows file operations (and maybe all
 ports?) is text mode. In this mode, line endings are automatically
 converted to and from Windows-style CR/LF (see
 https://en.wikipedia.org/wiki/Newline#In_programming_languages, this is
 considered part of the C standard). I should probably also point out that
 I built my Windows version of Chicken with mingw64. I don't know if a
 cygwin or other build would be different.

 In the tests, the `byte-blob` data that is supposed to be written to the
 file is `(10 9 8 9 6 9 4 9 2)`. However, this is written in text mode, and
 so the `10` is interpreted as a LF and converted to CR LF, so the
 resulting (binary) data in the file is actually `(13 10 9 8 9 6 9 4 9 2)`.

 When the `byte-blob-read` test is run, the port is opened in binary mode.
 The read specifies 9 bytes (since it was expected 9 bytes were written to
 the file). In this case, the 9 bytes are `(13 10 9 8 9 6 9 4 9)`. This
 makes the `byte-blob-read` test fail.

 The default (text mode) `file->byte-blob` test reads all the bytes in the
 file, `(13 10 9 8 9 6 9 4 9 2)` and with text mode line-ending conversion
 reports back `(10 9 8 9 6 9 4 9 2)`. This is the correct result as
 compared to the data written, so the default text mode test succeeds.

 The binary mode `file->byte-blob` test also reads all the bytes in the
 file, but does not do the line ending conversion so the data reported back
 is `(13 10 9 8 9 6 9 4 9 2)`. This test fails since it does not match what
 was supposed to have been written because it was written in text mode.

 Given that the data read for the port and file-based functions is read in
 binary mode in the test, it would make sense to me to make the temporary
 file that is created for this in the tests to also be written in
 `#:binary` mode. I think this can be done by adding the `#:binary` keyword
 to the `(open-output-file* fd)` call (as further excruciating detail, the
 `open-output-file*` call did not work for me on Windows; I had to change
 it to `(open-output-file temp-path)`. This was still OK since the `temp-
 port` value is never used anyway. The failure may be a result of the
 mingw64 environment, I don't know). Anyway, by writing the temporary test
 file in binary mode, all the tests pass.

 I think the change you made to `file->byte-blob` to add the optional
 argument is sufficient. `byte-blob-read` and `byte-blob-write` both take
 ports as arguments, so it's up to the user to make sure it's the correct
 (text or binary) mode when opening. Now that `file->byte-blob` has that
 option, the same is true as well. It is left to the user to assure that
 reading and writing of files and ports are done with the correct mode
 (text or binary).

--
Ticket URL: <http://bugs.call-cc.org/ticket/1200#comment:5>
CHICKEN Scheme <http://www.call-cc.org/>
CHICKEN Scheme is a compiler for the Scheme programming language.