Re: [Monotone-devel] OS-specific line endings

[Richard Levitte]

Good, this was discussed years ago, but the discussion drifted off
into nothingness.  A rehash could maybe take us closer to something
workable.

The usual problem is there, how to make certain we don't do
conversions with files that should be seen as binary but aren't
obviously so from a technical point of view.

Cheers,
Richard

In message <address@hidden> on Sat, 03 Jul 2010 12:37:39 -0400, Stephen Leake 
<address@hidden> said:

stephen_leake> I'd like to add some OS-specific line ending handling to mtn.
stephen_leake> 
stephen_leake> The rationale is to deal with a tool that doesn't quite do what 
I want,
stephen_leake> but isn't open source so I can't fix it.
stephen_leake> 
stephen_leake> First I looked thru various documentation to see what line ending
stephen_leake> handling is there.
stephen_leake> 
stephen_leake> NEWS mentions line endings in 0.19:
stephen_leake> 
stephen_leake>     - windows/unix working copy line ending conversion now
stephen_leake>       works correctly.  Thanks to Emile Snyder
stephen_leake> 
stephen_leake> But monotone.texi only mentions it in passing, in 
Internationalization:
stephen_leake> 
stephen_leake>     File contents
stephen_leake>     =============
stephen_leake> 
stephen_leake>        * Files are subject to character set conversion and line 
ending
stephen_leake>          conversion.
stephen_leake> 
stephen_leake>     Cert values
stephen_leake>     ===========
stephen_leake> 
stephen_leake>     Cert values may be either text or binary, depending on the 
return value
stephen_leake>     of the hook `cert_is_binary'. If binary, the cert value is 
never
stephen_leake>     printed to the screen (the literal string "<binary>" is 
displayed,
stephen_leake>     instead), and is never subjected to line ending or character
stephen_leake>     conversion. ...
stephen_leake> 
stephen_leake> The wiki mentions it in several places:
stephen_leake> 
stephen_leake> WorkspaceConflicts.mdwn:
stephen_leake>     * remove the file munging line ending conversion stuff which 
is all
stephen_leake>     tied up in this
stephen_leake> 
stephen_leake> LineEndingMunging.mdwn
stephen_leake>     We currently control this stuff with a hook. This is 
manifestly
stephen_leake>     broken. It should be controlled per-file with attrs (I think 
the
stephen_leake>     only reason it isn't is that attrs didn't exist back then).
stephen_leake> 
stephen_leake> SummerOfCode/2007.mdwn
stephen_leake>     lists line-ending handling as a project
stephen_leake> 
stephen_leake> LineEndingMunging.mdwn discusses some of the issues, but not 
much on
stephen_leake> actual implementation.
stephen_leake> 
stephen_leake> I assume the current state is that there is no line munging code
stephen_leake> in mtn. 
stephen_leake> 
stephen_leake> Interestingly, there is no NEWS entry saying we've dropped 
line-ending
stephen_leake> handling.
stephen_leake> 
stephen_leake> 
stephen_leake> My scenario is:
stephen_leake> 
stephen_leake> I'm using Altera Quartus to compile VHDL files. For each VHDL 
project, I
stephen_leake> keep two files (beyond the VHDL files) in mtn: project.pin, 
project.qsf.
stephen_leake> I run on both RedHat Gnu/Linux and MS Windows.
stephen_leake> 
stephen_leake> project.pin contains the FPGA pin assignments; this lets me see 
when
stephen_leake> they change.
stephen_leake> 
stephen_leake> project.qsf contains various compiler settings. 
stephen_leake> 
stephen_leake> project.pin is written by Quartus, using OS-specific line 
endings. So
stephen_leake> when I rebuild on the other OS, 'mtn status' reports a bogus 
change;
stephen_leake> only the line endings have changed.
stephen_leake> 
stephen_leake> project.qsf is written by me, and by Quartus (because it assumes 
I'm
stephen_leake> using the GUI to edit options, even though I'm actually using 
Emacs).
stephen_leake> Again, 'mtn status' reports bogus changes.
stephen_leake> 
stephen_leake> I can invent an attr 'os_line_endings', and add a function to the
stephen_leake> 'attr_functions' hook to change the file line ending on workspace
stephen_leake> update. But that doesn't fix the bogus changes reported by 'mtn 
status'.
stephen_leake> In fact, it makes it worse; I only get the bogus changes now 
when I
stephen_leake> actually run Quartus. The same workspace contains Ada code, 
which I
stephen_leake> compile far more often. With this attr, I'll get bogus changes 
all the
stephen_leake> time.
stephen_leake> 
stephen_leake> Looking thru the code for 'mtn status', it seems I need to modify
stephen_leake> file_io.cc 'calculate_indent' to apply the reverse line ending
stephen_leake> transformation before computing the file id.
stephen_leake> 
stephen_leake> However, I assume adding a call to a Lua hook in 
'calculate_ident' will
stephen_leake> slow things down a lot. So we'd need to have a fast table lookup 
of
stephen_leake> which files need special treatment, or something. There will be 
very few
stephen_leake> files in my typical workspace that have this attribute.
stephen_leake> 
stephen_leake> One way to handle it would be to call the Lua hook to invert the 
file
stephen_leake> transform only if the file_id check fails. That would work in 
work.cc
stephen_leake> workspace::update_current_roster_from_filesystem; I'm not clear 
about
stephen_leake> other places where calculate_ident is used.
stephen_leake> 
stephen_leake> Thoughts?
stephen_leake> 
stephen_leake> -- 
stephen_leake> -- Stephe

-- 
Richard Levitte                         address@hidden
                                        http://richard.levitte.org/

"Life is a tremendous celebration - and I'm invited!"
-- from a friend's blog, translated from Swedish