RE: [cedet-semantic] Latest CEDET on BZR does not compile with emacs 24.1

[Vincent Belaïche]

Feedback inserted below:

> Date: Fri, 5 Oct 2012 10:10:38 +0200
> From: address@hidden
> Subject: Re: [cedet-semantic] Latest CEDET on BZR does not compile with emacs 24.1
> To: address@hidden
> CC: address@hidden; address@hidden; address@hidden
>
> > From: Vincent Belaïche <address@hidden>
> > CC: "address@hidden"
> > <address@hidden>, emacs-devel <address@hidden>
> > Date: Fri, 5 Oct 2012 07:18:53 +0200
> >
> > Thank you Eli for your kind answer. I tried it with the mingw32-make a link to which you have sent to me. It is correct that it does not use MSYS paths like /c/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el, but the same kind of path that pwd -W outputs, i.e. c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el for the same file. EMACS is supposed to understand that kind of path. However that still does not work:
>
> My crystal ball says that pwd is an MSYS program. IOW, you still have
> MSYS's /bin directory on your general PATH,

Your crystal ball is perfectly correct

> and those MSYS tools,
> especially Bash, are being invoked by the MinGW Make.
>
> If you are invoking Make from the MSYS Bash console window, don't;
> invoke it from a regular cmd window instead.

I tried both actually, but, true, MSYS's /bin was in the PATH in both cases

> If you added MSYS's /bin
> directory to your PATH such that even the cmd window gets that PATH,
> edit your environment variables and remove the MSYS /bin directory
> from the PATH. Then these problems will go away, and you will be
> faced only with real problems: the Unixy features used by the Makefile
> that don't work on Windows. These you will have to solve, but at
> least you will be fighting known issues.

Well, call this the "real problems" if you like, I am slightly sceptical on what the real problem really is, most Makefiles I have seen in my life are delegating some file manipulation tasks to some bash shell. Even the documentation of Make is most of the time doing this assumption --- e.g. see those `rm ...' in the documentation of `clean' targets.

The reason why so is that GNU make may have lots of thing in its sleeve, but, AFAIK, it does not have builtins to do basical file manipation functions. So it has to resort to some shell call, most of the time Bash. 

In contrast to GNU make, ant has the  filterset, delete, mkdir, and copy tasks. So one does not need to delegate those basic things to any shell, and build.xml files are made more portable this way.

Considering CEDET I tried what you said, I wrote an MSDOS batch file with this content (the first line ensures that I have both mingw32-make and emacs in the PATH, but no more :

------- begin -------

SET PATH=c:\msys\1.0\mingw\bin;c:\Programme\GNU\emacs-24.1\bin
SET RM=DEL

CD cedet
mingw32-make

CD ..

 ------- end ------- 

and that failed quite soon with this output:

------- begin -------

C:\Programme\GNU\installation\cedet-install>SET PATH=c:\msys\1.0\mingw\bin;c:\Programme\GNU\emacs-24.1\bin

C:\Programme\GNU\installation\cedet-install>SET RM=DEL

C:\Programme\GNU\installation\cedet-install>CD cedet

C:\Programme\GNU\installation\cedet-install\cedet>mingw32-make
Emacs version: 24.1.1 nil on windows-nt
Removing loaddefs.el files from subprojects.
-f était inattendu.
mingw32-make: *** [clean-autoloads] Error 255

C:\Programme\GNU\installation\cedet-install\cedet>CD ..
 ------- end ------- 

As far as I can understand this failure happened when this statement was met:

    @$(foreach proj,$(PROJECTS_AUTOLOADS),cd $(CURDIR)/$(proj) && if [ -f $(LOADDEFS) ];then $(RM) -f $(LOADDEFS);fi;)

Quite surprising DOS does not bark on `cd  $(CURDIR)/$(proj)' despite the `/' inside it, but the `if [ -f ... ]; ....; fi' instead of `IF EXIST ... ( ... )' was a bit too much asking of DOS / Bash similarity.

It just seems to me quite challenging to write Makefile in a portable way while --- sorry for repeating myself --- GNU Make is missing those tasks called filterset, delete, mkdir, and copy in ant.

GNU Make simply does not have these things built-in, this is why most Makefile delegate them to Bash, and one can see find, rm, mkdir and cp bash commands instead, which means that you get into troubles when you want to use this Makefile on a plaform without Bash.

Here we are talking on doing macro definition to have some portable tasks like  filterset, delete, mkdir, and copy in ant --- i.e. not the same macro definition in DOS and in a unixy shell, but that reasoning is quite unstatisfactory, because that means that every time you are considering a new shell you will need to create again such a set of macros --- or better, user defined function that one would $(call ...).

So, you may argue that the real problem is that Makefile call some bash commands, but I am more opinionated that the real problem is that GNU Make is missing the builtin features that would allow one to do without those bash command in the statements in a standard way, and without user defined macros/function to ensure portability.

 

>
> > Debugger entered--Lisp error: (file-error "Opening output file" "no such file or directory" "d:/devel/emacs/release/emacs24/emacs-24.1/lisp/c;c:msys.0ProgrammeGNUinstallationcedet-installcedetlispcedetloaddefs.el")
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> See this nonsense? That's MSYS file syntax conversion in action.

[... SNIP ...]
> 

Thank you again for your commenting on it: I must admit that I was not aware that MSYS was so invasive in preprocessing arguments like that, I could not even imagine it, I had thought that emacs was the one doing those kind of argument preprocessing.

That is all the more surprising that the string that is transformed is within some simple-quotes.

> > What happens is that path `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' in the command line transformed by EMACS into `c;c:\msys\1.0\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el'.
>
> Yes. This is what MSYS does, and it does that for a good reason. The
> problem is that the algorithms MSYS applies for this conversion
> misfire when the program being invoked has its own ideas about the
> syntax of the command-line arguments. This is one such case.
> (Invoking a natively-compiled Sed is another prominent case where the
> MSYS file-name conversion badly misfires.)

>

Ok, I see, unless if you do the 's/find-this/replace-by-that/g' with some other character than `/', e.g. like that 's!find-this!replace-by-that!g'  

But your comment is quite interesting: would that mean that if emacs was compiled to be an MSYS application then it would be able to prevent MSYS from doing mischiefs by telling to MSYS which argument is supposed to be a path and which is not ?

I mean that EMACS people boasting that EMACS is ported on most known platform seem not to consider MSYS as a platform on its own, i.e. it seems that EMACS Win32 port does not  test at all whether MSYSTEM environment variable is set to MSYS or to MINGW32, nor whether OSTYPE is set to msys ...

>
> > Probably the reason for doing that is quite valid: the path should have been ` c:\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el' in the first place.
>
> No. (Btw, using "D:/foo/bar" syntax of file names is perfectly OK on
> Windows, because the Windows file I/O APIs understand forward slashes
> very well, but that's not the issue here.) The problem is elsewhere:
> this command-line argument is _not_ a file name. It is a Lisp string
> that will be interpreted as a file name by Emacs. But MSYS does not
> (and cannot) have the slightest idea about valid syntax of Lisp
> strings; in particular, it doesn't know that a backslash in a Lisp
> string needs to be doubled. So it ruins the command line by its
> simplistic conversion.

> 
> >My thought are rather that is it hard to write makefile that are portable because GNU Make is lacking the following features:
> >
> >
> > - a predefined variable, e.g. $(FS) expanding to the file separator, i.e. \ for MSWindows and / for Unixy shell
>
> Why do you need this? As I said, Windows programs, Emacs included,
> understand forward slashes in file names quite well. The only one
> that might have problems is cmd.exe, but you should not need to invoke
> cmd.exe's commands too much, if at all.

Well, for instance, as far as I remember DEL is an MSDOS internal command, so at least for all the clean and such like targets --- which quite common in any Makefile --- you need those backslash and not forward-slash, if you assume that you have only MSDOS to rely on, and not MSYS in your path, or at least RM variable defined to MSYS's rm.exe

> And if you do need internal
> cmd commands, there are the 'subst' and 'patsubst' Make functions you
> can use to convert forward slashes to backslashes on the cmd command
> lines.
>

But then that kind of mean using the same sort of tricks to which I resorted to compile with MSYS. I was also doing those subst to have the backslashes in the first place and be sure that no problem would happen become some SW (be it MSYS or EMACS) would do some mistake in badly converting forward slashed to backward slashes

> Also, do you know that Make converts all environment variables to Make
> variables, and expands them?
>
> > - a predefined variable, e.g. $(CFS), expanding to the file separator within a C-string-like environment, i.e. \\ for MSWindows and / for Unixy shell
>
> Again, you should not need this at all. This problem does not exist,
> as long as you invoke Emacs from the Makefile.
>
> > - a function, e.g. $(pathconvert ...) to convert a path name in the the Unixy like format that make internally uses with wildcards and $(abspath ...) returned value into the format used on the platform where make is running, i.e. in my case `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' would be converted to `c:\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el'
>
> Not needed, see above.
>
> > - and a function, e.g. $(cpathconvert ...) which would do the same thing but with output to go within some C-string-like environement, i.e. in my case
> > `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' would be converted to `c:\\Programme\\GNU\\installation\\cedet-install\\cedet\\lisp\\cedet\\loaddefs.el'
>
> Again, not needed.
>
> > Without those basic things, one has no other resorts than the kind of tricks which I played to make the Makefile platform independent, and which, I agree, may be fragile, though working ones.
>
> You will have to explain why you at all need these basic things.
>
> > Please note that this is why I mentioned ant: in contrast to GNU Make, ant has that sort of tricks that allow to do path name manipulations.
>
> GNU Make has enough up its sleeve for this job, and many similar ones,
> believe me.
>

I would be more than quite happy to believe you, but it seems that I don't have enough expertize in GNU make for having this belief. And, this lack of expertize --- of lack of things up the sleeve of GNU make, let us see --- is not just mine: see it, CEDET's Makefile's are not portable to DOS according to your criterion that the real problem is the use of Bash commands. E.g. CEDET Makefile's call bash specific command `if [ -f .... ]; then ... fi'. But surely CEDET people who created EDE know far more about writing Makefiles than I do. However CEDET's Makefile could be portable to Win32 if you take it for granted that GNU Make cannot leave without some Unixy shell nearby --- so MSYS under MSWindow ---, and therefore that the only remaining issue is path manipulation. Hence the FS, CFS, pathconvert and cpathconvert which I was suggesting.

VBR,

   Vincent.

>
>