auctex-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [AUCTeX-devel] files with non-ascii characters and LaTeX as in TeXLi


From: jfbu
Subject: Re: [AUCTeX-devel] files with non-ascii characters and LaTeX as in TeXLive 2018
Date: Thu, 24 May 2018 12:04:07 +0200
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:52.0) Gecko/20100101 Thunderbird/52.8.0

Hi again (!!!)

better approach using the format with & syntax at bottom


Le 24/05/2018 à 11:52, jfbu a écrit :
Hi Keita

ARRGH filenames with multiple contiguous spaces are a challenge

Because TeX coalesces multiples spaces into a single one.

The ini file can be modified simply but it is more complicated
for the generation of the format itself.

See at bottom of post.

Le 24/05/2018 à 10:46, jfbu a écrit :
Hi Keita

Le 24/05/2018 à 09:24, Ikumi Keita a écrit :
Hi Jean,

Ikumi Keita <address@hidden> writes:
Thank you very much for your all valuable works!  I'll test this with
other cases (TL2017, file names with space, on w32 platform) and propose
to commit again.

Unfortunately, it seems that the proposed method using /AUCTEXINPUT does
not work with file name with space.  The sample I'm using is a file named
"ppp qqq.tex" with the content:



Yes, I had a bad night tormented about this. But I understand why
and will explain below.



------- ppp qqq.tex --------------------------------------------------
\documentclass{article}
\begin{document}
Hello, World
\(xyz\)
\end{document}

%%% Local Variables:
%%% TeX-master: t
%%% End:
----------------------------------------------------------------------

The format file is generated successfully:
----------------------------------------------------------------------
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "&pdflatex" prv_ppp_qqq.ini    
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX)
  restricted \write18 enabled.
entering extended mode
(./prv_ppp_qqq.ini
LaTeX2e <2018-04-01> patch level 4
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
No auxiliary output files.

(./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file prv_ppp_qqq.fmt
  (preloaded format=prv_ppp_qqq 2018.5.24)
7585 strings of total length 123256
58367 memory locations dumped; current usage is 219&56905
4191 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/5=cmex10
[...]
\font\OT1/cmss/m/n/10=cmss10
5716 words of font info for 21 preloaded fonts
1141 hyphenation exceptions
Hyphenation trie of length 370691 has 8997 ops out of 35111
   143 for language 83
   110 for language 82
[...]
   181 for language 0
0 words of pdfTeX memory
0 indirect objects
No pages of output.
Transcript written on prv_ppp_qqq.log.

TeX Output finished at Thu May 24 16:09:49
----------------------------------------------------------------------

However, the next run with the dumped format doesn't make its way:


Because spaces are ignored in the filename, due to \input form (indirectly via 
/AUCTEXINPUT)

I will explain below.

----------------------------------------------------------------------
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
\&prv_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
         .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
         .

! LaTeX Error: File `"pppqqq.tex"' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: tex")

Enter file name:
! Emergency stop.
<read *>
<*> &prv_ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"}
                                             ^^M
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Thu May 24 16:11:23
LaTeX: LaTeX found no preview images
----------------------------------------------------------------------

When I replace \&prv_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the
error changes slightly:

That's a bit unexpected (by me).

----------------------------------------------------------------------
[bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ 
qqq.tex\"}
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=prv_ppp_qqq.fmt)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
         .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
         .
)
Runaway argument?
! File ended while scanning use of ^^M.
<inserted text>
                 \par
<*> "/AUCTEXINPUT{ppp qqq.tex}
                               "


I do not understand that for time being
but let's forget it as we have the \&prv_ppp_qqq.fmt way.

UPDATE: at bottom of this my new proposal seems ok with -fmt
at least at my locale.


! LaTeX Error: Missing \begin{document}.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
  ...
<*> "/AUCTEXINPUT{ppp qqq.tex}
                               "

! LaTeX Error: File `ppp qqq.tex' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: tex)

Enter file name:
! Emergency stop.
<read *>
<*> "/AUCTEXINPUT{ppp qqq.tex}
                               "^^M
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on .log.
----------------------------------------------------------------------

I'm currently stuck up with this situation.  It's hard to cope with
unusual file names...


Ok, let's not despair.

The key is in the usage of mylatex.ltx which is hidden in the "ini" file
for creating the format.

When one runs preview on "ppp qqq.tex" with stock AUCTeX one sees this in log

Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "&pdflatex" 
prv_ppp_qqq.ini    
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" \"ppp\ qqq.tex\"''

[lines cut]

Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
"&prv_ppp_qqq" \"ppp\ qqq.tex\"''


The first one generates the format and uses \input. In fact mylatex.ltx ends 
with
an \input, but AUCTeX cancels this by \relax, (relying on existence of file 
.tex,
because TeX always adds .tex suffix, here to file with empty name delimited
by \relax). The ini file contains indeed

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 
\pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax

It contains a hack of \dump. This \dump will be inserted by the
\begin{document} of the file whose preamble is cached: indeed \document
macro is redefined to do things ending with \dump. Right before the
\dump, it sets the catcode of space to "ignore". The hack resets
the catcode of space to "normal space" right before doing the \dump.

So when format is loaded in

Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
"&prv_ppp_qqq" \"ppp\ qqq.tex\"''


space is initially normal. I say initially, because \everyjob resets it
to "ignore", which is part of the tricks mylatex.ltx do for the preamble
to now be skipped (it is already in format). And \everyjob is executed
at the first encountered macro. There is no macro here because the file
is loaded by filename not by \input.

In short the AUCTeX hack to allow filenames with spaces here fails
automatically when one uses \input form or any TeX macro.

And /AUCTEXINPUT{file name} precisely is a TeX macro because my hack
assigned temporarily catcode "control sequence escape" to the / forward
slash.

In short, in presence of the /AUCTEXINPUT hack, the original space hack
simply ceases functioning because the space catcode is reset to "ignore"
too soon (at \everyjob)

How to fix this ?

With this new version of ini file

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%


Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b<space>123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.

I have also added quotes inside the \detokenize.

I tested with a file with filename "test mylátex.tex"

I. Generation of format

etex -ini "&pdflatex" prv_test_mylátex.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input{\detokenize{\"test mylátex.tex\"}}"

where file prv_test_mylátex.ini contains the above contents (from 
\ifx\pdfoutput to \relax%)
and attention to the added quotes inside the \detokenize argument.

II. Usage of format

pdflatex -fmt="prv_test_mylátex.fmt" /AUCTEXINPUT{test mylátex.tex}

Could you try this out ?


The above (step I) would have a problem with filenames containing multiple 
spaces
because \input{\detokenize{\"test    mylátex.tex\"}} is no different
to TeX than \input{\detokenize{\"test mylátex.tex\"}} when space is normal.
(which is the case when the format is generated with the help of mylatex.ltx)

The Step II also has a problem with filename having multiple contiguous spaces.
I initially thought it would be easy to fixed by configuring spaces to be
with catcode 12, but it seems some interference from pdftex scanning of
command arguments could not be overcomed by this only.

So I have found another approach which instead of making / of catcode 0
makes the " of catcode zero.

Let's illustrate this with file "test    mylátex.tex" with four spaces.

Contents of ini file

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 12 %
\catcode`\" 0 %
\def\AUCTEXINPUT##1{\catcode`\" 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%

Generation of format

etex -ini "&pdflatex" prv_test____mylátex.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "{\catcode32 12 \edef\x{\noexpand\input{\detokenize{\"test    mylátex.tex\"}}}\expandafter}\x"

Usage of format

pdflatex -fmt="prv_test____mylátex.fmt" "AUCTEXINPUT{test    mylátex.tex}"


I am sorry if this all looks a bit complicated.

there are many elements:

catcodes from mylatex.ltx

handling of contiguous spaces by TeX

mysteries of command line parsing by pdftex

etc...

I anticipate difficulties with Windows due to different quoting.

It is a bit of a mystery to me why exactly

pdflatex -fmt="prv_test____mylátex.fmt" "AUCTEXINPUT{test    mylátex.tex}"

works: the second " is apparently correctly discarded, but the first one
is correctly acting as catcode 0 to make a control sequence \AUCTEXINPUT


The problem I am trying to solve in the above is to get the filename
correctly fetched with four spaces, but in a context however that the format
gives TeX very weird catcodes.

I am afraid the above solution may depend on Unix handling of ",
and I can't test on Window.


It appears to be better and simpler to use & input form for format

So let's start all over again

contents of ini file

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 12 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%


This is same as in my first proposal except that space will be of catcode 12 at 
the time of the input


generation of format

etex -ini "&pdflatex" prv_test____mylátex.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "{\catcode32 12 \edef\x{\noexpand\input{\detokenize{\"test    mylátex.tex\"}}}\expandafter}\x"


usage of format

pdflatex \&prv_test____mylátex "/AUCTEXINPUT{test    mylátex.tex}"


Works at my locale...

Best

Jean-François










reply via email to

[Prev in Thread] Current Thread [Next in Thread]