branch master updated: More documentation/comments on binary strings, and decoding

[Patrice Dumas]

This is an automated email from the git hooks/post-receive script.

pertusus pushed a commit to branch master
in repository texinfo.

The following commit(s) were added to refs/heads/master by this push:
     new 1130c90f47 More documentation/comments on binary strings, and decoding
1130c90f47 is described below

commit 1130c90f4763190629632060de85ae297c1f6d25
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Sat Apr 16 23:41:49 2022 +0200

    More documentation/comments on binary strings, and decoding
---
 doc/customization_api.texi | 38 +++++++++++++++++++++++++++++++++-----
 tp/TODO                    | 13 -------------
 tp/Texinfo/Convert/HTML.pm |  2 +-
 tp/texi2any.pl             |  3 +++
 4 files changed, 37 insertions(+), 19 deletions(-)

diff --git a/doc/customization_api.texi b/doc/customization_api.texi
index 88c2b1caa6..2d6842b4b9 100644
--- a/doc/customization_api.texi
+++ b/doc/customization_api.texi
@@ -1302,10 +1302,14 @@ documentation, in particular a list of types and of 
information in the elements
 @code{extra} hash (@pxref{Texinfo::Parser TEXINFO TREE,,,tp_api}).
 
 
-@node Encoding file path strings
-@section Encoding file path strings
+@node Encoding and Decoding File Path Strings
+@section Encoding and Decoding File Path Strings
 
-The strings in the customization functions are character strings.  For most 
purposes,
+
+@node Encoding File Path Strings
+@subsection Encoding File Path Strings
+
+In general, the strings in the customization functions are character strings.  
For most purposes,
 this is right, and the encoding in output files is taken care of by the 
converter.
 Operations on directories and file names, however, such as the creation of a 
directory
 or the opening of a file require binary strings.
@@ -1359,6 +1363,26 @@ if (system($encoded_call)) @{
 @}
 @end example
 
+
+@node Decoding File Path Strings
+@subsection Decoding File Path Strings
+
+The binary strings that could be accessed correspond to the customization
+variables strings or arrays @code{INCLUDE_DIRECTORIES}, @code{CSS_FILES},
+@code{MACRO_EXPAND} and @code{INTERNAL_LINKS}.  If they need to be decoded into
+character strings, for example to appear in error messages, it is possible to
+use the @code{DATA_INPUT_ENCODING_NAME} customization variable value as
+encoding name to mimic how the decoding of these strings from the command line
+is done in the main program and in the converters.  For example:
+
+@example
+my $macro_expand_fname = $self->get_conf('MACRO_EXPAND');
+my $encoding = $self->get_conf('DATA_INPUT_ENCODING_NAME');
+if (defined($encoding)) @{
+  $macro_expand_fname = Encode::decode($encoding, $macro_expand_fname);
+@}
+@end example
+
 More information on perl and encodings in
 @uref{https://perldoc.perl.org/perlunifaq, perlunifaq}.
 
@@ -1502,10 +1526,14 @@ put into comments (@pxref{@code{@@copying},,, texinfo, 
Texinfo}).
 The file name of the current document unit being converted.
 
 @item destination_directory
-Destination directory for the output files.
+Destination directory for the output files.  It is common to use that string 
in directory
+or file paths with functions requiring binary strings.  In that case the
+character string needs to be encoded.  @xref{Encoding File Path Strings}.
 
 @item document_name
-Base name of the document.
+Base name of the document.  It is common to use that string in in directory or
+file paths with functions requiring binary strings.  In that case the character
+string needs to be encoded.  @xref{Encoding File Path Strings}.
 
 @item documentdescription_string
 @code{@@documentdescription} argument converted in a string context
diff --git a/tp/TODO b/tp/TODO
index ffbea7b9f5..7a5a532062 100644
--- a/tp/TODO
+++ b/tp/TODO
@@ -33,19 +33,6 @@ and @ref{Inserting Accents}.
 Texinfo input manual encoding?
 
 
-bytes.  document more somewhere?
-* texi2any.pl
- @input_files = @ARGV
-  $input_file_arg
- @include_dirs, 'INCLUDE_DIRECTORIES'
- @css_files, CSS_FILES
- MACRO_EXPAND
- INTERNAL_LINKS
-* Parser:
- $self->{'parser_info'}->{'input_file_name'}
- $self->{'info'}->{'input_directory'}
-
-
 Bugs
 ====
 
diff --git a/tp/Texinfo/Convert/HTML.pm b/tp/Texinfo/Convert/HTML.pm
index 457848dbd6..d28ce497e5 100644
--- a/tp/Texinfo/Convert/HTML.pm
+++ b/tp/Texinfo/Convert/HTML.pm
@@ -50,7 +50,7 @@ use File::Copy qw(copy);
 
 use Storable;
 
-use Encode qw(find_encoding);
+use Encode qw(find_encoding decode);
 
 use Texinfo::Common;
 use Texinfo::Config;
diff --git a/tp/texi2any.pl b/tp/texi2any.pl
index 6e24d33886..972a831aca 100755
--- a/tp/texi2any.pl
+++ b/tp/texi2any.pl
@@ -437,6 +437,8 @@ my @css_files = ();
 my @css_refs = ();
 my @include_dirs = ();
 my @expanded_formats = ();
+# note that CSS_FILES and INCLUDE_DIRECTORIES are not decoded when
+# read from the command line and should be binary strings
 my $cmdline_options = { 'CSS_FILES' => \@css_files,
                         'CSS_REFS' => \@css_refs,
                         'INCLUDE_DIRECTORIES' => \@include_dirs,
@@ -1228,6 +1230,7 @@ if (get_conf('SHOW_BUILTIN_CSS_RULES')) {
 
 # Main processing, process all the files given on the command line
 
+# Note that the input file names are binary strings and are not decoded
 my @input_files = @ARGV;
 # use STDIN if not a tty, like makeinfo does
 @input_files = ('-') if (!scalar(@input_files) and !-t STDIN);