Re: date conversion from locale-specific format to any other date-format

[Bob Proulx]

Derek Ashley Thomas wrote:
> I have a string with a custom date format written in Japanese: 2013年
> 1月8日20時19分. I wish to convert this string to any other format
> using date. I expected to use date -d "2013年1月8日 20時19分" +"%F
> %R" but this produces the wrong date 2013-01-08 20:13 because it
> sees the 2013 and converts it to hours-minutes (not year) and then
> forgets the rest and assumes "today".

There are multiple issues.  At one level the locale specific date
formats are not universially recognized.  Date won't know what to do
with the Japanese year, month, day symbols.  Sorry.

The next issue is that even without the localization issue that no
time was specified.  That is problematic.  If no time is specified
then 00:00 midnight is assumed.  This often runs into daylight saving
time problems.  Better to avoid it and either use UTC or specify 12:00
noon instead.

> When I updated my machine to 'coreutils 8.20', the same command produces
> the following error invalid date ‘2013年1月8日 20時19分’. While perhaps the
> error is best since the date produced originally was wrong,

Yes.  That was the intention.  Previously invalid dates did not
produce correct results and resulted in other bugs.

> is there any way using the date command to convert from a
> custom/locale format to any other format?

Unfortunately no.  It is possible to convert from standardized date
formats from one timezone to another timezone.  And to produce
localized output.  But not to parse localized date formats.

> In osx, the date command can be used to specifically convert
> formats like so:
> 
> timestamp="2013年1月8日 20時19分"
> date -j -f "%Y年%m月%d日 %H時%M分" "$timestamp" +"%F %R"

Sorry but no strptime(3) interface exists for GNU date.  If someone
were to put in the effort to produce such an interface I am sure the
feature would be considered.

At the moment the best I can suggest is to reduce the localization so
that "2013年1月8日 20時19分" would be "2013-01-08 20:19 +0900" which
can be parsed by GNU date.

Alternatively I suggest using Perl (or Python or Ruby) to do this
parsing.  For example:

  $ perl -MPOSIX::strptime -le 'my ($sec, $min, $hour, $mday, $mon, $year, 
$wday, $yday) = POSIX::strptime("$ARGV[0]","%Y-%m-%d 
%H:%M");$year+=1900;$mon+=1;printf("%04d-%02d-%02d 
%0d:%02d\n",$year,$mon,$mday,$hour,$min);' "2013-01-08 20:19"
  2013-01-08 20:19

And therefore:

  $ perl -MPOSIX::strptime -le 'my ($sec, $min, $hour, $mday, $mon, $year, 
$wday, $yday) = POSIX::strptime("$ARGV[0]","%Y年%m月%d日 
%H時%M分");$year+=1900;$mon+=1;printf("%04d-%02d-%02d 
%0d:%02d\n",$year,$mon,$mday,$hour,$min);' "2013年1月8日 20時19分"
  2013-01-08 20:19

And therefore:

  $ date -u -R -d "$(perl -MPOSIX::strptime -le 'my ($sec, $min, $hour, $mday, 
$mon, $year, $wday, $yday) = POSIX::strptime("$ARGV[0]","%Y年%m月%d日 
%H時%M分");$year+=1900;$mon+=1;printf("%04d-%02d-%02d 
%0d:%02d\n",$year,$mon,$mday,$hour,$min);' "2013年1月8日 20時19分")"
  Tue, 08 Jan 2013 20:19:00 +0000

You will probably not have POSIX::strptime installed by default and
will need to install the module first.

Bob