Re: [O] Bug: org-2ft and/or float-time is wrong [9.1.2 (9.1.2-22-ga2a034-elpaplus @ ~/.emacs.d/elpa/org-plus-contrib-20171023/)]

[Allen Li]

Apologies, my reply was partially omitted.  My full reply follows

On Wed, Nov 1, 2017 at 6:09 AM, Tim Cross <address@hidden> wrote:
>
> Allen Li <address@hidden> writes:
>
>> On Wed, Nov 1, 2017 at 12:18 AM, Tim Cross <address@hidden> wrote:
>>>
>>> My preferences would be
>>>
>>> 1. If a timestamp does not include the TZ, then assume the local TZ
>>> 2. If a timestamp does include the TZ, honour that TZ
>>
>> Org mode does not support TZ in time strings and adding support isn't
>> the topic of the current bug.
>>
>
> I disagree. The root cause of the bug is due to a lack of clarity
> regarding timezones. I also suspect this is also the basic cause of
> issues relating to the use of timestamps/time strings and daylight
> savings, especially when it comes to performing calculations etc.

Unless I missed something, Org mode does not support explicit
timezones in Org timestamps, and adding such support would take a
large effort [1].

[1]: https://lists.gnu.org/archive/html/emacs-orgmode/2011-04/msg00299.html

Unfortunately, I do not have time for the task.  Thus, Org mode
will have to make due with only Org timestamps without an
explicit timezone unless a savior appears.

>
>> I want to emphasize the distinction between timestamps and time
>> strings.  Timestamps are a
>> float value, Unix timestamps.  Time strings are Org mode specific,
>> like <2017-10-30 12:34:56>
>
> Org mode refers to what you are calling time strings as timestamps. In
> reality, there is no difference - one is just a numeric representation
> and the other is a string representation. It is good you have clarified
> your definition to reduce confusion. However, I think the problems are
> arising because of a lack of explicit TZ handling.

There's a pretty significant difference.  Unix timestamps are
standardized and used widely by most computers and Emacs itself.
Unix timestamps are time zone agnostics.

Org timestamps (time strings, whatever) are used exclusively by
Org mode.  As Org timestamps do not support explicit timezones,
they are not time zone agnostic.  The interpretation of an Org
timestamp depends on the time zone of the local machine, whereas
the interpretation of a Unix timestamp does not depend on the
time zone of the local machine.

>
>>
>> Timestamps do not have timezone information since they describe an
>> exact (well, minus leap seconds)
>> point in time, the number of seconds after the epoch.
>>
>
> A point in time is measured in number of seconds since epoch. However,
> if you want to use local (wall) time to display that point, you have to
> include TZ information when converting that number to a more
> meaningful/usable (for humans) format.

I'm not sure what you mean.  Time zone information is provided by
the local machine OS.  If I pass a Unix timestamp to Emacs/the OS
and tell it to format it in human readable format in the local
time zone (the default behavior), then it will be done, without
my having to attach time zone information anywhere.

> The point 'now' for me is UTC+1100
> and for you (based on your previous post) UTC-0700, so our
> representations in string format of this value will be different. Even
> on a single machine, it is also relevant. For example, if I have two org
> timestamps (your times strings) and I want to calculate the number of
> hours between the two, I need to include TZ information as one timestamp
> might be during daylight savings time and the other outside daylight
> savings time.

Ideally, except Org mode does not support including explicit TZ
information in Org timestamps.  Thus, Org mode can only current
interpret Org timestamps according to some blessed time zone.
Four months ago, that blessed time zone changed from the local
machine's time zone to UTC.

> Any calculation which fails to consider TZ information in
> this case will be out by an hour.
>
> If we just take the simple solution and ignore TZ information,

Org timestamps do not have TZ information.  It's not a matter of
not ignoring TZ information, support for TZ information would
have to be added, which as I understand is non-trivial.

> we will
> break calculations which cross daylight savings 'boarders'. I also
> suspect we will get inconsistent behaviour when integrating with other
> systems (such as external calendars, ical integration or sharing
> timestamp data with others etc).

As long as we use Unix timestamps when integrating with other
systems, there is no ambiguity.

>
>>> 3. If the timestamp does not include a time component, default to 0:00:0
>>
>> This is what Org mode does
>>
>>> for the local TZ
>>
>> This is what Org mode used to do, now it interprets it as 00:00:00 UTC.
>>
>>> 4. org-2ft should not enforce the UTC TZ. I agree this is incorrect.
>>>
>>> Rationale is that the user should have ability to fully control how
>>> their timestamps are represented. However, adding the TZ is probably
>>> just a pain for the majority of use cases, so defaulting to the local
>>> (wall) TZ is OK provided you can override this consistently by adding
>>> explicit TZ values.
>>>
>>> However, there is some devil in the details we need to work out. For
>>> example, should we support both TZ names (like AEDT or Australia/Sydney)
>>> and POSIX style +11/+11:00/+1100, should we add an option to tell org to
>>> always add TZ info in timestamps which include time components and if
>>> so, how complex will this need to be e.g. handle setting a future/past
>>> timestamp which is in a different (daylight savings) offset and what
>>> about the additional complexity in dealing with timestamp calculations
>>> where dates might be in different offsets due to daylight savings -
>>> while all quite possible, it does add significant complexity and this
>>> may have adverse impact on performance. Not to say we shouldn't do it,
>>> just that it will take significant work.
>>>
>>> I suspect just the first part won't have major impact - at least no more
>>> than enforcing UTC in org-2ft.
>>
>> Org mode does not support TZ in time strings currently.  While I would
>> like such a feature very much,
>> adding TZ support isn't the topic of the current bug, just fixing how
>> time strings are interpreted.
>>
>
> We could make the decision that all org timestamps are relative to the
> local TZ (ignoring the daylight savings issue) and that would probably
> be OK for most users

I agree, and think that this makes more sense than using UTC.

> but I feel the whole issue is due to a lack of
> adequate/consistent TZ interpretation.

Also agreed, but I cannot volunteer for the task of adding that
unfortunately.

> The problem with org-2ft is that
> it forces the conversion into UTC when the input time string may
> actually be in local time, resulting in a number which is out by
> whatever the local offset is.

Exactly

> When you then convert back, your string
> representation will be out by at least that amount (possibly more
> depending on whether TZ info is applied in the conversion back to a
> string).

TZ info is not applied in that way going back, since Unix
timestamps are time zone agnostic.  It may affect how the string
representation is formatted, but the time will not be even more
incorrect.

e.g., 2017-10-30T00:00:00+0000 vs 2017-10-30T07:00:00-0700.
Different strings due to the time zone used, but they both
describe the exact point of time as their input Unix timestamp.

>
> If it wasn't for the benefits of org files being essentially plain text
> files, the easiest solution would possibly be to represent timestamps as
> a number and use an overlay or similar to convert it to a human readable
> value based on whatever the user sets as the timezone for
> display/export. Unfortunately, this breaks one of the big benefits of
> org-mode over other solutions i.e. the data is just plain text and can
> be accessed/used by anything able to read plain text.
>
> Tim
>
>
>>>
>>> Tim
>>>
>>> P.S. when you start to think about it, it is easy to see how Java
>>> screwed up this stuff so badly!
>
>
> --
> Tim Cross