Re: [POLL] Proposed syntax for timestamps with time zone info (was: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda)

[Tim Cross]

Ihor Radchenko <yantar92@posteo.net> writes:

> Greg Minshall <minshall@umich.edu> writes:
>
>> just a thought/reminder.  there are "semantics" and "encoding".  a spec
>> like ISO-8601 specifies both.  the important thing for org-mode is to
>> use an encoding that
>>
>> 1. is easily parsable/understandable by the mere mortal
>>
>> 2. allows expression of all the semantics of the underlying spec/specs
>>    (be that ISO-8601, this new IETF spec, the Library of Congress spec,
>>    etc.)
>>
>> 3. and, importantly, is designed to *try* to follow updates to the
>>    underlying spec/specs (which will inevitably happen)
>
> I agree with these three points.
>
> Following the previous discussion and the various links provided, I have
> reviewed the main discussed timestamp standards and would like to
> propose the new Org timestamp syntax that will allow specifying time
> zone information.
>
> I will not follow the standards fully because the available standards
> are not designed to be easily understood by humans. I will also omit
> the ideas from the standards that are unrelated to time stamps (but
> still outline them below, and keep them in mind for
> forward-compatibility). I will, however, try to use the syntax close to
> the standards where possible.
>
> 1. https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/
>    proposal is extending ISO8601/RFC3339 with time zone information. In
>    addition to UTC offset defined in ISO8601, it allows specifying the
>    time zone identifier and auxiliary information.
>
>    Examples:
>
>    2022-07-08T02:14:07+02:00[Europe/Paris]
>    (both offset, and time zone are specified)
>    Note that we cannot use "[]" symbols because they are incompatible
>    with current timestamp syntax that must not contain closing "]>"
>    inside the timestamp.
>
>    1996-12-19T16:39:57-08:00[America/Los_Angeles][u-ca=hebrew]
>    (preferred calendar is specified in addition to time zone)
>    Note: calendar spec is out of scope of time zone discussion - if we
>    decide to add it in future, we can simply add new parts to
>    timestamps, just like repeater interval and warning period.
>
>    Further, the draft proposes an idea, which have also been discussed
>    in this thread: making use of redundant UTC offset + time zone
>    information to detect possible unexpected changes in time zone rules:
>
>    2022-07-08T00:14:07+00:00[!Europe/London]
>    ("!" identifies that +00:00 offset, if not consistent with
>    Europe/London at the timestamp time, must be signalled to user or
>    generally not ignored)
>
> 2. https://www.loc.gov/standards/datetime/ does not contain any new
>    ideas relevant to time zones, although it has many other ideas wrt
>    approximate/incomplete timestamps. I will outline them later, but
>    they do not directly affect the proposed new Org timestamp syntax.
>    
> |-----------------------------------|
> | The proposed new timestamp syntax |
> |-----------------------------------|
>
> I propose two forms of time zone information in Org timestamps
>
> 1. Timestamp with explicit UTC offset
>
>    YYYY-MM-DD [optional day name] HH:MM[^ \]>]*?[+-−]HH[MM]?
>    YYYY-MM-DD [optional day name] HH:MM[^ \]>]*?Z[ \]>]
>    
>    "-" is latin "HYPHEN-MINUS" (code 0x2D)
>    "−" is unicode "MINUS SIGN" (code 0x2212), as prescribed by ISO8601
>    we will not actually use it when generating timestamps, but allow it
>    to keep some compatibility with ISO standard.
>
>    "Z" in the second format refers to "Zulu" time (another name for UTC)
>    and must be either the last character in the timestamp or the
>    last character before space. Not sure if many users are familiar with
>    "Z" convention, but it is (1) in ISO; (2) succinct for users who are
>    familiar with it. We will prefer +00 number offset in auto-generated
>    timestamps.
>
>    Examples:
>    2022-11-12 12:00+02 # 12:00 UTC+2
>    2022-11-12 14:00+0230 # 14:00 UTC+2:30
>    2022-11-12 14:00-0230 # 14:00 UTC-2:30
>    2022-11-12 14:00Z # 14:00 UTC
>
>    The offset is a subset of what is defined by ISO8601.
>    
>    Note that unlike ISO8601, ":" is not allowed in the offset specifier.
>    This is to disambiguate with the time intervals in Org timestamps:
>    [2022-11-12 8:00-9:00] in Org is a time range from 8am to 9am.
>
>    For time ranges, we will only allow a single offset and time zone
>    specifier for both start and end times:
>    [2022-11-12 8:00-9:00+08]
>    If different time zones are necessary to specify the start/end times,
>    users can still use full timestamp range syntax
>    [2022-11-12 8:00+03]--[2022-11-12 22:00+08]
>
>    The format is also forward-compatible with extending Org timestamps
>    for second/sub-second precision: 2022-11-12 14:00:05.0012+0230 will
>    remain valid if we decide to adopt such format.
>
> 2. Timestamp with time zone specifier
>
>    YYYY-MM-DD [optional day name] HH:MM[^ ]* @[!]?<[^ \]>]>
>
>    For now, time zone name will only be processed when it follows TZ
>    POSIX format. If necessary, the proposed syntax will still allow
>    extensions to TZ POSIX.
>
>    Examples:
>    2022-11-12 12:00 @Asia/Singapore # tzdb syntax
>    2022-11-12 10:00 @Europe/Berlin
>    2022-11-12 10:00 @!Europe/Berlin # "!" does nothing here, see below
>    2022-11-12 10:00 @EST+5 # TZ syntax
>    2022-11-12 10:00 @EST+5EDT,M3.2.0/2,M11.1.0/2 # manual time zone spec 
> allowed in TZ
>
>    The "@" symbol is selected to disambiguate time zone specifier from
>    other auxiliary information in the timestamp. Like calendar name,
>    which might be added in future. Note that we cannot use [...] from
>    the standard draft. I selected "@" because it is read as "at" -
>    location specifier.
>
>    The "!" symbol is adapted from
>    https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/
>
>    I use space before "@" to improve readability. We deviate from the
>    standard here so may as well. In contrast, no space before [+-]offset
>    is closer to the standard yet not cluttering the timestamp too much
>    (IMHO).
>
> 3. (1) and (2) can be combined
>
>    2022-11-12 12:00+08 @Asia/Singapore
>
>    Org will unconditionally use +08 offset and ignore the time zone
>    name. We prefer absolute offset because it is non-ambiguous wrt
>    out-of-date tzdb on the computer. One may also not update the TZ
>    variable upon travelling - the system clock will again be more likely
>    accurate.
>
>    This redundant time offset info can serve as human-readable
>    indication of absolute time, while the time zone name will indicate
>    the location.
>
>    2022-11-12 12:00+07 @!Asia/Singapore
>
>    Org will calculate the internal time for both +07 offset and
>    Asia/Singapore time zone. If they do not match, Org will issue a
>    warning. The offset will still be preferred if we have to proceed.
>
>    This can be useful when planning future meetings and sending Org file
>    with a timestamp to someone else, potentially with obsolete tzdb.
>
> |-----------------------------------|
> |              <end>                |
> |-----------------------------------|
>
> Apart from the ideas mentioned above,
> https://www.loc.gov/standards/datetime/ contains a number of other
> interesting ideas that may or may not be adapted by Org in future.
> I will outline the ideas I find noteworthy to keep them in mind when
> considering changing (including current changes) in the timestamp
> syntax:
>
> 1. Reduced timestamp precision:
>    1985-04-12 (day precision, time omitted; available in Org)
>    1985-04 (month precision, day and time omitted, not available in Org)
>    1985 (year precision)
>
>    Current timestamp syntax proposal should not interfere.
>
> 2. Using "T" as date/time delimiter
>    1985-04-12T23:20:30 (not supported by Org)
>
> 3. Using "/" for time intervals
>    2004-02-01/2005-02-08 (not supported and incompatible with Org)
>
> 4. Seasons
>    2001-21 (Spring, 2001; not supported by Org)
>
> 5. Approximate dates
>    1984? (year uncertain)
>    2004-06~ (year-month approximate)
>    2004-06-11% (entire date (year-month-day) uncertain and approximate)
>    2004-06?-11 (year and month uncertain)
>    2004-?06-11 (just month uncertain)
>
> 6. Unspecified digits
>    1985-04-XX (day unspecified; might be an interesting idea for
>    defining repeater intervals)
>
> 7. Open time intervals
>    1985-04-12/.. (from 1985-04-12 to infinite)
>    1985-04-12/ (from 1985-04-12 to unknown)
>
> 8. Negative calendar year
>    -1985 (note: an alternative could be allowing AD/BC)
>
> 9. Set of dates
>    [1667,1668,1670..1672] (one of)
>    {1667,1668,1670..1672} (all)
>    [1760-01,1760-02,1760-12..]
>    (similar to regexp [...] groups; not compatible with Org syntax, but
>    the idea could be interesting for repeater intervals)


I think that syntax looks pretty good. It seems to support all the
identified use cases and would therefore provide the flexibility users
require.

The big challenge will be in implementing a UI which encapsulates this
flexibility, but it intuitive and guides users to selecting the correct
format for the specific use case of the timestamp. We may also need
additional options to manage how timestamps are exported to various
backends (I'm assuming we will also have some form of overlay to improve
display of timestamps - it is likely users will also want similar
control over how timestamps appear in various back ends)