How and why we use a custom representation for something like (Timestamp, Offset)

[Giovanni-Tably]

Hi!

As a follow up to #89 (comment), a few notes on how we are using the library:

• Context: the product this post refers to manages data for our user, and we don't necessarily have a lot of control over the state we get it in.
• In practice, in the wild you mostly see dates without timezone information, pure timestamps (often encoded with a UTC offset), and timestamps with offsets (but no timezone).
• On the last one, parsing a (Timestamp, Offset) into a Timestamp works, but will lose the offset. Parsing it into a Zoned using a fixed-offset timezone works, but will add 'semantic meaning'.
• Specifically, for some common operations:
  • Turn things into dates (for grouping, for example):
    • This can be done without additional user input for Zoned and (Timestamp, Offset).
    • This cannot be done for a Timestamp, as it's explicitly not 'tied' to any offset. It could be mapped to multiple dates.
    • So, if we were to parse a (Timestamp, Offset) into a Timestamp, we would lose useful info.
  • Add a week to the value (to define query ranges, for example):
    • This can be done without additional user input for Zoned.
    • This cannot be done for Timestamp and (Timestamp, Offset), as we don't know if a timezone transition would have happened.
    • So, if we were to parse a (Timestamp, Offset) into a Zoned, the type would be more flexible than we intended it to be.
• We ended up just using a custom representation for the values, and we convert it into a Zoned on the fly to perform operations we know to be correct.
  • This was straightforward, so I don't think it has been a huge issue for us.
  • We might have ended up doing this anyway, for flexibility and because we needed to also store things like offsets/timezones independently as constraints ('this column must have this timezone' and similar).
• This might add a some friction to the 'define type for random API that happens to use offsets' scenario, where someone might care about being precise.

Overall, I have felt the need for a (Timestamp, Offset) to capture the semantics of values in the wild precisely, but this is something that can also be done outside of jiff. Next time I need this I expect a lightweight custom wrapper to take very little to write.

[BurntSushi]

Thanks for sharing this! I really appreciate it.

Coming into this, I think my major question was "why can't you use Zoned for this?" And I think your answer is, basically, that if you use Zoned with a fixed offset time zone, then you also wind up opting into a host of operations that you specifically don't want to be valid. Such as "add 1 week." I think this is the crux of the issue, because I think Zoned with a fixed offset time zone is generally intended to be the preferred workflow here. So I have some follow-ups if you're willing:

• You mention you use a custom representation and convert to a Zoned on the fly. Can you custom representation instead just be Zoned and your wrapper exposes a more limited API that bans the things you don't want it to do?
• Do you actually add 1 week (or another calendar unit) at any point? If so, how do you do that when you only have an offset? (Or are you just saying that you don't want to be allowed, accidentally, to add 1 week?)
• While I'm not sure if this is a good idea, if the Zoned API returned errors when using calendar units when its time zone is a fixed offset, would that enable you to use Zoned?

[Giovanni-Tably]

And I think your answer is, basically, that if you use Zoned with a fixed offset time zone, then you also wind up opting into a host of operations that you specifically don't want to be valid.

Yep

I think Zoned with a fixed offset time zone is generally intended to be the preferred workflow here.

In the 'random API' scenario, for what it's worth, I kind of expect the 'default path' to be: use Zoned -> fail to parse (no tz) -> use Timestamp -> Offset lost. Though I don't remember the error there, that could help. Also interaction with the 'this is not an offset' values might be a bit tricky.

---

You mention you use a custom representation and convert to a Zoned on the fly. Can you custom representation instead just be Zoned and your wrapper exposes a more limited API that bans the things you don't want it to do?

In theory, yes. Though, in practice, once we are handling the various combinations (because in our case just a Timestamp and ~Zoned are also on the table), it becomes easier to keep the component parts and assemble them on the fly.
For the 'random API' use-case, the implementation details of the wrapper can be whatever makes sense, though keeping it Copy seems desirable.

Do you actually add 1 week (or another calendar unit) at any point? If so, how do you do that when you only have an offset? (Or are you just saying that you don't want to be allowed, accidentally, to add 1 week?)

(Sorry, product considerations ahead, but the situation is still blurry and it would be misleading to paint it otherwise.)

As of right now, no. But looking further:

• This is very much in development, and not all of our datetime+duration logic is ready.
• Specific examples are:
  • Various forms of queries like from {value} - 1 week to {value}, where {value} may be the latest timestamp in a column.
    Here exact UX is to be decided, but I think an ideal interface would fail without a timezone, with possibly an overlay that creates a removable conversion (Offset -> Tz(Offset)) at creation-time to avoid the user having to think about in the default case. Or possibly the magic happens during import so it's restricted to one place.
  • Various forms of triggers like do {action} at {last_timestamp} + 1 week.
    Any time there are side effects, I think we would want to enforce timezones.
• We have a general policy of 'do not mess with user data' which in this case would be reflected in them being able to distinguish between 'this is a (Timestamp, Offset)', and 'this a (Timestamp, Tz(Offset)' if they need to. Any 'simplification' would generally be on top of that so that it can be reversed. This does mean our use-case is probably weirdly picky about these things. (Hence the custom types!)

While I'm not sure if this is a good idea, if the Zoned API returned errors when using calendar units when its time zone is a fixed offset, would that enable you to use Zoned?

I think we will want the fallback of 'just-treat-this-as-a-timezone-pretty-please', and I suspect this is going to be even more true for jiff users that are not us, so I don't think this would solve the problem.

---

Additional note that came to mind: for simplicity, we basically decided to just bundle the tz db, and we didn't need custom timezones, so using our own types also allowed us to keep them all Copy.

---

At the end of the day though, I really want to emphasise that jiff provided the building blocks we needed, and I consider that successful. (Thank you!) So all of this is just me to elaborate on what intuitively feels right from a prospective of 'as strictly typed as possible, without loss of information'.

[BurntSushi]

Gotya, thank you for elaborating.

I do agree that this is a source of friction here. You're definitely not the only person I've heard this sort of thing from, but you've definitely provided the most details!

I think for now I'm not going to take any action items from your feedback since it seems like y'all landed on a reasonable solution. And I definitely don't want to over-correct by just jumping and adding something like a FixedOffsetTimestamp or something. (Which I think would be somewhat devastating from an API complexity standpoint. I think it would cause what I call "decision paralysis" where folks don't know what to use.)

Anyway, if you're able to come back and fill in more details once things have settled a bit more for you, that would be awesome. But don't feel obligated. :-)