Skip to content

start_date implementation #118

@visch

Description

@visch

Currently

    16     def _write_starting_replication_value(self, context: Optional[dict]) -> None:
    15         """Write the starting replication value, if available.
    14
    13         Args:
    12             context: Stream partition or context dictionary.
    11         """
    10         value = None
     9         state = self.get_context_state(context)
     8
     7         if self.replication_key:
     6             replication_key_value = state.get("replication_key_value")
     5             if replication_key_value and self.replication_key == state.get(
     4                 "replication_key"
     3             ):
     2                 value = replication_key_value
     1
  321              elif "start_date" in self.config:
     1                 value = self.config["start_date"]
     2
     3         write_starting_replication_value(state, value)

From the SDK brings in start_date, which means techincally if you pass in a unix timstamp for start_date it will work. We don't advertise this as the "standard" for start_date is to use an RFC3399 datetime type

Options

  1. We could just override start_date's value with the timestamp value. right at the initialization of the tap.
  2. We could add special handling of get_starting_timestamp like https://github.com/AutoIDM/tap-clickup/pull/117/files#diff-0b811eae9421519f5a8fc873c70bd3e821c57d9e584c88de748d635e4808d1c5R279 , but we'd have to figure out when the initial state is coming from start_date and when it's coming from state as start_date is RFC3399 and state is storing the value as a unix timestamp
  3. Convert state unix time stamps to use RFC3399 values, and convert these values to timestamps when needed like here https://github.com/AutoIDM/tap-clickup/pull/117/files#diff-0b811eae9421519f5a8fc873c70bd3e821c57d9e584c88de748d635e4808d1c5R252
  4. Probably others?

Instead of figuring this out right now we're just not going to officially support start_date.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions