-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Currently
16 def _write_starting_replication_value(self, context: Optional[dict]) -> None:
15 """Write the starting replication value, if available.
14
13 Args:
12 context: Stream partition or context dictionary.
11 """
10 value = None
9 state = self.get_context_state(context)
8
7 if self.replication_key:
6 replication_key_value = state.get("replication_key_value")
5 if replication_key_value and self.replication_key == state.get(
4 "replication_key"
3 ):
2 value = replication_key_value
1
321 elif "start_date" in self.config:
1 value = self.config["start_date"]
2
3 write_starting_replication_value(state, value)
From the SDK brings in start_date, which means techincally if you pass in a unix timstamp for start_date it will work. We don't advertise this as the "standard" for start_date is to use an RFC3399 datetime type
Options
- We could just override start_date's value with the timestamp value. right at the initialization of the tap.
- We could add special handling of get_starting_timestamp like https://github.com/AutoIDM/tap-clickup/pull/117/files#diff-0b811eae9421519f5a8fc873c70bd3e821c57d9e584c88de748d635e4808d1c5R279 , but we'd have to figure out when the initial state is coming from start_date and when it's coming from state as start_date is RFC3399 and state is storing the value as a unix timestamp
- Convert state unix time stamps to use RFC3399 values, and convert these values to timestamps when needed like here https://github.com/AutoIDM/tap-clickup/pull/117/files#diff-0b811eae9421519f5a8fc873c70bd3e821c57d9e584c88de748d635e4808d1c5R252
- Probably others?
Instead of figuring this out right now we're just not going to officially support start_date.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request