Skip to content

Conversation

@kbuma
Copy link
Collaborator

@kbuma kbuma commented Dec 19, 2025

Summary

Major changes:

  • remove parallelization of server requests
  • re-implement handling of list criteria for parameters in endpoints that do not accept lists (this was tied into the parallelization code previously)
  • re-implement handling of list criteria that is too large for a single request (this was tied into the parallelization code previously)

slice_size = num_params_min_chunk or 1
# If successful, continue with normal pagination
total_data = {"data": []} # type: dict
total_data["data"].extend(data["data"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should favor .append(...) w/itertools.chain.from_iterable(...) at the end rather than repeated calls to .extend (especially since there is a loop later).

lines: 656, 701, 732, 806

Comment on lines +685 to +686
for i in range(0, len(split_values), batch_size):
batch = split_values[i : i + batch_size]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be a ways off for being the minimum py version, but in 3.12 itertools introduced batched. I've used the approximate implementation from the docs before:

def batched(iterable, n, *, strict=False):
    # batched('ABCDEFG', 2) → AB CD EF G
    if n < 1:
        raise ValueError('n must be at least one')
    iterator = iter(iterable)
    while batch := tuple(islice(iterator, n)):
        if strict and len(batch) != n:
            raise ValueError('batched(): incomplete batch')
        yield batch

Copy link
Collaborator

@tsmathis tsmathis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really much to say on my end, I am be curious though about the performance/execution time of this implementation vs. the parallel approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants