Skip to content

Implement some sort of limit on how many part exports can be schedule in a given replica #1176

@arthurpassos

Description

@arthurpassos

AS of now, it is memory bound. The user can schedule as many as it wants, and depending on executor pool size, it will execute them.

This is a problem because it does not offer us any mechanism to prevent a single replica from locking ALL parts for a given export partition at once and leaving other replicas with nothing to do.

It is not as simple as checking the executor pool size because of TOCTOU. Maybe we need an extra layer aside from the executor pool?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions