feat: add multipart upload support with progress callbacks #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
dtfiedler
commented
Jan 21, 2026
dtfiedler
commented
- Add ChunkedUploader class for multipart upload lifecycle
- Add ChunkingParams for configurable chunk size (5-500 MiB), concurrency, and chunking mode (auto/force/disabled)
- Extend upload() with on_progress callback and chunking params
- Add custom upload_url/payment_url support to Turbo client
- Add TurboUploadStatus, ChunkedUploadInit types
- Add UnderfundedError, UploadValidationError, UploadFinalizationError
- Add unit tests for chunking logic (28 tests)
- Add performance benchmark tests (skipped by default)
- Update README with multipart upload documentation
- Add ChunkedUploader class for multipart upload lifecycle - Add ChunkingParams for configurable chunk size (5-500 MiB), concurrency, and chunking mode (auto/force/disabled) - Extend upload() with on_progress callback and chunking params - Add custom upload_url/payment_url support to Turbo client - Add TurboUploadStatus, ChunkedUploadInit types - Add UnderfundedError, UploadValidationError, UploadFinalizationError - Add unit tests for chunking logic (28 tests) - Add performance benchmark tests (skipped by default) - Update README with multipart upload documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add StreamingDataItem class that wraps data streams with DataItem headers - Add create_data_header() for building headers separately from data - Use sign_stream to compute signatures without loading entire file into memory - Update _upload_chunked() to use streaming by default for all inputs - Bytes inputs are wrapped in BytesIO for unified code path This enables uploading large files with constant memory usage regardless of file size. The stream is read twice: once for signing (computing the deep hash), and once for the actual upload. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.