[ENH] Add local Docker test server infrastructure #1615
+638
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements long-term solution for flaky tests and race conditions by introducing local Docker-based test infrastructure. ## Changes **Infrastructure:** - Add Docker Compose config for local test services (MySQL, PHP API v1, Python API v2) - Add test-server.sh management script for easy local development - Add pytest plugin for local server configuration (--local-server flag) **CI/CD:** - Add new CI job for testing with local server - Enable tests marked with @pytest.mark.uses_test_server to run locally - Eliminate dependency on remote test.openml.org for most tests **Documentation:** - Add comprehensive local_test_server.md guide - Update CONTRIBUTING.md with local testing instructions - Document migration path and troubleshooting steps ## Benefits ✅ Eliminates race conditions from parallel CI jobs ✅ Removes flaky failures from server load/timeouts ✅ Enables reliable local development and testing ✅ Provides foundation for v1→v2 API migration (openml#1575) ## Migration Path **Short-term:** Mock server implementation for CI validation **Mid-term:** Replace with official OpenML Docker images **Long-term:** Full local test environment with production-like data Fixes openml#1586 Co-authored-by: geetu040 (design from openml#1614)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements long-term solution for flaky tests and race conditions by introducing local Docker-based test infrastructure.
Changes
Infrastructure:
CI/CD:
Documentation:
Benefits
Migration Path
Short-term: Mock server implementation for CI validation
Mid-term: Replace with official OpenML Docker images
Long-term: Full local test environment with production-like data