Skip to content

Conversation

@asmacdo
Copy link
Member

@asmacdo asmacdo commented Jan 31, 2026

Summary

  • Add flaky rerun backoff to test_execution_summary to handle slow CI runners (Mac, PyPy) where the short sleep time doesn't allow enough samples to be collected

Closes #384

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings January 31, 2026 17:10
@asmacdo asmacdo added the semver-tests Add or improve existing tests label Jan 31, 2026
@codecov
Copy link

codecov bot commented Jan 31, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.81%. Comparing base (3756590) to head (6e2c894).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #394   +/-   ##
=======================================
  Coverage   91.81%   91.81%           
=======================================
  Files          15       15           
  Lines        1112     1112           
  Branches      138      138           
=======================================
  Hits         1021     1021           
  Misses         69       69           
  Partials       22       22           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds retry backoff logic to the test_execution_summary test to handle flaky failures on slow CI runners (Mac, PyPy). The test was failing because insufficient samples were being collected before the monitored process exited, resulting in None values for execution summary metrics.

Changes:

  • Added @pytest.mark.flaky(reruns=5) decorator to enable automatic retries
  • Implemented backoff mechanism that scales sleep time and report interval based on retry attempt number
  • Added pytest request fixture to access retry attempt information

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-tests Add or improve existing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mac 15-intel pypy flake on test_execution_summary

1 participant