Skip to content

Conversation

@Felixoid
Copy link
Contributor

@Felixoid Felixoid commented Nov 24, 2025

Unfortunately, the current code doesn't cover many edge cases for large/old PRs and issues. Here are the cases that I address in the PR:

  • The page size in GitHub is 30, so only 30 comments are fetched
  • Pagination is not used at all
  • If a PR/issue was updated after until, it will be filtered out by the first search query
  • The since parameter isn't used. For huge issues/PRs, it helps to reduce the number of requests significantly
> gh api 'repos/ClickHouse/ClickHouse/issues/90542/comments?per_page=100&since=2025-11-20T00:00:00' | jq '.[] | .created_at'
"2025-11-20T23:55:00Z"
"2025-11-21T00:12:32Z"
"2025-11-21T09:56:12Z"
> gh api 'repos/ClickHouse/ClickHouse/issues/90542/comments?per_page=100&since=2025-11-21T00:00:00' | jq '.[] | .created_at'
"2025-11-21T00:12:32Z"
"2025-11-21T09:56:12Z"

@Felixoid Felixoid force-pushed the fix-github-commented branch 5 times, most recently from c06a3b6 to d28a3fc Compare November 24, 2025 14:28
@Felixoid
Copy link
Contributor Author

Felixoid commented Nov 27, 2025

Dear @sandrobonazzola, @kwk, and @psss, did you have time to review this improvement?

Copy link
Collaborator

@sandrobonazzola sandrobonazzola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some inline comments

@Felixoid
Copy link
Contributor Author

Thanks for the review, it's ready for the next round.

@Felixoid Felixoid force-pushed the fix-github-commented branch 2 times, most recently from 01e1907 to d666b8a Compare November 28, 2025 10:06
@Felixoid
Copy link
Contributor Author

Felixoid commented Nov 28, 2025

Here's an example of executed did from 8beb05a vs from this branch:

Details
# master
> did last year --format=markdown --brief
Status report for the last year (2024-01-01 to 2024-12-31)
==========================================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Mikhail f. Shiryaev <mr.felixoid@gmail.com>
Mikhail f. Shiryaev <felixoid@clickhouse.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Issues created on GitHub: 32
* Issues commented on GitHub: 84 <-- 23 less issues
* Issues closed on GitHub: 30
* Pull requests created on GitHub: 223
* Pull requests commented on GitHub: 266 <-- 21 less PRs
* Pull requests closed on GitHub: 655
* Pull requests reviewed on GitHub: 92
Time: 2 min. 22.77 s.

# pull request
> did last year --format=markdown --brief
Status report for the last year (2024-01-01 to 2024-12-31)
==========================================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Mikhail f. Shiryaev <mr.felixoid@gmail.com>
Mikhail f. Shiryaev <felixoid@clickhouse.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Issues created on GitHub: 32
* Issues commented on GitHub: 107
* Issues closed on GitHub: 30
* Pull requests created on GitHub: 223
* Pull requests commented on GitHub: 287
* Pull requests closed on GitHub: 655
* Pull requests reviewed on GitHub: 92
Time: 4 min. 35.04 s.

It works longer, but finds more commented issues and PRs

@Felixoid
Copy link
Contributor Author

Felixoid commented Nov 28, 2025

I added another condition to the search query created:*..{until}, it filters out obviously wrong entities:

> did last year --format=markdown --brief
Status report for the last year (2024-01-01 to 2024-12-31)
==========================================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Mikhail f. Shiryaev <mr.felixoid@gmail.com>
Mikhail f. Shiryaev <felixoid@clickhouse.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Issues created on GitHub: 32
* Issues commented on GitHub: 107
* Issues closed on GitHub: 30
* Pull requests created on GitHub: 223
* Pull requests commented on GitHub: 287
* Pull requests closed on GitHub: 655
* Pull requests reviewed on GitHub: 92
Time: 2 min. 31.85 s.

It's just a bit longer than the master's code, but it finds all the issues and PRs from the last year.

And another comparison, the latest commit VS master, 20-30 entities difference:

Details
# master
> did --since=2023-01-01 --until=2023-12-31 --format=markdown --brief
Status report for given date range (2023-01-01 to 2023-12-31)
=============================================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Mikhail f. Shiryaev <mr.felixoid@gmail.com>
Mikhail f. Shiryaev <felixoid@clickhouse.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Issues created on GitHub: 55
* Issues commented on GitHub: 89
* Issues closed on GitHub: 30
* Pull requests created on GitHub: 239
* Pull requests commented on GitHub: 325
* Pull requests closed on GitHub: 723
* Pull requests reviewed on GitHub: 88
Time: 2 min. 30.18 s.

# PR
> did --since=2023-01-01 --until=2023-12-31 --format=markdown --brief
Status report for given date range (2023-01-01 to 2023-12-31)
=============================================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Mikhail f. Shiryaev <mr.felixoid@gmail.com>
Mikhail f. Shiryaev <felixoid@clickhouse.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Issues created on GitHub: 55
* Issues commented on GitHub: 118
* Issues closed on GitHub: 30
* Pull requests created on GitHub: 239
* Pull requests commented on GitHub: 345
* Pull requests closed on GitHub: 723
* Pull requests reviewed on GitHub: 88
Time: 2 min. 47.71 s.

@Felixoid
Copy link
Contributor Author

Felixoid commented Dec 2, 2025

Hello!

Can I kindly request another review round?

@kwk kwk removed their request for review December 2, 2025 10:38
@sandrobonazzola sandrobonazzola merged commit 33c8027 into psss:main Dec 19, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants