diff --git a/.claude/specs/til-workflow.md b/.claude/specs/til-workflow.md new file mode 100644 index 00000000..ca2ea575 --- /dev/null +++ b/.claude/specs/til-workflow.md @@ -0,0 +1,384 @@ +# TIL Workflow Spec + +This spec documents a workflow for Claude to suggest and draft TIL-style blog posts based on conversations, git history, and Notion content. + +## Goals + +- Enable Claude to organically suggest TIL topics during conversations +- Provide explicit commands to scan for TIL opportunities +- Draft TILs in the user's voice with proper Notion formatting +- Keep Claude's drafts clearly separated from user's content + +## Non-Goals + +- Auto-publishing (user always reviews and publishes) +- Editing existing user content (Claude only creates new pages) +- Complex multi-part blog posts (TILs only) + +--- + +## Architecture + +### Skills (in `~/.claude/skills/`) + +``` +skills/ + scanning-git-for-tils/ + SKILL.md + scan_git.py + scanning-notion-for-tils/ + SKILL.md + scan_notion.py + drafting-til/ + SKILL.md # Voice guide, format rules, property mappings +``` + +### Commands (in `~/.claude/commands/`) + +``` +commands/ + suggest-tils.md # Orchestrates the full workflow +``` + +### Database Changes + +Add "Claude Draft" status to Writing database (Status property). + +### CLAUDE.md Addition + +Add organic trigger hint to global CLAUDE.md. + +--- + +## Writing Database Schema + +**Database URL**: `https://www.notion.so/eb0cbc7a4fe4495499bd94c1bf861469` +**Data Source ID**: `c296db5b-d2f1-44d4-abc6-f9a05736b143` + +### Key Properties for TIL Creation + +| Property | Type | TIL Value | +| ----------- | ------------ | ---------------------------------- | +| Title | title | The TIL title | +| Status | status | "Claude Draft" (new status to add) | +| Type | select | "how-to" | +| Destination | multi_select | ["blog"] | +| Description | text | One-line summary | +| Slug | text | URL-friendly version of title | +| Topics | relation | Link to relevant Topics | +| Research | relation | Link to source Research items | +| Questions | relation | Link to source Questions | + +### Status Options (existing) + +- New, Researching, Drafting, Editing, Publishing, Published, Paused, Migrated to content repo, Archived + +**Add**: "Claude Draft" (in to_do group, distinct color like orange) + +--- + +## Voice Guide + +### Source Material Analyzed + +1. **Notion post**: "The filter(Boolean) trick" - ~500 words, detailed how-to +2. **Website post**: "Ignoring files you've already committed" - ~150 words +3. **Website post**: "A '!' prefix makes any Tailwind CSS class important" - ~50 words + +### Two TIL Formats + +**Ultra-short (50-150 words)** + +- Single tip with one code example +- Minimal explanation +- Best for simple gotchas or quick references + +**Standard (300-500 words)** + +- Problem → bad solution → good solution structure +- Multiple code examples +- More explanation and personality +- Best for concepts that need unpacking + +### Voice Characteristics + +1. **Direct titles** - State exactly what the reader will learn + - Good: "The filter(Boolean) trick" + - Good: "A '!' prefix makes any Tailwind CSS class important" + - Bad: "Understanding Array Methods in JavaScript" + +2. **Problem-first opening** - Start with the issue + - "If you try to `.gitignore` files _after_ committing them, you'll notice it doesn't work" + - "You have an array... But hiding in that array are some unusable null or undefined values" + +3. **Conversational tone** + - Use "you" to address reader directly + - Contractions are fine + - Second person throughout + +4. **Playful asides and humor** + - "Illegal! Now you're a criminal" + - "Oh noooo..." + - "Really, really no vertical margins" + - Don't overdo it - one or two per post + +5. **Code examples always included** + - Show the problem code + - Show the solution code + - Inline comments can have personality + +6. **No fluff** + - Get to the point quickly + - Short paragraphs + - Scannable structure + +7. **Helpful signoff** (optional) + - "Hope that helps!" + +### What NOT to Do + +- Don't be formal or academic +- Don't over-explain obvious things +- Don't use passive voice +- Don't add unnecessary caveats +- Don't start with "In this post, I'll show you..." + +--- + +## Skill Specifications + +### scanning-git-for-tils + +**Purpose**: Analyze recent git commits for TIL-worthy patterns + +**Description** (for SKILL.md): + +``` +Scans git history for commits that might make good TIL blog posts. +Looks for bug fixes, configuration changes, gotchas, and interesting +solutions. Returns a formatted list of suggestions with commit context. +Use when user asks for TIL ideas from their recent work. +``` + +**What to look for**: + +- Commits with "fix" that solved a non-obvious problem +- Configuration changes (dotfiles, CI, tooling) +- Dependency updates that required code changes +- Commits with detailed messages explaining "why" +- Patterns that repeat (user keeps solving same problem) + +**Output format**: + +``` +📝 TIL Opportunities from Git History (last 30 days): + +1. **Git: Ignoring already-tracked files** + - Commit: abc123 "fix: properly ignore .env after initial commit" + - Pattern: Removed cached files, updated .gitignore + - TIL angle: Common gotcha - .gitignore doesn't affect tracked files + +2. **Zsh: Fixing slow shell startup** + - Commits: def456, ghi789 (related) + - Pattern: Lazy-loaded nvm, deferred compinit + - TIL angle: Diagnose and fix slow shell initialization +``` + +### scanning-notion-for-tils + +**Purpose**: Find unpublished Writing items ready for TIL treatment + +**Description** (for SKILL.md): + +``` +Searches the Notion Writing database for unpublished items that could +become TIL posts. Prioritizes items with Status=New or Drafting, +Type=how-to, and recent activity. Returns suggestions with context. +Use when user wants to review their backlog for TIL opportunities. +``` + +**Search criteria**: + +- Status: New, Researching, or Drafting +- Type: how-to (preferred) or reference +- Has linked Research or Questions (indicates depth) +- Sorted by Last edited (recent activity) + +**Output format**: + +``` +📝 TIL Opportunities from Notion Backlog: + +1. **"Make TS understand Array.filter by using type predicates"** + - Status: Drafting | Last edited: 2 months ago + - Has: 2 Research links, 1 Question + - TIL angle: Type predicates let TS narrow filtered arrays + +2. **"How to filter a JS array with async/await"** + - Status: New | Last edited: 1 year ago + - Has: 1 Research link + - TIL angle: filter() doesn't await - need Promise.all pattern +``` + +### drafting-til + +**Purpose**: Create a TIL draft in Notion with proper voice and formatting + +**Description** (for SKILL.md): + +``` +Drafts a TIL blog post in the user's voice and creates it in Notion +with Status="Claude Draft". Uses the voice guide for tone and format. +Includes proper property mappings for the Writing database. +Use when user approves a TIL suggestion and wants a draft created. +``` + +**SKILL.md should include**: + +- Complete voice guide (from above) +- Property mappings +- Example TIL structures (ultra-short and standard) +- Instructions for using Notion MCP tools + +**Creation process**: + +1. Determine appropriate length (ultra-short vs standard) +2. Write title (direct, specific) +3. Write content following voice guide +4. Generate slug from title +5. Write one-line description +6. Create page with properties: + - Status: "Claude Draft" + - Type: "how-to" + - Destination: ["blog"] + - Topics: (link if obvious match) + - Research/Questions: (link to sources) + +--- + +## Command Specification + +### /suggest-tils + +**Purpose**: Orchestrate the full TIL suggestion and drafting workflow + +**Workflow**: + +``` +Phase 1: Source Selection +───────────────────────── +Which sources to scan? +1. Git history (last 30 days) +2. Notion backlog +3. Both +> +``` + +``` +Phase 2: Scan Results +───────────────────── +[Invoke appropriate skill(s)] +[Display combined suggestions] + +Select a topic to draft (number), or 'q' to quit: +> +``` + +``` +Phase 3: Draft Creation +─────────────────────── +[Invoke drafting-til skill with selected topic] +[Show preview of created page] + +✅ Draft created: "Your TIL Title" + Status: Claude Draft + URL: https://www.notion.so/... + +Actions: +o - Open in Notion +e - Edit properties +n - Draft another +q - Done +> +``` + +**State management**: Use TodoWrite to track workflow phase + +--- + +## CLAUDE.md Addition + +Add to global `~/.claude/CLAUDE.md` (symlinked from `tools/claude/config/CLAUDE.md`): + +```markdown +## TIL Suggestions + +When you help solve a non-trivial problem or explain something in detail, +consider if it would make a good TIL blog post. Look for: + +- Gotchas or surprising behavior +- Elegant solutions to common problems +- Things worth documenting for future reference + +Suggest naturally: "This could make a good TIL - want me to draft it?" + +To scan for TIL opportunities or draft posts, use the `/suggest-tils` command. +``` + +--- + +## Implementation Order + +1. **Add "Claude Draft" status** to Writing database + - Use `mcp__notion__notion-update-database` to add status option + +2. **Create drafting-til skill** first (other skills depend on understanding the output format) + - `~/.claude/skills/drafting-til/SKILL.md` + +3. **Create scanning-git-for-tils skill** + - `~/.claude/skills/scanning-git-for-tils/SKILL.md` + - `~/.claude/skills/scanning-git-for-tils/scan_git.py` + +4. **Create scanning-notion-for-tils skill** + - `~/.claude/skills/scanning-notion-for-tils/SKILL.md` + - `~/.claude/skills/scanning-notion-for-tils/scan_notion.py` + +5. **Create /suggest-tils command** + - `~/.claude/commands/suggest-tils.md` + +6. **Add CLAUDE.md hint** + - Update `tools/claude/config/CLAUDE.md` + +--- + +## Safety Rules + +These rules prevent Claude from making unwanted edits: + +1. **Never edit existing pages** unless explicitly asked +2. **Always use Status="Claude Draft"** for new pages +3. **Show content before creating** - user approves the draft text +4. **Link sources via relations** - don't modify source pages +5. **User publishes** - Claude never changes Status to Published + +--- + +## Testing the Workflow + +After implementation, test with: + +1. `/suggest-tils` → select "Git history" → verify scan results +2. `/suggest-tils` → select "Notion backlog" → verify scan results +3. Select a suggestion → verify draft created with correct properties +4. Check Writing database filtered by Status="Claude Draft" +5. Organic test: Solve a problem, see if Claude suggests TIL + +--- + +## Future Enhancements (Out of Scope) + +- Browser history scanning +- Slack conversation scanning +- Automatic topic detection/linking +- Draft quality scoring +- Publishing workflow automation diff --git a/.github/workflows/test-claude-skills.yml b/.github/workflows/test-claude-skills.yml new file mode 100644 index 00000000..069bd92f --- /dev/null +++ b/.github/workflows/test-claude-skills.yml @@ -0,0 +1,132 @@ +name: Test Claude + +on: + # Run on pull requests to main + pull_request: + branches: [main] + paths: + - 'tools/claude/config/skills/**/*.py' + - 'tools/claude/config/skills/**/pyproject.toml' + - '.github/workflows/test-claude-skills.yml' + + # Run on pushes to main branch + push: + branches: [main] + paths: + - 'tools/claude/config/skills/**/*.py' + - 'tools/claude/config/skills/**/pyproject.toml' + - '.github/workflows/test-claude-skills.yml' + + # Allow manual triggering for debugging + workflow_dispatch: + +jobs: + skills: + name: Skills + runs-on: macos-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Install uv + run: brew install uv + + - name: Run ruff checks + run: | + echo "Running ruff checks on all skills..." + cd tools/claude/config/skills + + for skill_dir in */; do + if [[ -f "$skill_dir/pyproject.toml" ]]; then + echo "" + echo "Checking $skill_dir with ruff..." + cd "$skill_dir" + uv run --with ruff ruff check . || exit 1 + cd .. + echo "✅ Passed" + fi + done + + echo "" + echo "✅ All ruff checks passed" + + - name: Run mypy checks + run: | + echo "Running mypy checks on all skills..." + cd tools/claude/config/skills + + for skill_dir in */; do + if [[ -f "$skill_dir/pyproject.toml" ]]; then + echo "" + echo "Type checking $skill_dir with mypy..." + cd "$skill_dir" + uv run --with mypy --with notion-client --with pydantic --with pytest mypy --python-version 3.11 . || exit 1 + cd .. + echo "✅ Passed" + fi + done + + echo "" + echo "✅ All mypy checks passed" + + - name: Run tests + run: | + echo "Searching for skill tests..." + cd tools/claude/config/skills + + FAILED_TESTS="" + PASSED_TESTS=0 + + # Find all test_*.py files + for test_file in */test_*.py; do + if [[ -f "$test_file" ]]; then + echo "" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "Running: $test_file" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + + if uv run "$test_file"; then + ((PASSED_TESTS++)) + echo "✅ Passed: $test_file" + else + FAILED_TESTS="$FAILED_TESTS$test_file\n" + echo "❌ Failed: $test_file" + fi + fi + done + + echo "" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "Test Summary" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + + if [[ -n "$FAILED_TESTS" ]]; then + echo "❌ Failed tests:" + echo -e "$FAILED_TESTS" + echo "" + echo "Passed: $PASSED_TESTS" + exit 1 + fi + + if [[ $PASSED_TESTS -eq 0 ]]; then + echo "⚠️ No test files found" + exit 0 + fi + + echo "✅ All $PASSED_TESTS test file(s) passed" + + - name: Summary + if: always() + run: | + echo "=== Test Summary ===" + echo "Repository: ${{ github.repository }}" + echo "Branch: ${{ github.ref_name }}" + echo "Commit: ${{ github.sha }}" + echo "Runner OS: ${{ runner.os }}" + + if [[ "${{ job.status }}" == "success" ]]; then + echo "🎉 All Claude skill tests passed!" + else + echo "❌ Some tests failed. Check the logs above for details." + fi diff --git a/.gitignore b/.gitignore index 7c302852..b88cab7d 100644 --- a/.gitignore +++ b/.gitignore @@ -2,9 +2,11 @@ .DS_Store *.log .nvimlog +*.pyc # Mine **/.claude/settings.local.json .cache .secrets Brewfile.lock.json +node_modules/ diff --git a/tools/bun/Brewfile b/tools/bun/Brewfile new file mode 100644 index 00000000..36e4d18a --- /dev/null +++ b/tools/bun/Brewfile @@ -0,0 +1 @@ +brew "oven-sh/bun/bun" diff --git a/tools/bun/install.bash b/tools/bun/install.bash new file mode 100755 index 00000000..aa1e1c65 --- /dev/null +++ b/tools/bun/install.bash @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail + +source "${DOTFILES}/tools/bash/utils.bash" + +info "🍞 Installing bun" +brew bundle --file="${DOTFILES}/tools/bun/Brewfile" diff --git a/tools/bun/uninstall.bash b/tools/bun/uninstall.bash new file mode 100755 index 00000000..0c984d6d --- /dev/null +++ b/tools/bun/uninstall.bash @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail + +source "${DOTFILES}/tools/bash/utils.bash" + +info "🍞 Uninstalling btop" +brew uninstall --formula bun diff --git a/tools/bun/update.bash b/tools/bun/update.bash new file mode 100755 index 00000000..e3c1398e --- /dev/null +++ b/tools/bun/update.bash @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail + +source "${DOTFILES}/tools/bash/utils.bash" + +info "🍞 Updating btop" +brew bundle --file="${DOTFILES}/tools/bun/Brewfile" diff --git a/tools/claude/config/CLAUDE.md b/tools/claude/config/CLAUDE.md index ad949da3..d8b861b7 100644 --- a/tools/claude/config/CLAUDE.md +++ b/tools/claude/config/CLAUDE.md @@ -14,12 +14,14 @@ ### 1. Skills First (Highest Priority) Use skills (`tools/claude/config/skills/`) for token-heavy operations: + - **When**: Heavy data processing, filtering, caching opportunities - **Why**: Process data in code (Python/bash), return only filtered summaries - **Token savings**: 80-98% reduction vs processing via Claude tools - **Examples**: `fetching-github-prs-to-review`, `inspecting-codefresh-failures` Skills should: + - Filter data in code before returning to Claude - Return formatted summaries, not raw data - Cache intermediate results to avoid redundant processing @@ -29,12 +31,14 @@ Skills should: #### Creating New Skills When creating skills, ALWAYS use the `/create-skill` command or reference the template: + - **Template location**: `tools/claude/config/skills/@template/` - **Complete guidance**: See `@template/README.md` for all best practices - **Naming convention**: gerund + noun (e.g., `fetching-github-prs-to-review`, `analyzing-python-code`) - **Examples**: See existing skills in `tools/claude/config/skills/` The template includes: + - SKILL.md structure with workflow patterns - Example Python script with all best practices - Anti-patterns to avoid @@ -43,6 +47,7 @@ The template includes: ### 2. Agents Second Use agents (`tools/claude/config/agents`) for complex exploration: + - **When**: Tasks requiring multiple tool calls, exploration, or investigation - **Why**: Agents can autonomously explore and make decisions - **Examples**: `atomic-committer`, `pr-creator`, ephemeral `Explore`/`Plan` agents @@ -51,6 +56,7 @@ Use agents (`tools/claude/config/agents`) for complex exploration: ### 3. Direct Tool Usage Last Use Claude tools directly only for: + - Simple, one-off operations - Tasks requiring immediate context from the conversation - Operations where overhead of a skill/agent isn't justified @@ -69,6 +75,18 @@ Use Claude tools directly only for: - Implement changes one small theme at-a-time - Pause after each theme is implemented (behavior + test case(s) + documentation) to let me commit myself +## TIL Suggestions + +When you help solve a non-trivial problem or explain something in detail, consider if it would make a good TIL blog post. Look for: + +- Gotchas or surprising behavior +- Elegant solutions to common problems +- Things worth documenting for future reference + +Suggest naturally: "This could make a good TIL - want me to draft it?" + +To scan for TIL opportunities or draft posts, use the `/suggest-tils` command. + ## CI System Information ### Recursion Pharma Organization @@ -81,6 +99,7 @@ Use Claude tools directly only for: When you see a CI failure in a recursionpharma PR, **use the `inspecting-codefresh-failures` skill** to analyze it. The skill will: + - Extract build IDs from PR status checks - Fetch build logs from Codefresh - Identify specific errors with file:line references @@ -88,3 +107,7 @@ The skill will: - Return a formatted report ready to include in reviews **Always investigate CI failures** - include specific error details in your review (not just "CI is failing"). Distinguish between errors introduced by the PR vs pre-existing issues. + +## Python Preferences + +- Prefer `from __future__ import annotations` over `from typing import TYPE_CHECKING` diff --git a/tools/claude/config/commands/suggest-tils.md b/tools/claude/config/commands/suggest-tils.md new file mode 100644 index 00000000..b37add2d --- /dev/null +++ b/tools/claude/config/commands/suggest-tils.md @@ -0,0 +1,99 @@ +# Suggest TILs - Find and draft TIL blog posts + +Scan git history for TIL opportunities, then draft selected topics. + +## Phase 1: Date Range Selection + +Ask the user how far back to search: + +``` +📝 TIL Suggestions from Git History + +How far back should I search? +- Enter number of days (default: 30) +- Or just press Enter for 30 days + +> +``` + +Note: Very large ranges (365+ days) may take longer but will find more candidates. + +## Phase 2: Scan Git History + +Use the `scanning-git-for-tils` skill: + +1. Run the scan script with the specified days +2. Script automatically fetches assessed commits from Notion +3. Script fetches and filters GitHub commits +4. Display the markdown results for evaluation + +## Phase 3: Selection + +After displaying results: + +``` +Select a commit to draft (enter number), or: +- 's' to scan again with different date range +- 'q' to quit + +> +``` + +## Phase 4: Draft TIL + +When user selects a commit: + +Use the `drafting-til` skill: + +1. Look up full commit data using the index from `new_commits` array +2. Generate TIL content following voice guide +3. Show draft to user for approval +4. When approved, pass JSON to `publish_til.py` script +5. Display Writing page URL from script output + +## Phase 5: Post-Creation + +After successfully publishing a draft: + +``` +✅ Draft published to Writing database + +Title: "Your TIL Title" +Status: Claude Draft +URL: https://www.notion.so/... + +Actions: +- Select another commit number to draft +- 's' to scan again with different date range +- 'q' to finish + +> +``` + +## State Management + +Use TodoWrite to track workflow state: + +``` +- "Scanning git history for TILs" (in_progress) +- "Waiting for user selection" (pending) +- "Drafting TIL" (pending) +- "Publishing draft" (pending) +``` + +Update todos as you progress through phases. + +## Safety Rules + +1. **Show content before publishing** - user must approve draft text +2. **Use `publish_til.py` for all Notion operations** - don't create pages manually +3. **Reference commits by index** - use `new_commits[index]` for full data +4. **Let script handle duplicates** - it checks for existing tracker entries + +## Notes + +- This command orchestrates the git scanning and drafting workflow +- User can draft multiple TILs from one scan +- Only drafted commits get marked as assessed (others stay in pool for next scan) +- All drafts go into Writing database with Status="Claude Draft" +- Tracker entries link back to drafts via Writing relation diff --git a/tools/claude/config/skills/@template/README.md b/tools/claude/config/skills/@template/README.md index 4ace6d0f..ab3fc515 100644 --- a/tools/claude/config/skills/@template/README.md +++ b/tools/claude/config/skills/@template/README.md @@ -12,6 +12,485 @@ Create a skill (instead of a command or agent) when: 4. **Caching opportunities**: Results can be cached to avoid redundant operations 5. **Type safety matters**: Structured data with typed interfaces improves reliability +## Implementation Approach: Python + uv + +Skills are implemented in **Python with uv** using the single-file script pattern ([PEP 723](https://peps.python.org/pep-0723/)): + +### Why Python + uv? + +✅ **Self-contained scripts** - Dependencies declared inline, auto-install with `uv run` +✅ **Rich ecosystem** - Excellent SDKs for most external services (GitHub, Notion, Slack, etc.) +✅ **Pragmatic typing** - Good-enough type safety without fighting the type system +✅ **Fast iteration** - No build step, quick prototyping +✅ **Familiar tooling** - Standard Python ecosystem (mypy, ruff, pytest) + +### Key Principles + +1. **Use SDKs over raw HTTP** - Prefer official/well-maintained SDKs for external data sources (GitHub, Notion, Jira, etc.) rather than building your own API clients +2. **Single-file scripts** - Use PEP 723 inline dependencies, not pyproject.toml package definitions +3. **Pragmatic type safety** - Type hints everywhere, Pydantic at API boundaries, mypy in non-strict mode + +## Type Safety Patterns + +### Pydantic for Parsing Outputs (Not Inputs) + +Use Pydantic to **validate API responses** (outputs from external APIs), not to validate your own function inputs: + +```python +from pydantic import BaseModel, ConfigDict + +class NotionPageResponse(BaseModel): + """Validated Notion API response.""" + + model_config = ConfigDict(extra="ignore") # Pydantic v2 pattern + + url: str + id: str + +def create_page(notion, title: str) -> str: + """Create page and return URL.""" + # Call external API + response = notion.pages.create(...) + + # ✅ Validate immediately after API call + page = NotionPageResponse.model_validate(response) + + # ✅ Extract and return values immediately + return page.url # Don't pass Pydantic models around +``` + +**Pattern:** +1. External API returns data (untyped `dict`) +2. Validate with Pydantic **immediately** +3. Extract needed values (primitives or dataclasses) +4. Use extracted values in rest of code + +**Why:** Pydantic provides runtime validation at the boundary where types are uncertain (external APIs). Once validated, use native Python types. + +### Dataclasses for Internal Data + +Use dataclasses for internal data structures: + +```python +from dataclasses import dataclass + +@dataclass +class Commit: + """A git commit with metadata.""" + + hash: str + message: str + repo: str + date: str + files: list[str] + +def process_commits(commits: list[Commit]) -> str: + """Process commits - type-safe throughout.""" + # Work with well-typed data, no runtime validation needed + return format_markdown(commits) +``` + +**Why:** Dataclasses are lightweight, well-integrated with mypy, and perfect for internal data that's already validated. + +### TypeGuard for Narrowing Types + +When you need to narrow types beyond simple `isinstance` checks: + +```python +from typing import TypeGuard + +def is_valid_commit(data: dict) -> TypeGuard[dict]: + """Type guard to narrow dict to valid commit structure.""" + return ( + isinstance(data.get("hash"), str) and + isinstance(data.get("message"), str) and + isinstance(data.get("repo"), str) + ) + +def process_data(items: list[dict]) -> list[Commit]: + """Process raw data with type narrowing.""" + commits = [] + for item in items: + if is_valid_commit(item): + # mypy knows item has required fields + commits.append(Commit( + hash=item["hash"], + message=item["message"], + repo=item["repo"], + date=item.get("date", ""), + files=item.get("files", []) + )) + return commits +``` + +**When to use:** Complex validation logic that you want mypy to understand. + +### Pydantic → Dataclass Conversion + +For complex APIs, validate with Pydantic then convert to dataclass: + +```python +from pydantic import BaseModel +from dataclasses import dataclass + +class NotionPageValidation(BaseModel): + """Pydantic model for validation only.""" + url: str + id: str + properties: dict + +@dataclass +class NotionPage: + """Dataclass for internal use.""" + url: str + id: str + title: str + +def fetch_page(page_id: str) -> NotionPage: + """Fetch and validate page from API.""" + response = notion.pages.retrieve(page_id) + + # Validate with Pydantic + validated = NotionPageValidation.model_validate(response) + + # Extract to dataclass + title = validated.properties.get("Title", {}).get("title", [{}])[0].get("plain_text", "") + return NotionPage( + url=validated.url, + id=validated.id, + title=title + ) +``` + +**When to use:** When you need both runtime validation (Pydantic) and clean internal types (dataclass). + +## Mitigating Non-Strict Mypy + +Since we use `strict = false` for pragmatic SDK integration, compensate with these patterns: + +### 1. Type Hints Everywhere + +```python +# ✅ Always include type hints +def get_commits(days: int, username: str) -> list[Commit]: + """Fetch commits from GitHub API.""" + # ... + +# ✅ Modern syntax (list[T], not List[T]) +from __future__ import annotations + +def process(items: list[str]) -> dict[str, int]: + # ... + +# ✅ Union types with | +def find_user(username: str) -> User | None: + # ... + +# ❌ Don't rely on inference +def process(items): # Type unknown! + # ... +``` + +### 2. Explicit Return Types + +```python +# ✅ Always declare return type +def parse_date(date_str: str) -> str: + if not date_str: + return "unknown" + # ... + return formatted + +# ❌ Don't let mypy infer +def parse_date(date_str: str): # Return type inferred (fragile) + # ... +``` + +### 3. Handle None Explicitly + +```python +# ✅ Check for None before using +def get_title(page: dict) -> str: + title_prop = page.get("properties", {}).get("Title") + if not title_prop: + return "" + + title_content = title_prop.get("title", []) + if not title_content: + return "" + + return title_content[0].get("plain_text", "") + +# ❌ Don't assume values exist +def get_title(page: dict) -> str: + return page["properties"]["Title"]["title"][0]["plain_text"] # Can crash! +``` + +### 4. Accept Any at SDK Boundaries Only + +```python +from typing import Any + +# ✅ Accept Any from SDK, validate immediately +def create_page(notion, title: str) -> str: + response: Any = notion.pages.create(...) # SDK returns Any + page = NotionPageResponse.model_validate(response) # Validate + return page.url # Type-safe from here + +# ✅ Internal functions are fully typed +def format_markdown(commits: list[Commit]) -> str: + # No Any types here + # ... +``` + +### 5. Small, Well-Typed Helper Functions + +```python +# ✅ Break complex logic into small, typed pieces +def _map_language_alias(language: str) -> str: + """Map language names to Notion's expected values.""" + lang_map = { + "js": "javascript", + "ts": "typescript", + "py": "python", + } + return lang_map.get(language, language) or "plain text" + +def _create_code_block(lines: list[str], start_index: int) -> tuple[dict, int]: + """Create code block from markdown. + + Returns: (block dict, next line index) + """ + language = _map_language_alias(lines[start_index][3:].strip()) + # ... + return block, next_index +``` + +**Why:** Small functions are easier to type correctly and verify. + +## Testing Patterns + +### Comprehensive Unit Tests for Pure Functions + +Test all pure functions (no I/O, deterministic output): + +```python +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["pytest", "notion-client", "pydantic"] +# /// + +from git.formatting import format_markdown, should_skip_commit +from git.types import Commit + +class TestFormatMarkdown: + """Test markdown formatting.""" + + def test_formats_empty_list(self): + result = format_markdown([], 30, 0, 0) + assert "No commits found" in result + + def test_formats_single_commit(self): + commit = Commit( + hash="abc1234", + message="feat: add feature", + repo="owner/repo", + date="2 days ago", + files=["main.py"] + ) + result = format_markdown([commit], 30, 1, 1) + + assert "[owner/repo] feat: add feature" in result + assert "Hash: abc1234" in result + +if __name__ == "__main__": + import pytest + import sys + sys.exit(pytest.main([__file__, "-v"])) +``` + +### Test Helpers for Readability + +Create helpers to make tests clear and maintainable: + +```python +def make_notion_page(commit_hash: str) -> dict: + """Helper: create mock Notion page with commit hash.""" + return { + "properties": { + "Commit Hash": {"title": [{"plain_text": commit_hash}]} + } + } + +def make_notion_response(hashes: list[str]) -> dict: + """Helper: create mock Notion SDK response.""" + return { + "results": [make_notion_page(h) for h in hashes], + "has_more": False, + } + +class TestGetAssessedCommits: + def test_returns_commit_hashes(self): + with patch("notion_client.Client") as MockClient: + mock_client = MockClient.return_value + mock_client.data_sources.query.return_value = make_notion_response( + ["abc123", "def456"] + ) + + result = get_assessed_commits() + assert result == {"abc123", "def456"} +``` + +### Test Edge Cases + +Always test edge cases: + +```python +class TestFormatRelativeDate: + def test_handles_invalid_date(self): + result = format_relative_date("not-a-date") + assert result == "unknown" + + def test_handles_empty_string(self): + result = format_relative_date("") + assert result == "unknown" + +class TestShouldSkipCommit: + def test_skips_dependabot(self): + commit = Commit(hash="abc", message="Bump dependency from 1.0 to 2.0", ...) + assert should_skip_commit(commit) is True + + def test_keeps_normal_commits(self): + commit = Commit(hash="abc", message="fix: handle null values", ...) + assert should_skip_commit(commit) is False +``` + +### Mock External Dependencies + +Mock all external I/O (APIs, CLIs, file system): + +```python +from unittest.mock import patch + +class TestGetAssessedCommits: + def test_returns_empty_set_when_no_token(self): + with patch("notion.commits.get_op_secret", side_effect=RuntimeError("Failed")): + result = get_assessed_commits_from_notion() + assert result == set() + + def test_handles_api_error_gracefully(self): + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client") as MockClient, + ): + MockClient.side_effect = Exception("Connection error") + + result = get_assessed_commits_from_notion() + assert result == set() +``` + +### Class-Based Test Organization + +Group related tests in classes: + +```python +class TestFormatMarkdown: + """Test markdown formatting.""" + # All markdown tests here + +class TestShouldSkipCommit: + """Test commit filtering.""" + # All filtering tests here + +class TestExtractPageId: + """Test Notion URL parsing.""" + # All URL tests here +``` + +**Why:** Clear organization, easy to run subsets (`pytest test_file.py::TestClass`) + +### Descriptive Test Names + +Use names that describe what's being tested: + +```python +# ✅ Clear what's being tested +def test_formats_commit_with_long_body(self): +def test_handles_pagination(self): +def test_returns_empty_set_when_no_token(self): +def test_skips_pages_without_commit_hash(self): + +# ❌ Vague names +def test_format(self): +def test_pagination(self): +def test_error(self): +``` + +## Error Handling Patterns + +### Raise Clear Exceptions at Boundaries + +```python +def get_op_secret(path: str) -> str: + """Fetch secret from 1Password. + + Raises: + RuntimeError: If 1Password CLI fails with error details. + """ + result = subprocess.run(["op", "read", path], capture_output=True, text=True) + + if result.returncode != 0: + # ✅ Raise with clear message including details + raise RuntimeError( + f"Failed to retrieve secret from 1Password: {result.stderr.strip()}" + ) + + return result.stdout.strip() +``` + +### Handle Errors Gracefully at Call Sites + +```python +def get_assessed_commits_from_notion() -> set[str]: + """Fetch assessed commits from Notion. + + Returns empty set on any error for graceful degradation. + """ + try: + token = get_op_secret(OP_NOTION_TOKEN_PATH) # Can raise RuntimeError + notion = Client(auth=token) + except Exception: + # ✅ Graceful degradation - return empty set + return set() + + try: + pages = notion.data_sources.query(...) + return {extract_hash(p) for p in pages} + except Exception: + # ✅ API errors also degrade gracefully + return set() +``` + +### Document Error Behavior + +```python +def create_page(notion, title: str) -> str: + """Create page in Notion. + + Returns: + URL of created page. + + Raises: + Exception: If page creation fails with error details. + """ + response = notion.pages.create(...) + page = NotionPageResponse.model_validate(response) + return page.url +``` + +**Pattern:** Raise at boundaries with details, handle gracefully at call sites, document behavior. + ## Skills vs Agents vs Commands | Approach | Use When | Example | @@ -80,9 +559,21 @@ Edit `SKILL.md` to define: Update the sections to describe your specific skill. -### 3. Implement Your Logic +### 3. Choose Your Language -In `example_skill.py` (rename to match your skill): +**For Python skills:** +- Use PEP 723 inline script metadata for dependencies +- Include type hints for reliability +- See `example_skill.py` for the template + +**For TypeScript/Bun skills:** +- Use inline version specifiers: `import { z } from "zod@^3.22.4"` +- Leverage Zod for validation + types (single system) +- See `scanning-git-for-tils` skill for real-world example + +### 4. Implement Your Logic + +**In `example_skill.py` (Python):** 1. **Configuration** (lines 18-26): Update cache file names, TTLs, etc. 2. **Type Definitions** (lines 30-48): Define your data structures @@ -90,7 +581,12 @@ In `example_skill.py` (rename to match your skill): 4. **filter_and_process()** (lines 129-171): Filter and transform data 5. **format_markdown()** (lines 177-203): Format output for display -### 4. Test Your Skill +**In TypeScript/Bun:** +- Define Zod schemas for validation AND types +- Use `Bun.spawn()` for process execution +- Return JSON to stdout for Claude to parse + +### 5. Test Your Skill ```bash # Test locally first @@ -100,7 +596,7 @@ python3 ~/.claude/skills/your-skill-name/your_script.py python3 ~/.claude/skills/your-skill-name/your_script.py # Should use cache ``` -### 5. Document Token Savings +### 6. Document Token Savings After implementation, measure and document: - Tokens before (if processed via Claude tools) diff --git a/tools/claude/config/skills/drafting-til/SKILL.md b/tools/claude/config/skills/drafting-til/SKILL.md new file mode 100644 index 00000000..d8b2356e --- /dev/null +++ b/tools/claude/config/skills/drafting-til/SKILL.md @@ -0,0 +1,299 @@ +--- +name: drafting-til +description: Drafts a TIL blog post in the user's voice and creates it in Notion with Status="Claude Draft". Contains voice guide for matching the user's writing style. Use when user approves a TIL topic and wants a draft created. +--- + +# Draft TIL Skill + +Creates a TIL blog post draft in Notion following the user's voice and style. + +## Voice Guide + +### Spirit + +1. **Learning in public** - I'm sharing things I found helpful; you might too +2. **We don't take ourselves seriously** - Coding is fun, not solemn +3. **Respect your time** - Get to the point quickly +4. **Keep it light** - Direct but not dry + +Every technique below serves these principles. + +### Two TIL Formats + +**Ultra-short (50-150 words)** +- Single tip with one code example +- Minimal explanation +- Best for simple gotchas or quick references + +**Standard (300-500 words)** +- Problem → bad solution → good solution structure +- Multiple code examples +- More explanation and personality +- Best for concepts that need unpacking + +### Voice Characteristics + +1. **Direct titles** - State exactly what the reader will learn + - Good: "The filter(Boolean) trick" + - Good: "A '!' prefix makes any Tailwind CSS class important" + - Bad: "Understanding Array Methods in JavaScript" + +2. **Problem-first opening** - Start with the issue + - "If you try to `.gitignore` files _after_ committing them, you'll notice it doesn't work" + - "You have an array... But hiding in that array are some unusable null or undefined values" + +3. **Conversational tone** + - Use "you" to address reader directly + - Contractions are fine + - Second person throughout + +4. **Code examples always included** + - Show the problem code + - Show the solution code + - Inline comments can have personality + +5. **No fluff** + - Get to the point quickly + - Short paragraphs + - Scannable structure + +### Rhythm and Tonal Variation + +**Critical**: Don't be relentlessly direct. Alternate rapid-fire teaching with moments of personality. + +**The pattern**: Direct instruction → tonal break → direct instruction → tonal break + +**Types of tonal breaks** (use 2-4 per standard post): + +1. **Playful asides** - Brief moments of humor + - "Illegal! Now you're a criminal." + - "Really, really no vertical margins" + - "Oh noooo..." + +2. **Casual satisfaction** - Express relief or confidence + - "Happily, there's a cleaner way." + - "It worked like a charm." + - End with "😎" after a satisfying solution + +3. **Honest reflection** - Admit limitations or show thought process + - "the example I showed above doesn't actually work out very well!" + - "After trying a number of workarounds..." + - "this was a fun exercise in the meantime" + +4. **Varied closings** - Don't repeat the same signoff + - "Hope that clears things up." + - "That's the trick." + - Or just end on the last code example (no closing needed) + +**Example rhythm**: +``` +[Direct: state problem] +[Direct: show bad code] +[Tonal break: "Illegal! Now you're a criminal."] +[Direct: explain solution] +[Direct: show good code] +[Tonal break: casual closing or 😎] +``` + +### What NOT to Do + +- Don't be formal or academic +- Don't over-explain obvious things +- Don't use passive voice +- Don't add unnecessary caveats +- Don't start with "In this post, I'll show you..." + +--- + +## Notion Databases + +### Writing Database +**Data Source ID**: `c296db5b-d2f1-44d4-abc6-f9a05736b143` + +### TIL Assessed Commits Database +**Data Source ID**: `cba80148-aeef-49c9-ba45-5157668b17b3` + +--- + +## Property Mappings + +When creating a TIL page in the Writing database, set these properties: + +| Property | Value | +|----------|-------| +| Title | The TIL title (direct, specific) | +| Status | "Claude Draft" | +| Type | "how-to" | +| Destination | ["blog"] | +| Description | One-line summary of what reader will learn | +| Slug | URL-friendly version of title (lowercase, hyphens) | +| Topics | Link to relevant Topics if obvious match exists | +| Research | Link to source Research items if applicable | +| Questions | Link to source Questions if applicable | + +--- + +## Creation Process + +1. **Determine format** - Ultra-short or standard based on topic complexity +2. **Write title** - Direct, specific, states what reader will learn +3. **Write content** - Follow voice guide above +4. **Generate slug** - Lowercase title with hyphens, no special chars +5. **Write description** - One sentence summarizing the takeaway +6. **Publish via Notion MCP** - Two-step process: + +### Step 1: Create Writing Page + +Use `mcp__notion__notion-create-pages` with Writing data source: + +```javascript +{ + "parent": {"data_source_id": "c296db5b-d2f1-44d4-abc6-f9a05736b143"}, + "pages": [{ + "properties": { + "Title": "Your TIL Title Here", + "Status": "Claude Draft", + "Type": "how-to", + "Destination": "blog", + "Description": "One-line summary here", + "Slug": "your-til-title-here" + }, + "content": "Your markdown content here..." + }] +} +``` + +**Capture the Writing page URL from the response** - you'll need it for step 2. + +### Step 2: Create Tracker Entry + +Use `mcp__notion__notion-create-pages` with TIL Assessed Commits data source: + +```javascript +{ + "parent": {"data_source_id": "cba80148-aeef-49c9-ba45-5157668b17b3"}, + "pages": [{ + "properties": { + "Commit Hash": "full-sha-hash", + "Message": "original commit message", + "Repo": "owner/repo", + "date:Commit Date:start": "2025-01-15", + "date:Commit Date:is_datetime": 0, + "date:Assessed:start": "2025-01-22", + "date:Assessed:is_datetime": 0, + "Writing": "[\"https://www.notion.so/writing-page-url\"]" + } + }] +} +``` + +**Important**: The `Writing` property must be a JSON array of URLs as a string: `"[\"url\"]"` + +Both operations return URLs - display both to the user. + +--- + +## Example: Ultra-Short TIL + +**Title**: Tmux's version command is -V + +**Content**: +```markdown +Most CLI tools use `--version` or `-v` to show their version. + +Tmux uses `-V` (capital V): + +```bash +tmux -V +# tmux 3.4 +``` + +The lowercase `-v` enables verbose mode instead. +``` + +**Properties**: +- Status: Claude Draft +- Type: how-to +- Destination: ["blog"] +- Description: Check tmux version with -V (capital V), not -v +- Slug: tmux-version + +--- + +## Example: Standard TIL + +**Title**: The filter(Boolean) trick + +**Content**: +```markdown +Here's a trick I often find helpful. + +## Bad array. Very, very bad. + +You have an array of whatever: + +```javascript +const array = [{ stuff }, { moreStuff }, ...] +``` + +But hiding in that array are some unusable `null` or `undefined` values: + +```javascript +const array = [{ good }, null, { great }, undefined] +``` + +## Looping over null data + +If you try to perform actions on every item in the array, you'll run into errors: + +```javascript +const newArray = array.map(item => { + const assumption = item.thing +}) + +// 🚨 Error: Cannot read property "thing" of undefined. +``` + +Illegal! Now you're a criminal. + +## The truth and only the truth + +Here's my favourite way to quickly remove all empty items: + +```javascript +const truthyArray = array.filter(Boolean) +// [{ good }, { great }] +``` + +The `filter(Boolean)` step passes each item to `Boolean()`, which coerces it to `true` or `false`. If truthy, we keep it. 😎 +``` + +**Properties**: +- Status: Claude Draft +- Type: how-to +- Destination: ["blog"] +- Description: How to remove empty values from an array +- Slug: javascript-filter-boolean + +--- + +## Safety Rules + +1. **Always use Status="Claude Draft"** - Never use any other status +2. **Never edit existing pages** - Only create new ones +3. **Show draft to user first** - Display content before creating page +4. **Link sources via relations** - Don't modify source pages + +--- + +## After Creation + +After both MCP operations complete successfully: + +1. Display both URLs: + - Writing page URL (for reviewing/editing the draft) + - Tracker page URL (confirms commit was marked as assessed) +2. Remind user they can review and edit in Notion +3. Offer to draft another or return to suggestions + +The tracker entry is automatically linked to the Writing page via the relation property. diff --git a/tools/claude/config/skills/scanning-git-for-tils/README.md b/tools/claude/config/skills/scanning-git-for-tils/README.md new file mode 100644 index 00000000..bbdfa160 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/README.md @@ -0,0 +1,151 @@ +# scanning-git-for-tils + +Scans GitHub commit history for TIL-worthy commits and drafts blog posts in Notion. + +## What It Does + +1. **Scans commits** - Fetches your recent GitHub commits via `gh` CLI +2. **Filters candidates** - Skips dependabot, merges, bumps +3. **Checks assessed** - Queries Notion to avoid re-evaluating commits +4. **Returns formatted list** - Markdown summary for Claude to evaluate +5. **Drafts TILs** - Creates Notion pages with "Claude Draft" status + +## Requirements + +- Python 3.11+ +- `uv` (for dependency management) +- `gh` CLI (authenticated to GitHub) +- `op` CLI (authenticated to 1Password for Notion token) +- Notion integration with access to: + - Writing database (for drafts) + - TIL Assessed Commits database (for tracking) + +## Development Setup + +```bash +# Install uv (if not already installed) +curl -LsSf https://astral.sh/uv/install.sh | sh + +# No package installation needed - scripts use PEP 723 inline dependencies +# Dependencies auto-install when you run scripts with `uv run` +``` + +## Running Scripts + +Scripts are self-contained with inline dependencies (PEP 723): + +```bash +# Scan for TIL candidates (last 30 days) +uv run scan_git.py + +# Scan custom time range +uv run scan_git.py 60 + +# Publish a TIL to Notion +uv run publish_til.py +``` + +## Running Tests + +```bash +# Run all tests +uv run test_pure_functions.py + +# Run with pytest for verbose output +uv run pytest test_pure_functions.py -v + +# Run specific test class +uv run pytest test_pure_functions.py::TestFormatMarkdown -v +``` + +## Linting and Type Checking + +```bash +# Run ruff (linting) +uv run --with ruff ruff check . + +# Run mypy (type checking) +uv run --with mypy --with notion-client --with pydantic --with pytest \ + mypy --python-version 3.11 . +``` + +## Project Structure + +``` +scanning-git-for-tils/ +├── git/ +│ ├── commits.py # GitHub API integration +│ ├── formatting.py # Markdown formatting utilities +│ └── types.py # Commit dataclass +├── notion/ +│ ├── blocks.py # Markdown → Notion blocks converter +│ ├── client.py # Notion client factory +│ ├── commits.py # Assessed commits tracking +│ ├── validation.py # Pydantic models for API validation +│ └── writing.py # Writing database operations +├── op/ +│ └── secrets.py # 1Password secret retrieval +├── scan_git.py # Main script: scan for TIL candidates +├── publish_til.py # Publishing script: create Notion drafts +├── test_pure_functions.py # Test suite +├── pyproject.toml # Tool configuration (ruff, mypy) +└── SKILL.md # Claude skill definition +``` + +## Dependencies + +Declared inline using [PEP 723](https://peps.python.org/pep-0723/) script metadata: + +**Runtime:** + +- `notion-client>=2.2.0` - Notion API v2025-09-03 support +- `pydantic>=2.0.0` - Runtime validation with v2 ConfigDict + +**Development:** + +- `pytest>=7.0.0` - Test framework +- `mypy>=1.0.0` - Static type checking +- `ruff>=0.1.0` - Linting and formatting + +Dependencies auto-install when running scripts with `uv run`. + +## Key Implementation Details + +### Type Safety Approach + +Uses Python with pragmatic type safety: + +- Accept `Any` at SDK boundaries (GitHub, Notion APIs) +- Use Pydantic for runtime validation immediately after API calls +- Type hints throughout internal code +- Mypy configured for pragmatic checking (not strict mode) + +### Notion API v2025-09-03 + +Uses latest Notion API patterns: + +- `data_sources.query()` instead of `databases.query()` +- `collect_paginated_api()` helper for automatic pagination +- Pydantic validation on all API responses + +### Error Handling + +- 1Password failures raise `RuntimeError` with clear messages +- Notion/GitHub API errors caught and return empty sets gracefully +- Test suite validates all error paths + +## Configuration + +Tool configuration in `pyproject.toml`: + +**Ruff:** + +- Line length: 100 +- Target: Python 3.11 +- Import sorting (I) and pyupgrade (UP) enabled + +**Mypy:** + +- Python 3.11 syntax +- Non-strict mode (pragmatic for SDK code) +- Excludes .venv/ and build directories diff --git a/tools/claude/config/skills/scanning-git-for-tils/SKILL.md b/tools/claude/config/skills/scanning-git-for-tils/SKILL.md new file mode 100644 index 00000000..30ffc235 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/SKILL.md @@ -0,0 +1,136 @@ +--- +name: scanning-git-for-tils +description: Scans GitHub commit history for commits that might make good TIL blog posts. Queries all your repos across all orgs via GitHub API. Tracks assessed commits in Notion to avoid duplicates across machines. Use when user asks for TIL ideas from their recent work. +allowed-tools: [Bash] +--- + +# Scan Git for TILs Skill + +Analyzes recent GitHub commits across all your repos to find TIL-worthy topics. + +## Notion Database + +**TIL Assessed Commits Database** +- Database ID: `928fcd9e47a84f98824790ac5a6d37ca` +- Data Source ID: `cba80148-aeef-49c9-ba45-5157668b17b3` + +Properties: +- `Commit Hash` (title): Full SHA hash +- `Message`: Commit message +- `Repo`: Repository full name +- `Commit Date` (date): When the commit was made +- `Writing` (relation): Link to Writing database if TIL was drafted +- `Assessed` (date): When commit was assessed + +## Usage + +### Step 1: Run the script + +```bash +python3 ~/.claude/skills/scanning-git-for-tils/scan_git.py [days] +``` + +**Arguments:** +- `days` (optional): Number of days to look back (default: 30) + +The script automatically: +- Fetches assessed commit hashes from Notion (via 1Password for auth) +- Fetches your commits from GitHub +- Filters out already-assessed commits + +**Output:** JSON with: +- `markdown`: Commit details for Claude to evaluate +- `new_commits`: Array of commits with hash, message, repo, date + +### Step 2: Evaluate commits + +Review the commits in the `markdown` field and identify the top 5-10 that would make good TILs. + +**Important**: The markdown shows commits with an `(index: N)` - this maps to `new_commits[N]` array which contains full commit data you'll need for publishing. + +**Good TIL candidates have:** +- Solved a non-obvious problem (gotchas, edge cases, surprising behavior) +- Learned something worth sharing (new technique, tool usage, configuration) +- Fixed a bug that others might encounter +- Set up tooling or configuration that was tricky +- Implemented a pattern that could help others + +**Skip commits that are:** +- Routine maintenance (version bumps, dependency updates, cleanup) +- Trivial changes (typos, formatting, simple renames) +- Chores without learning value (CI tweaks, file reorganization) +- Too project-specific to be useful to others + +For each selected commit: +1. Note the index number from markdown +2. Look up full commit data in `new_commits[index]` +3. Generate: + - **Suggested title**: Clear, direct (e.g., "How to X" or "Why Y happens") + - **TIL angle**: The specific learning worth documenting + +### Step 3: Display results + +Present suggestions **ranked from best to worst by TIL potential**: + +``` +📝 TIL Opportunities from Git History (last N days): + +1. **Suggested Title Here** [BEST] + - Repo: owner/repo + - Commit: abc1234 "original commit message" + - Date: 3 days ago + - Files: file1.py, file2.py + - TIL angle: What makes this worth documenting + - URL: https://github.com/... + +2. **Second Best Title** + ... + +10. **Still Worth Documenting** + ... +``` + +**Ranking criteria (highest priority first):** +1. **Broad applicability** - Will help many developers, not project-specific +2. **Non-obvious insight** - Gotcha, surprising behavior, or clever solution +3. **Recency** - More recent commits are fresher to write about +4. **Clear learning** - Easy to extract a concrete takeaway + +**Note**: Don't create tracker entries at this stage. The `publish_til.py` script will create tracker entries when drafts are actually published. This prevents duplicates and ensures only drafted commits are marked as assessed. + +## What It Returns + +JSON output example: + +```json +{ + "markdown": "Git commits from last 30 days:\n\n1. [ooloth/dotfiles] fix: properly ignore .env\n Hash: abc1234 | Date: 3 days ago\n ...", + "new_commits": [ + { + "hash": "abc1234567890...", + "message": "fix: properly ignore .env after initial commit", + "repo": "ooloth/dotfiles", + "date": "2025-01-15" + }, + ... + ] +} +``` + +## How It Works + +1. **Script fetches commits** - Queries GitHub API for your recent commits across all repos +2. **Filters obvious skips** - Removes merge commits, dependabot, already-assessed +3. **Returns all candidates** - Outputs commit details for Claude to evaluate +4. **Claude evaluates** - Reviews commits and selects top 5-10 TIL candidates +5. **Records suggestions to Notion** - Only suggested commits are marked as assessed (allows incremental backlog review) + +## Notes + +- Requires `gh` CLI installed and authenticated +- Requires `op` CLI installed and authenticated (1Password) +- Notion token stored at `op://Scripts/Notion/api-access-token` +- Searches commits authored by your GitHub username (includes any repos where you've committed) +- Script filters merge commits and dependency bot commits +- Claude evaluates remaining commits for TIL potential +- Notion sync prevents duplicate suggestions across machines diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/git/__init__.py new file mode 100644 index 00000000..860b2c77 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/git/__init__.py @@ -0,0 +1 @@ +"""Git and GitHub integration for TIL workflow.""" diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/commits.py b/tools/claude/config/skills/scanning-git-for-tils/git/commits.py new file mode 100644 index 00000000..4f88bf53 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/git/commits.py @@ -0,0 +1,184 @@ +"""GitHub commit fetching utilities.""" + +from __future__ import annotations + +import json +import subprocess +import sys +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timedelta + +from pydantic import RootModel, ValidationError + +from git.formatting import format_relative_date +from git.types import Commit + + +class CommitFilesResponse(RootModel[list[str]]): + """Validated GitHub API response for commit files (from jq filter).""" + + +def get_github_username() -> str: + """Get the authenticated GitHub username.""" + result = subprocess.run( + ["gh", "api", "user", "--jq", ".login"], + capture_output=True, + text=True, + ) + if result.returncode != 0: + return "" + return result.stdout.strip() + + +def get_commit_files(repo: str, sha: str) -> list[str]: + """Get files changed in a commit.""" + if not sha: + return [] + + result = subprocess.run( + [ + "gh", "api", f"repos/{repo}/commits/{sha}", + "--jq", "[.files[].filename]", + ], + capture_output=True, + text=True, + ) + + if result.returncode != 0: + return [] + + try: + raw_data = json.loads(result.stdout) + validated = CommitFilesResponse.model_validate(raw_data) + return validated.root + except (json.JSONDecodeError, ValidationError): + return [] + + +def get_commits(days: int, username: str) -> list[Commit]: + """Fetch commits from GitHub API.""" + since_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ") + + # Search for commits by the user + query = f"author:{username} committer-date:>={since_date[:10]}" + + result = subprocess.run( + [ + "gh", "api", "search/commits", + "-X", "GET", + "-f", f"q={query}", + "-f", "sort=committer-date", + "-f", "per_page=100", + "--jq", ".items", + ], + capture_output=True, + text=True, + ) + + if result.returncode != 0: + # Try alternative: list events + return get_commits_from_events(days, username) + + try: + items = json.loads(result.stdout) + except json.JSONDecodeError: + return [] + + # Build commits list without files first + commits = [] + for item in items: + commit_data = item.get("commit", {}) + repo = item.get("repository", {}).get("full_name", "unknown") + + commit_date = commit_data.get("committer", {}).get("date", "") + message_lines = commit_data.get("message", "").split("\n") + + commits.append(Commit( + hash=item.get("sha", "")[:7], + full_hash=item.get("sha", ""), + subject=message_lines[0], + body="\n".join(message_lines[1:]).strip(), + date=format_relative_date(commit_date), + iso_date=commit_date[:10] if commit_date else "", + repo=repo, + files=[], + url=item.get("html_url", ""), + )) + + # Fetch files in parallel (limit concurrency to avoid rate limits) + if commits: + with ThreadPoolExecutor(max_workers=5) as executor: + future_to_commit = { + executor.submit(get_commit_files, c.repo, c.full_hash): c + for c in commits + } + for future in as_completed(future_to_commit): + commit = future_to_commit[future] + try: + commit.files = future.result() + except Exception as e: + print(f"Warning: Failed to fetch files for {commit.hash}: {e}", file=sys.stderr) + commit.files = [] + + return commits + + +def get_commits_from_events(days: int, username: str) -> list[Commit]: + """Fallback: get commits from user events.""" + result = subprocess.run( + [ + "gh", "api", f"users/{username}/events", + "--jq", '[.[] | select(.type == "PushEvent")]', + ], + capture_output=True, + text=True, + ) + + if result.returncode != 0: + print( + f"Error: Failed to fetch user events via gh api " + f"(exit code {result.returncode}): {result.stderr.strip()}", + file=sys.stderr, + ) + return [] + + try: + events = json.loads(result.stdout) + except json.JSONDecodeError: + print("Error: Failed to parse JSON output from gh api user events.", file=sys.stderr) + return [] + + commits = [] + seen_hashes = set() + cutoff = datetime.now() - timedelta(days=days) + + for event in events: + created = datetime.fromisoformat(event.get("created_at", "").replace("Z", "+00:00")) + if created.replace(tzinfo=None) < cutoff: + continue + + repo = event.get("repo", {}).get("name", "unknown") + + for commit_data in event.get("payload", {}).get("commits", []): + sha = commit_data.get("sha", "") + if sha in seen_hashes: + continue + seen_hashes.add(sha) + + message = commit_data.get("message", "") + message_lines = message.split("\n") + event_date = event.get("created_at", "") + + commits.append(Commit( + hash=sha[:7], + full_hash=sha, + subject=message_lines[0], + body="\n".join(message_lines[1:]).strip(), + date=format_relative_date(event_date), + iso_date=event_date[:10] if event_date else "", + repo=repo, + files=[], # Events don't include files + url=f"https://github.com/{repo}/commit/{sha}", + )) + + return commits diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/formatting.py b/tools/claude/config/skills/scanning-git-for-tils/git/formatting.py new file mode 100644 index 00000000..353d0f6e --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/git/formatting.py @@ -0,0 +1,82 @@ +"""Git commit formatting utilities.""" + +from __future__ import annotations + +from datetime import datetime + +from git.types import Commit + + +def format_relative_date(iso_date: str) -> str: + """Convert ISO date to relative format.""" + if not iso_date: + return "unknown" + + try: + dt = datetime.fromisoformat(iso_date.replace("Z", "+00:00")) + now = datetime.now(dt.tzinfo) + diff = now - dt + + if diff.days == 0: + hours = diff.seconds // 3600 + if hours == 0: + return "just now" + return f"{hours} hour{'s' if hours != 1 else ''} ago" + elif diff.days == 1: + return "yesterday" + elif diff.days < 7: + return f"{diff.days} days ago" + elif diff.days < 30: + weeks = diff.days // 7 + return f"{weeks} week{'s' if weeks != 1 else ''} ago" + else: + months = diff.days // 30 + return f"{months} month{'s' if months != 1 else ''} ago" + except (ValueError, TypeError): + return "unknown" + + +def should_skip_commit(commit: Commit) -> bool: + """Check if commit should be filtered out entirely.""" + subject = commit.subject.lower() + + # Skip dependency bot commits + if "dependabot" in subject or ("bump" in subject and "from" in subject): + return True + + # Skip merge commits + if subject.startswith("merge"): + return True + + return False + + +def format_markdown(commits: list[Commit], days: int, new_count: int, total_count: int) -> str: + """Format commits as markdown for Claude to evaluate.""" + header = f"Git commits from last {days} days:\n" + + if total_count > 0 and new_count == 0: + return f"{header}\nNo new commits to assess ({total_count} commits already reviewed)." + + if not commits: + return f"{header}\nNo commits found. Try increasing the date range." + + lines = [header] + if new_count < total_count: + lines.append(f"({new_count} new, {total_count - new_count} already reviewed)\n") + + for i, commit in enumerate(commits, 1): + files_str = ", ".join(commit.files[:5]) if commit.files else "(no files)" + if len(commit.files) > 5: + files_str += f" (+{len(commit.files) - 5} more)" + + lines.append(f"{i}. [{commit.repo}] {commit.subject}") + lines.append(f" Hash: {commit.hash} (index: {i-1}) | Date: {commit.date}") + if commit.body: + body_preview = commit.body[:200] + "..." if len(commit.body) > 200 else commit.body + lines.append(f" Body: {body_preview}") + lines.append(f" Files: {files_str}") + lines.append(f" URL: {commit.url}") + lines.append("") + + return "\n".join(lines) diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/tests/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/git/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/tests/test_formatting.py b/tools/claude/config/skills/scanning-git-for-tils/git/tests/test_formatting.py new file mode 100644 index 00000000..7cbe859e --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/git/tests/test_formatting.py @@ -0,0 +1,285 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["pytest"] +# /// +"""Tests for git formatting utilities.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +# Add parent directory to path for imports +sys.path.insert(0, str(Path(__file__).parent.parent.parent)) + +from git.formatting import format_markdown, format_relative_date, should_skip_commit +from git.types import Commit + + +class TestFormatRelativeDate: + """Test relative date formatting.""" + + def test_formats_recent_as_hours_or_just_now(self) -> None: + from datetime import datetime + + now = datetime.now().isoformat() + "Z" + result = format_relative_date(now) + # Could be "just now" or "N hours ago" depending on timing + assert "ago" in result or result == "just now" + + def test_formats_yesterday(self) -> None: + from datetime import datetime, timedelta + + yesterday = (datetime.now() - timedelta(days=1)).isoformat() + "Z" + result = format_relative_date(yesterday) + assert result == "yesterday" + + def test_formats_days_ago(self) -> None: + result = format_relative_date("2025-01-15T12:00:00Z") + # Will be "N days ago" depending on current date + assert "ago" in result + + def test_handles_invalid_date(self) -> None: + result = format_relative_date("not-a-date") + assert result == "unknown" + + def test_handles_empty_string(self) -> None: + result = format_relative_date("") + assert result == "unknown" + + +class TestShouldSkipCommit: + """Test commit filtering logic.""" + + def test_skips_dependabot(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="Bump dependency from 1.0 to 2.0", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123", + ) + assert should_skip_commit(commit) is True + + def test_skips_bump_commits(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="bump version from 1.0 to 2.0", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123", + ) + assert should_skip_commit(commit) is True + + def test_skips_merge_commits(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="merge pull request #123", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123", + ) + assert should_skip_commit(commit) is True + + def test_keeps_normal_commits(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="fix: handle null values properly", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123", + ) + assert should_skip_commit(commit) is False + + def test_keeps_feature_commits(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="feat: add new TIL workflow", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123", + ) + assert should_skip_commit(commit) is False + + +class TestFormatMarkdown: + """Test markdown formatting for commits.""" + + def test_formats_empty_list(self) -> None: + result = format_markdown([], 30, 0, 0) + assert "Git commits from last 30 days:" in result + assert "No commits found" in result + + def test_formats_all_already_reviewed(self) -> None: + result = format_markdown([], 30, 0, 5) + assert "Git commits from last 30 days:" in result + assert "No new commits to assess" in result + assert "5 commits already reviewed" in result + + def test_formats_single_commit_basic(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123456789", + subject="feat: add new feature", + body="", + date="2 days ago", + iso_date="2025-01-15", + repo="owner/repo", + files=["src/main.py"], + url="https://github.com/owner/repo/commit/abc123456789", + ) + result = format_markdown([commit], 30, 1, 1) + + assert "Git commits from last 30 days:" in result + assert "1. [owner/repo] feat: add new feature" in result + assert "Hash: abc1234 (index: 0) | Date: 2 days ago" in result + assert "Files: src/main.py" in result + assert "URL: https://github.com/owner/repo/commit/abc123456789" in result + + def test_formats_commit_with_body(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123456789", + subject="fix: handle edge case", + body="This fixes an issue where null values weren't handled properly.", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=["src/handler.py"], + url="https://github.com/owner/repo/commit/abc123456789", + ) + result = format_markdown([commit], 30, 1, 1) + + assert "Body: This fixes an issue where null values weren't handled properly." in result + + def test_formats_commit_with_long_body(self) -> None: + long_body = "a" * 250 + commit = Commit( + hash="abc1234", + full_hash="abc123456789", + subject="feat: major refactor", + body=long_body, + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=["src/main.py"], + url="https://github.com/owner/repo/commit/abc123456789", + ) + result = format_markdown([commit], 30, 1, 1) + + assert "Body: " + "a" * 200 + "..." in result + assert len([line for line in result.split("\n") if "Body:" in line][0]) < 220 + + def test_formats_commit_with_no_files(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123456789", + subject="chore: update docs", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=[], + url="https://github.com/owner/repo/commit/abc123456789", + ) + result = format_markdown([commit], 30, 1, 1) + + assert "Files: (no files)" in result + + def test_formats_commit_with_many_files(self) -> None: + files = [f"file{i}.py" for i in range(10)] + commit = Commit( + hash="abc1234", + full_hash="abc123456789", + subject="refactor: reorganize code", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=files, + url="https://github.com/owner/repo/commit/abc123456789", + ) + result = format_markdown([commit], 30, 1, 1) + + # Should show first 5 files + assert "file0.py, file1.py, file2.py, file3.py, file4.py" in result + # Should indicate there are more + assert "(+5 more)" in result + # Should NOT show file5 or later + assert "file5.py" not in result + + def test_formats_multiple_commits(self) -> None: + commits = [ + Commit( + hash="abc1234", + full_hash="abc123", + subject="First commit", + body="", + date="2 days ago", + iso_date="2025-01-15", + repo="owner/repo1", + files=["a.py"], + url="https://github.com/owner/repo1/commit/abc123", + ), + Commit( + hash="def5678", + full_hash="def567", + subject="Second commit", + body="", + date="yesterday", + iso_date="2025-01-16", + repo="owner/repo2", + files=["b.py"], + url="https://github.com/owner/repo2/commit/def567", + ), + ] + result = format_markdown(commits, 7, 2, 2) + + assert "1. [owner/repo1] First commit" in result + assert "Hash: abc1234 (index: 0)" in result + assert "2. [owner/repo2] Second commit" in result + assert "Hash: def5678 (index: 1)" in result + + def test_shows_review_status_when_some_already_reviewed(self) -> None: + commit = Commit( + hash="abc1234", + full_hash="abc123", + subject="New commit", + body="", + date="yesterday", + iso_date="2025-01-15", + repo="owner/repo", + files=["a.py"], + url="https://github.com/owner/repo/commit/abc123", + ) + result = format_markdown([commit], 30, 1, 5) + + assert "Git commits from last 30 days:" in result + assert "(1 new, 4 already reviewed)" in result + + +if __name__ == "__main__": + import pytest + + sys.exit(pytest.main([__file__, "-v"])) diff --git a/tools/claude/config/skills/scanning-git-for-tils/git/types.py b/tools/claude/config/skills/scanning-git-for-tils/git/types.py new file mode 100644 index 00000000..ca0fc082 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/git/types.py @@ -0,0 +1,20 @@ +"""Git commit types and data structures.""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass +class Commit: + """A git commit with metadata.""" + + hash: str # Short hash (7 chars) + full_hash: str # Full SHA + subject: str # First line of commit message + body: str # Remaining lines of commit message + date: str # Relative date (e.g., "2 days ago") + iso_date: str # ISO date (YYYY-MM-DD) + repo: str # Repository name (owner/repo) + files: list[str] # Files changed + url: str # GitHub URL diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/notion/__init__.py new file mode 100644 index 00000000..59bee967 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/__init__.py @@ -0,0 +1 @@ +"""Notion integration for TIL workflow.""" diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/blocks.py b/tools/claude/config/skills/scanning-git-for-tils/notion/blocks.py new file mode 100644 index 00000000..0c637161 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/blocks.py @@ -0,0 +1,167 @@ +"""Notion block conversion utilities.""" + +from __future__ import annotations + + +def _map_language_alias(language: str) -> str: + """Map common language names to Notion's expected values.""" + lang_map = { + "": "plain text", + "js": "javascript", + "ts": "typescript", + "py": "python", + "sh": "shell", + "bash": "shell", + "zsh": "shell", + } + return lang_map.get(language, language) or "plain text" + + +def _create_code_block(lines: list[str], start_index: int) -> tuple[dict, int]: + """Create a code block from markdown fenced code. + + Returns: (block dict, new index after closing ```) + """ + language = lines[start_index].strip()[3:].strip() + language = _map_language_alias(language) + + code_lines = [] + i = start_index + 1 + + # Collect all lines until closing ``` + while i < len(lines): + if lines[i].strip().startswith("```"): + break + code_lines.append(lines[i]) + i += 1 + + code_content = "\n".join(code_lines) + block = { + "type": "code", + "code": { + "rich_text": [{"type": "text", "text": {"content": code_content}}], + "language": language, + }, + } + + return block, i + 1 + + +def _create_heading_block(line: str) -> dict | None: + """Create a heading block from markdown heading syntax. + + Returns: block dict or None if not a heading + """ + if line.startswith("### "): + return { + "type": "heading_3", + "heading_3": { + "rich_text": [{"type": "text", "text": {"content": line[4:]}}] + }, + } + elif line.startswith("## "): + return { + "type": "heading_2", + "heading_2": { + "rich_text": [{"type": "text", "text": {"content": line[3:]}}] + }, + } + elif line.startswith("# "): + return { + "type": "heading_1", + "heading_1": { + "rich_text": [{"type": "text", "text": {"content": line[2:]}}] + }, + } + return None + + +def _create_list_item_block(line: str) -> dict | None: + """Create a list item block from markdown list syntax. + + Returns: block dict or None if not a list item + """ + if line.startswith("- "): + return { + "type": "bulleted_list_item", + "bulleted_list_item": { + "rich_text": [{"type": "text", "text": {"content": line[2:]}}] + }, + } + elif len(line) > 2 and line[0].isdigit() and line[1:3] == ". ": + return { + "type": "numbered_list_item", + "numbered_list_item": { + "rich_text": [{"type": "text", "text": {"content": line[3:]}}] + }, + } + return None + + +def _create_paragraph_block(line: str) -> dict: + """Create a paragraph block from text content.""" + if not line.strip(): + # Empty line - create empty paragraph for spacing + return {"type": "paragraph", "paragraph": {"rich_text": []}} + else: + # Regular paragraph with content + return { + "type": "paragraph", + "paragraph": {"rich_text": [{"type": "text", "text": {"content": line}}]}, + } + + +def markdown_to_blocks(content: str) -> list: + """Convert markdown content to Notion blocks. + + Handles: headings, code blocks, lists, paragraphs, inline code. + """ + blocks = [] + lines = content.split("\n") + i = 0 + + while i < len(lines): + line = lines[i] + + # Code blocks + if line.strip().startswith("```"): + block, new_index = _create_code_block(lines, i) + blocks.append(block) + i = new_index + continue + + # Headings + heading_block = _create_heading_block(line) + if heading_block: + blocks.append(heading_block) + i += 1 + continue + + # List items + list_block = _create_list_item_block(line) + if list_block: + blocks.append(list_block) + i += 1 + continue + + # Paragraphs (including empty lines) + blocks.append(_create_paragraph_block(line)) + i += 1 + + return blocks + + +def extract_page_id(url: str) -> str: + """Extract page ID from Notion URL.""" + # URL format: https://www.notion.so/Page-Title- + # or https://www.notion.so/ + if not url: + return "" + parts = url.rstrip("/").split("-") + if parts: + # Last part after final dash, or the whole path segment + candidate = parts[-1].split("/")[-1] + # Remove any query params + candidate = candidate.split("?")[0] + return candidate + return "" diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/client.py b/tools/claude/config/skills/scanning-git-for-tils/notion/client.py new file mode 100644 index 00000000..3bab67de --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/client.py @@ -0,0 +1,19 @@ +"""Notion API client utilities.""" + +from __future__ import annotations + +from notion_client import Client + +from op.secrets import OP_NOTION_TOKEN_PATH, get_op_secret + + +def get_notion_client() -> Client: + """Create authenticated Notion client. + + Raises: + RuntimeError: If 1Password secret retrieval fails. + """ + from notion_client import Client + + token = get_op_secret(OP_NOTION_TOKEN_PATH) + return Client(auth=token) diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/commits.py b/tools/claude/config/skills/scanning-git-for-tils/notion/commits.py new file mode 100644 index 00000000..d3874b9a --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/commits.py @@ -0,0 +1,111 @@ +"""Notion assessed commits tracking utilities.""" + +from __future__ import annotations + +from datetime import date + +from notion_client import Client + +from notion.validation import ( + AssessedCommitPage, + CommitInput, + NotionPageResponse, + NotionQueryResponse, +) +from op.secrets import OP_NOTION_TOKEN_PATH, get_op_secret + +# Notion database IDs +ASSESSED_COMMITS_DATA_SOURCE_ID = "cba80148-aeef-49c9-ba45-5157668b17b3" +NOTION_ASSESSED_COMMITS_DB = "928fcd9e47a84f98824790ac5a6d37ca" + + +def get_assessed_commits_from_notion() -> set[str]: + """Fetch all assessed commit hashes from Notion database.""" + from notion_client.helpers import collect_paginated_api + + try: + token = get_op_secret(OP_NOTION_TOKEN_PATH) + notion = Client(auth=token) + except Exception: + return set() + + try: + # Use helper to automatically handle pagination (Notion API v2025-09-03) + raw_pages = collect_paginated_api( + notion.data_sources.query, + data_source_id=ASSESSED_COMMITS_DATA_SOURCE_ID, + ) + + # Extract commit hashes from results + assessed_hashes = set() + for raw_page in raw_pages: + # Validate page structure and extract commit hash + page = AssessedCommitPage.model_validate(raw_page) + commit_hash = page.properties.commit_hash.text + if commit_hash: + assessed_hashes.add(commit_hash) + + return assessed_hashes + + except Exception: + return set() + + +def find_existing_tracker_entry(notion: Client, commit_hash: str) -> str: + """Check if tracker entry already exists for this commit. Returns page ID if found.""" + try: + raw_response = notion.data_sources.query( + data_source_id=ASSESSED_COMMITS_DATA_SOURCE_ID, + filter={"property": "Commit Hash", "title": {"equals": commit_hash}}, + ) + # Validate response immediately + response = NotionQueryResponse.model_validate(raw_response) + if response.results: + # Access validated page ID via dot notation + return response.results[0].id + except Exception: + pass + + return "" + + +def update_tracker_entry(notion: Client, page_id: str, writing_page_id: str) -> str: + """Update existing tracker entry to link to Writing page. Returns page URL.""" + try: + response = notion.pages.update( + page_id=page_id, + properties={ + "Writing": {"relation": [{"id": writing_page_id}]}, + "Assessed": {"date": {"start": date.today().isoformat()}}, + }, + ) + # Parse response immediately to validate structure + page = NotionPageResponse.model_validate(response) + return page.url + except Exception as e: + raise Exception(f"Failed to update tracker: {e}") + + +def create_tracker_entry(notion: Client, commit: CommitInput, writing_page_id: str) -> str: + """Create an entry in TIL Assessed Commits and link to Writing page. Returns page URL.""" + + properties = { + "Commit Hash": {"title": [{"type": "text", "text": {"content": commit.hash}}]}, + "Message": {"rich_text": [{"type": "text", "text": {"content": commit.message[:2000]}}]}, + "Repo": {"rich_text": [{"type": "text", "text": {"content": commit.repo}}]}, + "Assessed": {"date": {"start": date.today().isoformat()}}, + "Writing": {"relation": [{"id": writing_page_id}]}, + } + + # Only add Commit Date if present (None breaks Notion API) + if commit.date: + properties["Commit Date"] = {"date": {"start": commit.date}} + + response = notion.pages.create( + parent={"data_source_id": ASSESSED_COMMITS_DATA_SOURCE_ID}, + properties=properties, + ) + + # Parse response immediately to validate structure + page = NotionPageResponse.model_validate(response) + return page.url diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/tests/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/notion/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_blocks.py b/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_blocks.py new file mode 100644 index 00000000..969cd190 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_blocks.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["pytest"] +# /// +"""Tests for Notion blocks utilities.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +# Add parent directory to path for imports +sys.path.insert(0, str(Path(__file__).parent.parent.parent)) + +from notion.blocks import extract_page_id, markdown_to_blocks + + +class TestExtractPageId: + """Test Notion URL page ID extraction.""" + + def test_extracts_from_standard_url(self) -> None: + url = "https://www.notion.so/Page-Title-abc123def456" + result = extract_page_id(url) + assert result == "abc123def456" + + def test_extracts_from_url_with_query_params(self) -> None: + url = "https://www.notion.so/Page-Title-abc123def456?v=xyz" + result = extract_page_id(url) + assert result == "abc123def456" + + def test_extracts_from_short_url(self) -> None: + url = "https://notion.so/abc123def456" + result = extract_page_id(url) + assert result == "abc123def456" + + def test_handles_trailing_slash(self) -> None: + url = "https://www.notion.so/Page-Title-abc123def456/" + result = extract_page_id(url) + assert result == "abc123def456" + + def test_handles_empty_string(self) -> None: + result = extract_page_id("") + assert result == "" + + def test_extracts_uuid_with_dashes(self) -> None: + # Notion IDs can have dashes in UUID format + url = "https://www.notion.so/12345678-90ab-cdef-1234-567890abcdef" + result = extract_page_id(url) + # Should get the whole UUID including trailing segment + assert len(result) > 0 + + +class TestMarkdownToBlocks: + """Test markdown to Notion blocks conversion.""" + + def test_converts_code_blocks(self) -> None: + markdown = "```python\nprint('hello')\n```" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 1 + assert blocks[0]["type"] == "code" + assert blocks[0]["code"]["language"] == "python" + assert blocks[0]["code"]["rich_text"][0]["text"]["content"] == "print('hello')" + + def test_maps_language_aliases(self) -> None: + markdown = "```js\nconsole.log('test')\n```" + blocks = markdown_to_blocks(markdown) + + assert blocks[0]["code"]["language"] == "javascript" + + def test_converts_headings(self) -> None: + markdown = "# H1\n## H2\n### H3" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 3 + assert blocks[0]["type"] == "heading_1" + assert blocks[1]["type"] == "heading_2" + assert blocks[2]["type"] == "heading_3" + + def test_converts_bullet_lists(self) -> None: + markdown = "- Item 1\n- Item 2" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 2 + assert blocks[0]["type"] == "bulleted_list_item" + assert blocks[0]["bulleted_list_item"]["rich_text"][0]["text"]["content"] == "Item 1" + + def test_converts_numbered_lists(self) -> None: + markdown = "1. First\n2. Second" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 2 + assert blocks[0]["type"] == "numbered_list_item" + assert blocks[1]["type"] == "numbered_list_item" + + def test_converts_paragraphs(self) -> None: + markdown = "This is a paragraph" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 1 + assert blocks[0]["type"] == "paragraph" + assert blocks[0]["paragraph"]["rich_text"][0]["text"]["content"] == "This is a paragraph" + + def test_handles_empty_lines(self) -> None: + markdown = "Line 1\n\nLine 2" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 3 + assert blocks[1]["type"] == "paragraph" + assert blocks[1]["paragraph"]["rich_text"] == [] + + def test_handles_multiline_code_blocks(self) -> None: + markdown = "```python\nline1\nline2\nline3\n```" + blocks = markdown_to_blocks(markdown) + + assert len(blocks) == 1 + assert "line1\nline2\nline3" in blocks[0]["code"]["rich_text"][0]["text"]["content"] + + +if __name__ == "__main__": + import pytest + + sys.exit(pytest.main([__file__, "-v"])) diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_commits.py b/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_commits.py new file mode 100644 index 00000000..071dd092 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/tests/test_commits.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["pytest", "notion-client", "pydantic"] +# /// +"""Tests for Notion commits tracking.""" + +from __future__ import annotations + +import sys +from pathlib import Path +from unittest.mock import patch + +# Add parent directory to path for imports +sys.path.insert(0, str(Path(__file__).parent.parent.parent)) + +from notion.commits import get_assessed_commits_from_notion + + +def make_notion_page(commit_hash: str) -> dict: + """Helper: create a mock Notion page with a commit hash.""" + return { + "id": f"page-{commit_hash}", + "url": f"https://notion.so/page-{commit_hash}", + "properties": {"Commit Hash": {"title": [{"plain_text": commit_hash}]}}, + } + + +def make_notion_response( + hashes: list[str], has_more: bool = False, next_cursor: str | None = None +) -> dict: + """Helper: create a mock Notion SDK response.""" + return { + "results": [make_notion_page(h) for h in hashes], + "has_more": has_more, + "next_cursor": next_cursor, + } + + +def mock_collect_paginated_api(pages: list[dict]) -> list[dict]: + """Helper: mock collect_paginated_api to return all pages as a flat list.""" + all_results: list[dict] = [] + for page_response in pages: + all_results.extend(page_response["results"]) + return all_results + + +class TestGetAssessedCommitsFromNotion: + """Test fetching assessed commits from Notion.""" + + def test_returns_empty_set_when_no_token(self) -> None: + with patch("notion.commits.get_op_secret", side_effect=RuntimeError("Failed")): + result = get_assessed_commits_from_notion() + assert result == set() + + def test_returns_commit_hashes_from_single_page(self) -> None: + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client"), + patch("notion_client.helpers.collect_paginated_api") as mock_paginate, + ): + pages = [make_notion_response(["abc123", "def456", "ghi789"])] + mock_paginate.return_value = mock_collect_paginated_api(pages) + + result = get_assessed_commits_from_notion() + assert result == {"abc123", "def456", "ghi789"} + + def test_handles_pagination(self) -> None: + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client"), + patch("notion_client.helpers.collect_paginated_api") as mock_paginate, + ): + # First page with more results + first_response = make_notion_response( + ["abc123", "def456"], has_more=True, next_cursor="cursor-1" + ) + # Second page, final + second_response = make_notion_response(["ghi789", "jkl012"], has_more=False) + + # collect_paginated_api handles pagination internally, returns all results + pages = [first_response, second_response] + mock_paginate.return_value = mock_collect_paginated_api(pages) + + result = get_assessed_commits_from_notion() + assert result == {"abc123", "def456", "ghi789", "jkl012"} + + def test_handles_client_error_gracefully(self) -> None: + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client") as MockClient, + ): + MockClient.side_effect = Exception("Connection error") + + result = get_assessed_commits_from_notion() + assert result == set() + + def test_handles_query_error_gracefully(self) -> None: + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client"), + patch("notion_client.helpers.collect_paginated_api") as mock_paginate, + ): + mock_paginate.side_effect = Exception("Query error") + + result = get_assessed_commits_from_notion() + assert result == set() + + def test_skips_pages_without_commit_hash(self) -> None: + with ( + patch("notion.commits.get_op_secret", return_value="fake-token"), + patch("notion_client.Client"), + patch("notion_client.helpers.collect_paginated_api") as mock_paginate, + ): + response = { + "results": [ + make_notion_page("abc123"), + { # Empty title + "id": "page-empty", + "url": "https://notion.so/page-empty", + "properties": {"Commit Hash": {"title": []}}, + }, + make_notion_page("def456"), + ], + "has_more": False, + "next_cursor": None, + } + + mock_paginate.return_value = mock_collect_paginated_api([response]) + + result = get_assessed_commits_from_notion() + assert result == {"abc123", "def456"} + + +if __name__ == "__main__": + import pytest + + sys.exit(pytest.main([__file__, "-v"])) diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/validation.py b/tools/claude/config/skills/scanning-git-for-tils/notion/validation.py new file mode 100644 index 00000000..a7e4bf40 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/validation.py @@ -0,0 +1,70 @@ +"""Pydantic models for validating Notion API requests and responses.""" + +from __future__ import annotations + +from pydantic import BaseModel, ConfigDict, Field + + +class CommitInput(BaseModel): + """Commit metadata for creating tracker entries.""" + + hash: str = Field(..., min_length=1) + message: str = Field(..., min_length=1) + repo: str = Field(..., min_length=1) + date: str | None = None + + +class NotionRichText(BaseModel): + """Rich text content in Notion.""" + + model_config = ConfigDict(extra="ignore") + + plain_text: str + + +class NotionTitleProperty(BaseModel): + """Title property structure in Notion.""" + + model_config = ConfigDict(extra="ignore") + + title: list[NotionRichText] + + @property + def text(self) -> str: + """Extract first title text or empty string.""" + return self.title[0].plain_text if self.title else "" + + +class AssessedCommitProperties(BaseModel): + """Properties for TIL Assessed Commits database pages.""" + + model_config = ConfigDict(extra="ignore") + + commit_hash: NotionTitleProperty = Field(alias="Commit Hash") + + +class AssessedCommitPage(BaseModel): + """Page from TIL Assessed Commits database.""" + + model_config = ConfigDict(extra="ignore") + + id: str + url: str + properties: AssessedCommitProperties + + +class NotionPageResponse(BaseModel): + """Validated Notion page response (create/update).""" + + model_config = ConfigDict(extra="ignore") + + url: str + id: str + + +class NotionQueryResponse(BaseModel): + """Validated Notion query response for assessed commits.""" + + model_config = ConfigDict(extra="ignore") + + results: list[AssessedCommitPage] diff --git a/tools/claude/config/skills/scanning-git-for-tils/notion/writing.py b/tools/claude/config/skills/scanning-git-for-tils/notion/writing.py new file mode 100644 index 00000000..35b82de3 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/notion/writing.py @@ -0,0 +1,32 @@ +"""Notion Writing database utilities.""" + +from __future__ import annotations + +from notion_client import Client + +from notion.blocks import markdown_to_blocks +from notion.validation import NotionPageResponse + +# Notion database IDs +WRITING_DATA_SOURCE_ID = "c296db5b-d2f1-44d4-abc6-f9a05736b143" + + +def create_writing_page(notion: Client, title: str, content: str, slug: str, description: str) -> str: + """Create a TIL draft in the Writing database. Returns page URL.""" + + response = notion.pages.create( + parent={"data_source_id": WRITING_DATA_SOURCE_ID}, + properties={ + "Title": {"title": [{"type": "text", "text": {"content": title}}]}, + "Status": {"status": {"name": "Claude Draft"}}, + "Type": {"select": {"name": "how-to"}}, + "Destination": {"multi_select": [{"name": "blog"}]}, + "Description": {"rich_text": [{"type": "text", "text": {"content": description}}]}, + "Slug": {"rich_text": [{"type": "text", "text": {"content": slug}}]}, + }, + children=markdown_to_blocks(content), + ) + + # Validate response and extract URL + page = NotionPageResponse.model_validate(response) + return page.url diff --git a/tools/claude/config/skills/scanning-git-for-tils/op/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/op/__init__.py new file mode 100644 index 00000000..ea36a8c7 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/op/__init__.py @@ -0,0 +1,3 @@ +"""1Password integration utilities.""" + +from __future__ import annotations diff --git a/tools/claude/config/skills/scanning-git-for-tils/op/secrets.py b/tools/claude/config/skills/scanning-git-for-tils/op/secrets.py new file mode 100644 index 00000000..c012c5b5 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/op/secrets.py @@ -0,0 +1,17 @@ +"""1Password secret retrieval utilities.""" + +from __future__ import annotations + +import subprocess + +OP_NOTION_TOKEN_PATH = "op://Scripts/Notion/api-access-token" + + +def get_op_secret(path: str) -> str: + """Fetch a secret from 1Password.""" + result = subprocess.run(["op", "read", path], capture_output=True, text=True) + + if result.returncode != 0: + raise RuntimeError(f"Failed to retrieve secret from 1Password: {result.stderr.strip()}") + + return result.stdout.strip() diff --git a/tools/claude/config/skills/scanning-git-for-tils/publish_til.py b/tools/claude/config/skills/scanning-git-for-tils/publish_til.py new file mode 100644 index 00000000..e2209d1a --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/publish_til.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["notion-client", "pydantic"] +# /// +""" +Publish a TIL draft to Notion and update the tracker. + +Usage: + echo '' | uv run publish_til.py + +Input (JSON via stdin): + { + "title": "TIL Title", + "content": "Markdown content", + "slug": "til-slug", + "description": "One-line summary", + "commit": { + "hash": "full-sha-hash", + "message": "commit message", + "repo": "owner/repo", + "date": "2025-01-15" + } + } + +Output (JSON): + { + "writing_url": "https://notion.so/...", + "tracker_url": "https://notion.so/..." + } + +Requires: + - op CLI installed and authenticated (1Password) + - uv (for dependency management) +""" + +from __future__ import annotations + +import json +import sys +from dataclasses import asdict, dataclass + +from pydantic import BaseModel, Field, ValidationError + +from notion.blocks import extract_page_id +from notion.client import get_notion_client +from notion.commits import ( + create_tracker_entry, + find_existing_tracker_entry, + update_tracker_entry, +) +from notion.validation import CommitInput +from notion.writing import create_writing_page + + +class PublishTilInput(BaseModel): + """Input for publishing a TIL to Notion.""" + + title: str = Field(..., min_length=1, max_length=2000) + content: str = Field(..., min_length=1) + slug: str = Field(..., min_length=1) + description: str = Field(..., min_length=1, max_length=2000) + commit: CommitInput + + +@dataclass +class PublishTilOutput: + """Output from publishing a TIL to Notion.""" + + writing_url: str + tracker_url: str + + +def main() -> None: + # Read and validate JSON input from stdin + try: + raw_input = json.loads(sys.stdin.read()) + input_data = PublishTilInput.model_validate(raw_input) + except json.JSONDecodeError as e: + print(json.dumps({"error": f"Invalid JSON input: {e}"})) + sys.exit(1) + except ValidationError as e: + print(json.dumps({"error": f"Validation error: {e}"})) + sys.exit(1) + + try: + # Create Notion client + notion = get_notion_client() + + # Create Writing page + writing_url = create_writing_page( + notion, + input_data.title, + input_data.content, + input_data.slug, + input_data.description, + ) + + if not writing_url: + print(json.dumps({"error": "Failed to create Writing page"})) + sys.exit(1) + + # Extract page ID for relation + writing_page_id = extract_page_id(writing_url) + + # Check if tracker entry already exists + existing_tracker_id = find_existing_tracker_entry(notion, input_data.commit.hash) + + if existing_tracker_id: + # Update existing entry with Writing relation + tracker_url = update_tracker_entry(notion, existing_tracker_id, writing_page_id) + else: + # Create new tracker entry with relation to Writing page + tracker_url = create_tracker_entry(notion, input_data.commit, writing_page_id) + + # Output results as dataclass + output = PublishTilOutput( + writing_url=writing_url, + tracker_url=tracker_url, + ) + print(json.dumps(asdict(output), indent=2)) + + except Exception as e: + print(json.dumps({"error": str(e)})) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/tools/claude/config/skills/scanning-git-for-tils/pyproject.toml b/tools/claude/config/skills/scanning-git-for-tils/pyproject.toml new file mode 100644 index 00000000..2c9e9864 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/pyproject.toml @@ -0,0 +1,38 @@ +[project] +name = "scanning-git-for-tils" +version = "0.1.0" +requires-python = ">=3.11" + +[tool.ruff] +line-length = 100 +target-version = "py311" + +[tool.ruff.lint] +select = [ + "I", # import sorting + "UP", # pyupgrade: modernize syntax (e.g., list[str] instead of List[str]) +] + +[tool.mypy] +python_version = "3.11" +strict = false + +# Enforce type hints on all functions (including return types) +disallow_untyped_defs = true + +# Require explicit Optional (str | None, not str = None) +no_implicit_optional = true + +# Keep type: ignore comments clean +warn_unused_ignores = true + +# Warn when returning Any from typed functions +# We handle this by validating with Pydantic immediately after API calls +warn_return_any = true + +exclude = [ + "^.venv/", + "^venv/", + "^build/", + "^dist/", +] diff --git a/tools/claude/config/skills/scanning-git-for-tils/scan_git.py b/tools/claude/config/skills/scanning-git-for-tils/scan_git.py new file mode 100755 index 00000000..1e47cbe0 --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/scan_git.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["notion-client", "pydantic"] +# /// +""" +Scan GitHub commit history for TIL-worthy commits. + +Usage: + python3 scan_git.py [days] + +Arguments: + days: Number of days to look back (default: 30) + +Output: + JSON with commits for Claude to evaluate + +Requires: + - gh CLI installed and authenticated + - op CLI installed and authenticated (1Password) + - uv (for dependency management) +""" + +from __future__ import annotations + +import json +import sys +from dataclasses import asdict, dataclass + +from git.commits import get_commits, get_github_username +from git.formatting import format_markdown, should_skip_commit +from notion.commits import get_assessed_commits_from_notion + + +@dataclass +class CommitSummary: + """Summary of a commit for TIL evaluation.""" + + hash: str + message: str + repo: str + date: str + + +@dataclass +class ScanGitOutput: + """Output from scanning git commits.""" + + markdown: str + new_commits: list[CommitSummary] + + +def main() -> None: + # Parse arguments + days = 30 + if len(sys.argv) > 1: + try: + days = int(sys.argv[1]) + except ValueError: + pass + + # Fetch assessed commits from Notion + assessed_hashes = get_assessed_commits_from_notion() + + # Get GitHub username + username = get_github_username() + if not username: + print( + json.dumps( + { + "error": "Could not get GitHub username. Is `gh` authenticated?", + "markdown": "", + "new_commits": [], + } + ) + ) + sys.exit(1) + + # Get commits + commits = get_commits(days, username) + total_count = len(commits) + + if not commits: + output = ScanGitOutput(markdown=format_markdown([], days, 0, 0), new_commits=[]) + print(json.dumps(asdict(output))) + sys.exit(0) + + # Filter out already assessed commits and skippable commits + new_commits = [ + c + for c in commits + if c.full_hash not in assessed_hashes and not should_skip_commit(c) + ] + new_count = len(new_commits) + + # Prepare output + output = ScanGitOutput( + markdown=format_markdown(new_commits, days, new_count, total_count), + new_commits=[ + CommitSummary( + hash=c.full_hash, message=c.subject, repo=c.repo, date=c.iso_date + ) + for c in new_commits + ], + ) + + print(json.dumps(asdict(output), indent=2)) + + +if __name__ == "__main__": + main() diff --git a/tools/claude/config/skills/scanning-git-for-tils/tests/__init__.py b/tools/claude/config/skills/scanning-git-for-tils/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tools/claude/config/skills/scanning-git-for-tils/uv.lock b/tools/claude/config/skills/scanning-git-for-tils/uv.lock new file mode 100644 index 00000000..d2a6b1ec --- /dev/null +++ b/tools/claude/config/skills/scanning-git-for-tils/uv.lock @@ -0,0 +1,8 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[[package]] +name = "scanning-git-for-tils" +version = "0.1.0" +source = { virtual = "." } diff --git a/tools/claude/config/skills/scanning-notion-for-tils/SKILL.md b/tools/claude/config/skills/scanning-notion-for-tils/SKILL.md new file mode 100644 index 00000000..5a458c50 --- /dev/null +++ b/tools/claude/config/skills/scanning-notion-for-tils/SKILL.md @@ -0,0 +1,130 @@ +--- +name: scanning-notion-for-tils +description: [UNDER DEVELOPMENT - DO NOT USE] Searches the Notion Writing database for unpublished items that could become TIL posts. This skill is not yet integrated with the publishing workflow. +--- + +# Scan Notion for TILs Skill + +**Status: Under Development** - This skill is not ready for use. Focus on git-only workflow for now. + +Finds unpublished Writing items that could become TIL blog posts. + +## Writing Database + +- Database ID: `eb0cbc7a-4fe4-4954-99bd-94c1bf861469` +- Data Source ID: `c296db5b-d2f1-44d4-abc6-f9a05736b143` + +## Usage + +### Step 1: Search for blog-destined items + +Use `mcp__notion__notion-search` to find Writing items: + +``` +Search the Writing database for items with: +- Destination includes "blog" +- Status NOT in [Published, Paused, Archived, Migrated to content repo] +``` + +### Step 2: Filter out already-assessed items + +Skip items that have a Writing relation pointing to a page with Status = "Claude Draft". + +This means the item already has a TIL draft created for it. + +### Step 3: Fetch candidate items + +For each remaining item, use `mcp__notion__notion-fetch` to get: +- Full title and description +- Status and Type +- Related Research, Questions, and Topics +- Last edited date +- Page content (to assess depth) +- Writing relations (to check for Claude Draft links) + +### Step 4: Score and categorize + +**Ready to draft** (have enough content): +- Type = "how-to" (highest priority) +- Have linked Research or Questions (indicates depth) +- Have substantial content already +- Short/focused topics (TIL-appropriate) + +**Need development help** (good topic, needs work): +- Title only or minimal content +- No Research or Questions linked yet +- Topic is clear but needs exploration + +Both categories are valid suggestions - offer to draft TILs for ready items, offer to help develop items that need work. + +### Step 5: Format output + +Present suggestions in this format: + +``` +📝 TIL Opportunities from Notion Backlog: + +🟢 READY TO DRAFT: + +1. **"Make TS understand Array.filter by using type predicates"** + - Status: Drafting | Type: how-to + - Last edited: 2 months ago + - Has: 2 Research links, 1 Question + - Content: ~200 words already written + - TIL angle: Type predicates let TS narrow filtered arrays + - URL: https://www.notion.so/... + +🟡 NEED DEVELOPMENT: + +2. **"How to filter a JS array with async/await"** + - Status: New | Type: how-to + - Last edited: 1 year ago + - Has: 1 Research link + - Content: Title only + - Suggestion: Research async filtering patterns, find good examples + - URL: https://www.notion.so/... + +Select a number to: +- Draft a TIL (for ready items) +- Help develop the topic (for items needing work) +``` + +## What to Look For + +Good TIL candidates from Notion: + +1. **How-to items** - Already tagged as instructional content +2. **Items with Research links** - Have supporting material to draw from +3. **Items with Questions** - Answered a real question worth sharing +4. **Recently edited** - Topic is fresh, easier to write about +5. **Partially drafted** - Already has content to build on + +## TIL Angle Generation + +Based on the item's content, suggest a TIL angle: + +- **For code patterns**: "How to [do X] using [technique]" +- **For gotchas**: "Why [X] doesn't work and what to do instead" +- **For configuration**: "Setting up [tool] for [use case]" +- **For debugging**: "How to diagnose [problem]" + +## Linking Drafts Back + +**IMPORTANT: Never edit existing Writing items. Always create new pages in the Writing database.** + +When working with a Notion item (drafting or developing): +1. Create a NEW page in the Writing database with Status = "Claude Draft" +2. Put all content/improvements in the new Writing page +3. Link the new draft TO the source item via Writing relation +4. This marks the source as "assessed" for future scans + +The original item stays untouched - it's a reference, not something to modify. All Claude's work goes into new Writing database pages. + +## Notes + +- Only scans items with Destination = "blog" +- Skips items with Status in Published, Paused, Archived, Migrated +- Skips items that already have a Claude Draft linked via Writing relation +- Items needing development get help, not skipped +- TIL angles are suggestions based on title/content - refine as needed +- User may want to consolidate multiple related items into one TIL