feat: add ScoutQA Testing Skill #604
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new skill called "scoutqa-test" that enables AI-powered exploratory web testing using the ScoutQA CLI. The skill is designed to help developers test web applications for issues like accessibility problems, form validation, and user flows.
Changes:
- Adds a new skill folder
skills/scoutqa-test/with a SKILL.md file - Provides comprehensive documentation on using ScoutQA for automated testing
- Includes examples for various testing scenarios (smoke tests, accessibility audits, e-commerce flows)
| description: | | ||
| This skill should be used when the user asks to "test this website", "run exploratory testing", "check for accessibility issues", "verify the login flow works", "find bugs on this page", or requests automated QA testing. Triggers on web application testing scenarios including smoke tests, accessibility audits, e-commerce flows, and user flow validation using ScoutQA CLI. IMPORTANT: Use this skill proactively after implementing web application features to verify they work correctly - don't wait for the user to ask for testing. |
There was a problem hiding this comment.
The description field should be wrapped in single quotes instead of using the pipe (|) syntax. This matches the convention used by the majority of skills in the repository (e.g., refactor, git-commit, github-issues). Change from:
description: |
This skill should be used...
to:
description: 'This skill should be used...'
| **IMPORTANT**: Use the Bash tool's timeout parameter (5000ms = 5 seconds) to capture execution details: | ||
|
|
||
| When calling the Bash tool, set `timeout: 5000` as a parameter: | ||
| - This is the Bash tool's built-in timeout parameter in Claude Code (NOT the Unix `timeout` command) | ||
| - After 5 seconds, the Bash tool returns control with a task ID and the process continues running in the background | ||
| - This is different from Unix `timeout` which kills the process - here the process keeps running | ||
| - The first 5 seconds capture the execution ID and browser URL from ScoutQA's output | ||
| - The test continues running remotely on ScoutQA's infrastructure with the background task |
There was a problem hiding this comment.
This skill contains specific references to "Claude Code" and its "Bash tool's timeout parameter", which are not applicable to GitHub Copilot. This repository is specifically for GitHub Copilot customizations (see README.md). The skill instructions should be rewritten to be compatible with GitHub Copilot's capabilities, or the references to Claude-specific features should be removed or made generic. If the ScoutQA CLI requires specific timeout handling, provide instructions that work with GitHub Copilot's tooling.
| Launch multiple tests in parallel by making multiple Bash tool calls in a single message, each with the Bash tool's `timeout` parameter set to `5000` (milliseconds): | ||
|
|
||
| ```bash | ||
| # Test 1: Authentication & security | ||
| scoutqa --url "https://app.example.com" --prompt " | ||
| Explore authentication: login/logout, session handling, password reset, | ||
| and security edge cases. | ||
| " | ||
|
|
||
| # Test 2: Core features (runs in parallel) | ||
| scoutqa --url "https://app.example.com" --prompt " | ||
| Test dashboard and main user workflows. Verify data loading, | ||
| CRUD operations, and search functionality. | ||
| " | ||
|
|
||
| # Test 3: Accessibility (runs in parallel) | ||
| scoutqa --url "https://app.example.com" --prompt " | ||
| Conduct accessibility audit: WCAG compliance, keyboard navigation, | ||
| screen reader support, color contrast. | ||
| " | ||
| ``` | ||
|
|
||
| **Implementation**: Send a single message with three Bash tool calls. For each Bash tool invocation, set the `timeout` parameter to `5000` milliseconds. After 5 seconds, each Bash call returns with a task ID while the processes continue running in the background. This captures the execution ID and browser URL from each test in the initial output, then all three continue running in parallel (both as background tasks locally and remotely on ScoutQA's infrastructure). | ||
|
|
||
| **Key guidelines:** | ||
|
|
||
| - Describe **what to test**, not **how to test** (ScoutQA figures out the steps) | ||
| - Focus on goals, edge cases, and concerns | ||
| - Run multiple parallel executions for different test areas | ||
| - Trust ScoutQA to autonomously explore and discover issues | ||
| - Always set the Bash tool's `timeout` parameter to `5000` milliseconds when calling scoutqa commands (this returns control after 5 seconds while the process continues in the background) | ||
| - For parallel tests, make multiple Bash tool calls in a single message | ||
| - Remember: Bash tool timeout ≠ Unix timeout command (Bash timeout continues the process in background, Unix timeout kills it) |
There was a problem hiding this comment.
Multiple references to "Bash tool" and its Claude-specific timeout parameter appear throughout the document. These need to be rewritten to be compatible with GitHub Copilot. Consider providing generic instructions for running the scoutqa command, or use standard shell approaches that work across different AI coding assistants.
| 3. To check results later, visit the browser URL provided when the test started | ||
| 4. Alternatively, use `scoutqa get-execution --execution-id <id>` to fetch results via CLI | ||
|
|
||
| **Best practice**: Start tests by setting the Bash tool's `timeout` parameter to `5000` milliseconds. After 5 seconds, the Bash tool returns control with a task ID and the execution details (execution ID and browser URL) while the test continues running in the background. You can then continue other work and check results on ScoutQA's website or via CLI when needed. |
There was a problem hiding this comment.
Reference to "Bash tool's timeout parameter" is specific to Claude Code and not applicable to GitHub Copilot. Remove or replace with GitHub Copilot-compatible instructions.
| **Best practice**: Start tests by setting the Bash tool's `timeout` parameter to `5000` milliseconds. After 5 seconds, the Bash tool returns control with a task ID and the execution details (execution ID and browser URL) while the test continues running in the background. You can then continue other work and check results on ScoutQA's website or via CLI when needed. | |
| **Best practice**: When starting tests from your editor, terminal, or CI workflow, use a short initial wait (for example, about 5 seconds) so the `scoutqa` CLI can return quickly with a task ID and the execution details (execution ID and browser URL) while the test continues running remotely in the background. You can then continue other work and check results on ScoutQA's website or via CLI when needed. |
| --- | ||
| name: scoutqa-test | ||
| description: | | ||
| This skill should be used when the user asks to "test this website", "run exploratory testing", "check for accessibility issues", "verify the login flow works", "find bugs on this page", or requests automated QA testing. Triggers on web application testing scenarios including smoke tests, accessibility audits, e-commerce flows, and user flow validation using ScoutQA CLI. IMPORTANT: Use this skill proactively after implementing web application features to verify they work correctly - don't wait for the user to ask for testing. | ||
| --- |
There was a problem hiding this comment.
The PR description claims "I have run npm start and verified that README.md is up to date", but the skill is not present in docs/README.skills.md. According to CONTRIBUTING.md, you must run npm start (which executes npm run build and updates the README files) before submitting. Please run this command and commit the updated README files.
1 participant
Pull Request Checklist
npm startand verified thatREADME.mdis up to date.Description
Summary
Adds scoutqa-test skill for AI-powered exploratory web testing via ScoutQA CLI
Enables automated smoke tests, accessibility audits, e-commerce flows, and user flow validation
Why This Skill?
Fills a gap: While webapp-testing focuses on Playwright-based local testing, ScoutQA provides autonomous AI-driven exploration that discovers issues humans might miss.
Proactive quality: Encourages testing immediately after implementing features - catching bugs while context is fresh.
Parallel execution: Run multiple test scenarios simultaneously (auth, accessibility, core features) for comprehensive coverage in less time.
Zero scripting: Describe what to test in natural language, not how - ScoutQA figures out the steps autonomously.
Type of Contribution
Additional Notes
By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.