Enable gemini agent #69
Conversation
Scenario 1 can be run but Gemini CLI does not seem to call an MCP tool yet.
Updates eval configs to use correct agent spec format and cleans up gitignore
WalkthroughThe PR extends the NetEdge examples with support for the Gemini agent. It introduces a Gemini agent configuration with a runPrompt script and six evaluation YAML files for different test scenarios. Documentation is updated to describe Gemini usage alongside existing Codex instructions. A new gitignore pattern is added. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The Gemini CLI's mcp add command defaults to stdio transport. When gevals proxies an MCP server via HTTP, the agent.yaml script extracts the HTTP URL but doesn't explicitly specify the transport type. This causes the Gemini CLI to incorrectly interpret the URL as a stdio command, preventing proper tool registration. This commit adds the --transport http flag to the gemini mcp add command in examples/net-edge/gemini-agent/agent.yaml. This ensures the Gemini CLI correctly connects to the HTTP proxy for the Netedge MCP tools.
Signed-off-by: Brett Tofel <btofel@redhat.com>
bentito
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
examples/net-edge/README.md (1)
85-87: Update agent reference to be generic or mention both agents.Lines 86-87 still refer specifically to "The Codex agent" but this description now applies to both Codex and Gemini agents since the setup/verify scripts are shared.
Apply this diff:
`setup.sh` deploys the hello workload, then intentionally breaks the Service selector so the Route loses its -endpoints. The Codex agent must diagnose and repair the mismatch, after which `verify.sh` confirms the selector +endpoints. The agent must diagnose and repair the mismatch, after which `verify.sh` confirms the selector and endpoints are healthy. Results are written to `gevals-netedge-selector-mismatch-out.json` by default.
🧹 Nitpick comments (3)
examples/net-edge/gemini-agent/agent.yaml (3)
29-48: Consider adding error logging for kubeconfig copy failures.The kubeconfig preservation logic is sound. However, line 43 uses
|| trueto silence copy errors, which might hide legitimate issues. Consider logging failures when DEBUG_DIR is set.Apply this diff to add debug logging:
mkdir -p "${TMP_HOME}/.kube" - cp -R "${ORIGINAL_HOME}/.kube/." "${TMP_HOME}/.kube" 2>/dev/null || true + if ! cp -R "${ORIGINAL_HOME}/.kube/." "${TMP_HOME}/.kube" 2>/dev/null; then + if [[ -n "${DEBUG_DIR}" ]]; then + echo "Warning: Failed to copy .kube directory" >> "${DEBUG_DIR}/debug.log" + fi + fi
86-86: Document the "yolo" approval mode usage.The
--approval-mode yolosetting auto-approves all Gemini actions without confirmation. While this is likely intentional for automated testing scenarios, it should be clearly documented in the README or comments to ensure users understand the security implications.
81-81: Consider logging MCP configuration output when DEBUG_DIR is set.The
gemini mcp addcommand output is redirected to/dev/null, which may hide useful configuration messages or errors. Consider preserving this output when debug mode is enabled.Apply this diff:
# Configure MCP server for Gemini + if [[ -n "${DEBUG_DIR}" ]]; then + gemini mcp add netedge "${NETEDGE_URL}" --transport http 2>&1 | tee -a "${DEBUG_DIR}/debug.log" + else gemini mcp add netedge "${NETEDGE_URL}" --transport http >/dev/null + fi
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
.gitignore(1 hunks)examples/net-edge/README.md(2 hunks)examples/net-edge/gemini-agent/agent.yaml(1 hunks)examples/net-edge/gemini-agent/eval_1_selector-mismatch.yaml(1 hunks)examples/net-edge/gemini-agent/eval_2_nxdomain.yaml(1 hunks)examples/net-edge/gemini-agent/eval_3_networkpolicy.yaml(1 hunks)examples/net-edge/gemini-agent/eval_4_reencrypt-tls.yaml(1 hunks)examples/net-edge/gemini-agent/eval_5_loadbalancer.yaml(1 hunks)examples/net-edge/gemini-agent/eval_6_referencegrant.yaml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-11-18T20:44:43.077Z
Learnt from: Cali0707
Repo: genmcp/gevals PR: 39
File: .claude/skills/create-eval/SKILL.md:20-20
Timestamp: 2025-11-18T20:44:43.077Z
Learning: In the .claude/skills/create-eval/SKILL.md file, the eval creation instructions reference documentation files (.md) that explain each component (tasks.md, mcpConfig.md, agent.md, eval.md), not the actual YAML configuration files. The eval.md file contains documentation describing how to create eval.yaml files.
Applied to files:
examples/net-edge/README.mdexamples/net-edge/gemini-agent/eval_6_referencegrant.yamlexamples/net-edge/gemini-agent/eval_2_nxdomain.yamlexamples/net-edge/gemini-agent/eval_5_loadbalancer.yamlexamples/net-edge/gemini-agent/eval_1_selector-mismatch.yamlexamples/net-edge/gemini-agent/eval_3_networkpolicy.yamlexamples/net-edge/gemini-agent/eval_4_reencrypt-tls.yaml
🔇 Additional comments (13)
.gitignore (1)
16-16: LGTM!The addition of
*-out.jsonto ignore test artifacts is appropriate and well-placed in the test artifacts section.examples/net-edge/gemini-agent/agent.yaml (3)
1-8: LGTM!The Agent metadata and command configuration are correctly structured.
10-27: LGTM!The dependency and MCP server validation checks are thorough and include appropriate error handling.
50-75: LGTM!The cleanup function is well-implemented with proper error handling, debug artifact collection, and security-conscious removal of sensitive .kube data from debug output.
examples/net-edge/gemini-agent/eval_1_selector-mismatch.yaml (1)
1-15: LGTM!The Eval configuration is well-structured with appropriate assertions and correctly references the agent and task configurations.
examples/net-edge/gemini-agent/eval_3_networkpolicy.yaml (1)
1-15: LGTM!The configuration follows the established pattern and is correctly structured for the networkpolicy-block scenario.
examples/net-edge/gemini-agent/eval_6_referencegrant.yaml (1)
1-15: LGTM!The configuration is consistent with other eval files and correctly configured for the referencegrant-missing scenario.
examples/net-edge/README.md (3)
15-17: LGTM!The layout section correctly documents the new gemini-agent directory structure.
63-72: LGTM!The section is appropriately renamed to "Running with Codex" and the command is updated to use
gevals evalinstead ofgevals run.
74-83: LGTM!The new "Running with Gemini" section provides clear instructions for using the Gemini agent, including prerequisites and the evaluation command.
examples/net-edge/gemini-agent/eval_4_reencrypt-tls.yaml (1)
1-15: LGTM!The configuration follows the established pattern for the reencrypt-no-backend-tls scenario.
examples/net-edge/gemini-agent/eval_2_nxdomain.yaml (1)
1-15: LGTM!The configuration is consistent with other eval files and correctly structured for the nxdomain-host scenario.
examples/net-edge/gemini-agent/eval_5_loadbalancer.yaml (1)
1-15: LGTM! Configuration follows the established pattern and is correctly structured.All referenced files exist and the Eval YAML structure conforms to the documentation schema. The configuration is consistent with other eval files in the gemini-agent directory (eval_1, eval_2, eval_3, eval_4, eval_6).
Cali0707
left a comment
Cali0707
left a comment
There was a problem hiding this comment.
@bentito do you know if you could add the gemini agent as a builtin agent? (see the claude example here: https://github.com/genmcp/gevals/blob/main/pkg/agent/claude_code.go)
2 participants
Summary by CodeRabbit
Release Notes
New Features
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.