Mobile Agent #265
Mobile Agent #265
Conversation
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces Mobile Agent, a comprehensive Android device automation framework for UFO³. The implementation adds support for mobile devices as a third platform alongside Windows and Linux, enabling cross-platform automation workflows through ADB (Android Debug Bridge) integration.
Key Changes:
- Added MobileAgent with 3-state FSM and 4-phase processing pipeline for Android device automation
- Implemented dual MCP servers (data collection and action) for device interaction via ADB
- Extended platform support from 2 to 3 platforms (Windows, Linux, Mobile)
- Added comprehensive documentation including tutorials, strategies, and command references
Reviewed Changes
Copilot reviewed 52 out of 52 changed files in this pull request and generated 23 comments.
Show a summary per file
| File | Description |
|---|---|
ufo/server/ws/handler.py |
Fixed emoji display issues in WebSocket logs |
ufo/server/services/session_manager.py |
Extended platform support to include 'mobile' |
ufo/server/app.py |
Added 'mobile' to platform choices |
ufo/prompts/third_party/mobile_agent*.yaml |
Added prompt templates and examples for MobileAgent |
ufo/prompter/customized/mobile_agent_prompter.py |
Implemented prompter for mobile-specific prompts |
ufo/module/sessions/platform_session.py |
Added MobileBaseSession class |
ufo/module/sessions/mobile_session.py |
Implemented mobile session management |
ufo/module/session_pool.py |
Extended session pool for mobile sessions |
ufo/client/websocket.py |
Improved WebSocket error handling and logging |
ufo/client/ufo_client.py |
Updated platform documentation |
ufo/client/mcp/http_servers/mobile_mcp_server.py |
Implemented dual MCP servers for Android |
ufo/client/client.py |
Extended platform detection |
ufo/agents/states/mobile_agent_state.py |
Implemented 3-state FSM for MobileAgent |
ufo/agents/processors/strategies/mobile_agent_strategy.py |
Implemented 4-phase processing strategies |
ufo/agents/processors/schemas/target.py |
Fixed rect comment format documentation |
ufo/agents/processors/customized/*.py |
Added MobileAgentProcessor |
ufo/agents/agent/customized_agent.py |
Implemented MobileAgent class |
tests/integration/*.py |
Added integration tests and verification scripts |
tests/aip/test_binary_transfer.py |
Added unit tests for binary transfer |
documents/**/*.md |
Added comprehensive documentation |
galaxy/README*.md |
Updated feature lists for mobile support |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| stdout=asyncio.subprocess.PIPE, | ||
| stderr=asyncio.subprocess.PIPE, | ||
| ) | ||
| stdout, stderr = await proc.communicate() |
There was a problem hiding this comment.
Variable stdout is not used.
| stdout=asyncio.subprocess.PIPE, | ||
| stderr=asyncio.subprocess.PIPE, | ||
| ) | ||
| stdout, stderr = await proc.communicate() |
There was a problem hiding this comment.
Variable stderr is not used.
| """Integration tests for Mobile MCP Servers""" | ||
|
|
||
| @pytest.fixture(scope="class") | ||
| def check_adb_connection(self): |
There was a problem hiding this comment.
Mixing implicit and explicit returns may indicate an error, as implicit returns always return None.
| def get_prompter( | ||
| self, is_visual: bool, main_prompt: str, example_prompt: str | ||
| ) -> MobileAgentPrompter: |
There was a problem hiding this comment.
This method requires 4 positional arguments, whereas overridden BasicAgent.get_prompter requires 1.
| def message_constructor( | ||
| self, | ||
| dynamic_examples: List[str], | ||
| dynamic_knowledge: str, | ||
| plan: List[str], | ||
| request: str, | ||
| installed_apps: List[Dict[str, Any]], | ||
| current_controls: List[Dict[str, Any]], | ||
| screenshot_url: str = None, | ||
| annotated_screenshot_url: str = None, | ||
| blackboard_prompt: List[Dict[str, str]] = None, | ||
| last_success_actions: List[Dict[str, Any]] = None, | ||
| ) -> List[Dict[str, Union[str, List[Dict[str, str]]]]]: |
There was a problem hiding this comment.
This method requires at most 11 positional arguments, whereas overridden AppAgent.message_constructor requires 14.
This method requires at least 7 positional arguments, whereas overridden BasicAgent.message_constructor requires 1.
| ) | ||
| if result.returncode == 0: | ||
| return result.stdout.strip().split("\n")[0] | ||
| except: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| ) | ||
| if result.returncode == 0: | ||
| return result.stdout.strip().split("\n")[0] | ||
| except: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| ) | ||
| if result.returncode == 0: | ||
| return result.stdout.strip().split("\n")[0] | ||
| except: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| """ | ||
| # Implement evaluation logic specific to Mobile sessions | ||
| self.logger.warning("Evaluation not yet implemented for Mobile sessions.") | ||
| pass |
There was a problem hiding this comment.
Unnecessary 'pass' statement.
| """ | ||
| # Implement markdown logging specific to Mobile sessions | ||
| self.logger.warning("Markdown logging not yet implemented for Mobile sessions.") | ||
| pass |
There was a problem hiding this comment.
Unnecessary 'pass' statement.
No description provided.