Skip to content

add new doc for observer #263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vyokky merged 2 commits into from
Nov 13, 2025
Merged

add new doc for observer #263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5a06746
fix client doc
vyokky Nov 13, 2025
2980520
add observe doc
vyokky Nov 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 82 additions & 101 deletions documents/docs/galaxy/client/aip_integration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@

The Agent Interaction Protocol (AIP) is the communication protocol used throughout Galaxy Client for device coordination. This document explains how Galaxy Client integrates with AIP, the message flow patterns, and how different components use the protocol.

**Related Documentation:**
## Related Documentation

- [Overview](./overview.md) - Overall architecture
- [DeviceManager](./device_manager.md) - Connection management
- [Components](./components.md) - Component details
- [Overview](./overview.md) - Overall Galaxy Client architecture
- [DeviceManager](./device_manager.md) - Connection management using AIP
- [Components](./components.md) - Component-level AIP usage
- [AIP Protocol Specification](../../aip/overview.md) - Complete protocol reference
- [AIP Message Reference](../../aip/messages.md) - Detailed message structures

---

Expand Down Expand Up @@ -205,17 +206,15 @@ sequenceDiagram
**Step 3 - REGISTER_CONFIRMATION:**
```json
{
"type": "register_confirmation",
"type": "heartbeat",
"status": "ok",
"payload": {
"device_id": "windows_pc",
"session_id": "sess_abc123",
"message": "Registration successful"
},
"timestamp": "2025-11-06T10:30:01Z"
"timestamp": "2025-11-06T10:30:01Z",
"response_id": "reg_conf_abc123"
}
```

Note: The server confirms registration by sending a HEARTBEAT message with OK status, which serves as the registration confirmation in the AIP protocol.

**Step 4 - DEVICE_INFO_REQUEST:**
```json
{
Expand All @@ -234,8 +233,8 @@ sequenceDiagram
{
"type": "device_info_response",
"status": "ok",
"payload": {
"request_id": "req_xyz789",
"result": {
"device_id": "windows_pc",
"device_info": {
"os": "Windows 11",
"cpu_count": 8,
Expand All @@ -245,7 +244,8 @@ sequenceDiagram
"installed_apps": ["Microsoft Office", "Chrome", "VSCode"]
}
},
"timestamp": "2025-11-06T10:30:03Z"
"timestamp": "2025-11-06T10:30:03Z",
"response_id": "info_resp_xyz789"
}
```

Expand Down Expand Up @@ -280,11 +280,9 @@ sequenceDiagram
{
"type": "heartbeat",
"client_type": "constellation",
"payload": {
"device_id": "windows_pc",
"timestamp": "2025-11-06T10:35:00Z"
},
"status": "ok"
"client_id": "constellation_client_id",
"status": "ok",
"timestamp": "2025-11-06T10:35:00Z"
}
```

Expand All @@ -293,10 +291,8 @@ sequenceDiagram
{
"type": "heartbeat",
"status": "ok",
"payload": {
"device_id": "windows_pc",
"server_time": "2025-11-06T10:35:00Z"
}
"timestamp": "2025-11-06T10:35:00Z",
"response_id": "hb_resp_123"
}
```

Expand Down Expand Up @@ -342,47 +338,36 @@ sequenceDiagram
```json
{
"type": "task",
"status": "ok",
"payload": {
"task_id": "task_abc123",
"description": "Open Excel and create a chart",
"data": {
"file_path": "sales_report.xlsx",
"chart_type": "bar",
"data_range": "A1:C10"
},
"metadata": {
"priority": "high",
"timeout": 300
}
},
"timestamp": "2025-11-06T10:40:00Z"
"status": "continue",
"user_request": "Open Excel and create a chart",
"task_name": "galaxy/production/excel_task",
"session_id": "sess_task_abc123",
"timestamp": "2025-11-06T10:40:00Z",
"response_id": "task_req_001"
}
```

**COMMAND Message (server→client):**
```json
{
"type": "command",
"status": "ok",
"payload": {
"command_id": "cmd_001",
"task_id": "task_abc123",
"commands": [
{
"action": "launch_app",
"parameters": {
"app_name": "Excel"
}
},
{
"action": "open_file",
"parameters": {
"file_path": "sales_report.xlsx"
}
"status": "continue",
"actions": [
{
"action": "launch_app",
"parameters": {
"app_name": "Excel"
}
]
}
},
{
"action": "open_file",
"parameters": {
"file_path": "sales_report.xlsx"
}
}
],
"session_id": "sess_task_abc123",
"response_id": "cmd_001"
}
```

Expand All @@ -391,23 +376,22 @@ sequenceDiagram
{
"type": "command_results",
"client_type": "device",
"status": "ok",
"payload": {
"command_id": "cmd_001",
"task_id": "task_abc123",
"results": [
{
"action": "launch_app",
"status": "completed",
"output": "Excel launched successfully"
},
{
"action": "open_file",
"status": "completed",
"output": "File opened: sales_report.xlsx"
}
]
}
"client_id": "device_agent_id",
"status": "continue",
"action_results": [
{
"action": "launch_app",
"status": "completed",
"result": "Excel launched successfully"
},
{
"action": "open_file",
"status": "completed",
"result": "File opened: sales_report.xlsx"
}
],
"session_id": "sess_task_abc123",
"prev_response_id": "cmd_001"
}
```

Expand All @@ -416,22 +400,19 @@ sequenceDiagram
{
"type": "task_end",
"client_type": "device",
"client_id": "device_agent_id",
"status": "completed",
"payload": {
"task_id": "task_abc123",
"result": {
"success": true,
"output": "Created bar chart showing quarterly sales",
"artifacts": [
{
"type": "file",
"path": "sales_report_with_chart.xlsx"
}
]
},
"execution_time": 12.5,
"steps_completed": 3
"result": {
"success": true,
"output": "Created bar chart showing quarterly sales",
"artifacts": [
{
"type": "file",
"path": "sales_report_with_chart.xlsx"
}
]
},
"session_id": "sess_task_abc123",
"timestamp": "2025-11-06T10:40:15Z"
}
```
Expand Down Expand Up @@ -468,16 +449,14 @@ AIP uses ERROR messages for protocol-level errors:
{
"type": "error",
"status": "error",
"payload": {
"error": "Task execution exceeded 300 second timeout",
"session_id": "sess_task_abc123",
"timestamp": "2025-11-06T10:45:00Z",
"response_id": "err_001",
"metadata": {
"error_code": "TASK_TIMEOUT",
"error_message": "Task execution exceeded 300 second timeout",
"task_id": "task_abc123",
"device_id": "windows_pc",
"timestamp": "2025-11-06T10:45:00Z",
"context": {
"elapsed_time": 315.2,
"last_command": "create_chart"
}
"elapsed_time": 315.2,
"last_command": "create_chart"
}
}
```
Expand Down Expand Up @@ -1041,16 +1020,16 @@ This allows the same server to support both direct device connections and conste

AIP integration in Galaxy Client follows a layered architecture:

1. **Transport**: WebSocketConnectionManager handles raw WebSocket I/O
2. **Protocol**: AIPProtocol handles message serialization and middleware
1. **Transport**: WebSocketConnectionManager handles raw WebSocket I/O via AIP WebSocketTransport
2. **Protocol**: AIP protocol classes (RegistrationProtocol, TaskExecutionProtocol, HeartbeatProtocol, DeviceInfoProtocol) handle message serialization and protocol logic
3. **Message Processing**: MessageProcessor routes messages to handlers
4. **Application**: DeviceManager and ConstellationClient use messages for coordination
5. **Server Routing**: UFOWebSocketHandler routes messages between constellation clients and devices
6. **Device Execution**: Device agents execute tasks and return results

**Key Message Flows:**

- **Registration**: REGISTER → REGISTER_CONFIRMATION → DEVICE_INFO_REQUEST → DEVICE_INFO_RESPONSE
- **Registration**: REGISTER → HEARTBEAT (OK) → DEVICE_INFO_REQUEST → DEVICE_INFO_RESPONSE
- **Heartbeat**: HEARTBEAT (request) → HEARTBEAT (response), every 30 seconds
- **Task Execution (Constellation)**: ConstellationClient TASK → Server routes → Device executes → Server routes → ConstellationClient TASK_END
- **Task Execution (Direct)**: Device TASK → Server orchestrates → Device TASK_END
Expand Down Expand Up @@ -1083,9 +1062,11 @@ WebSocket → Device Agent (task execution)

AIP provides a robust, extensible protocol for agent communication with strong typing, clear message flows, comprehensive error handling, and intelligent routing between constellation clients and devices.

**Next Steps:**
## Next Steps

- See [DeviceManager](./device_manager.md) for connection management details
- See [Components](./components.md) for MessageProcessor implementation
- See [Components](./components.md) for MessageProcessor and WebSocketConnectionManager implementation
- See [ConstellationClient](./constellation_client.md) for device coordination API
- See [AIP Protocol Specification](../../aip/overview.md) for complete protocol reference
- See [AIP Message Reference](../../aip/messages.md) for detailed message structures and examples
- See [Server Documentation](../../server/websocket_handler.md) for server-side routing details
Loading