Skip to content

Optimize httpx: Replace per-request AsyncClient with shared client in translate.py #1930

@crivetimihai

Description

@crivetimihai

Summary

The translate.py module creates new httpx.AsyncClient() instances for each SSE/Streamable HTTP bridge operation, causing unnecessary connection overhead (15-20ms per request).

Problem

Each translate operation creates a new client instead of reusing the shared HTTP client:

# Current anti-pattern (4 locations)
async with httpx.AsyncClient(verify=ssl_context, timeout=timeout) as client:
    response = await client.post(...)

Affected File

File Lines Context
mcpgateway/translate.py 1179 SSE to HTTP translation
mcpgateway/translate.py 1336 HTTP request handling
mcpgateway/translate.py 1587 Streamable HTTP client
mcpgateway/translate.py 1703 Additional HTTP bridge

Proposed Solution

Use the existing isolated client factory from http_client_service.py:

from mcpgateway.services.http_client_service import get_isolated_http_client

# For custom SSL context needs
async with get_isolated_http_client(verify=ssl_context, timeout=timeout) as client:
    response = await client.post(...)

Or for standard cases, use the shared singleton:

from mcpgateway.services.http_client_service import get_http_client

client = await get_http_client()
response = await client.post(...)

Impact

  • Performance: 15-20ms saved per translate operation
  • Resource usage: Connection pooling reuse across operations
  • Consistency: Aligns with existing HTTP client patterns in other services

Acceptance Criteria

  • All 4 per-request httpx.AsyncClient() instances replaced
  • Custom SSL context support preserved via get_isolated_http_client()
  • Connection limits applied from global configuration
  • Existing tests pass

References

  • mcpgateway/services/http_client_service.py - Shared client implementation
  • mcpgateway/services/tool_service.py - Reference usage pattern
  • todo/pooling-optimizations.md - Full optimization analysis

Metadata

Metadata

Assignees

Labels

performancePerformance related items

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions