Skip to content

Conversation

@ochafik
Copy link
Collaborator

@ochafik ochafik commented Jan 12, 2026

Summary

Adds transcript-server example demonstrating live speech-to-text transcription using Web Speech API.

Features

  • Live transcription: Real-time speech-to-text using browser's Web Speech API
  • Transitional context via ui/update-model-context: Streams interim transcriptions to the model context in real-time, allowing the model to see what the user is saying as they speak
  • Final messages via ui/message: Sends completed transcriptions when user clicks Send
  • Permissions: Uses microphone and clipboard-write permissions (passed through loadSandboxProxy)
  • Host theme integration: Supports light/dark mode from host context

Dependencies

CleanShot 2026-01-12 at 18 52 29

🤖 Generated with Claude Code

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jan 12, 2026

Open in StackBlitz

@modelcontextprotocol/ext-apps

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/ext-apps@240

@modelcontextprotocol/server-basic-react

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-react@240

@modelcontextprotocol/server-basic-vanillajs

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-vanillajs@240

@modelcontextprotocol/server-budget-allocator

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-budget-allocator@240

@modelcontextprotocol/server-cohort-heatmap

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-cohort-heatmap@240

@modelcontextprotocol/server-customer-segmentation

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-customer-segmentation@240

@modelcontextprotocol/server-scenario-modeler

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-scenario-modeler@240

@modelcontextprotocol/server-system-monitor

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-system-monitor@240

@modelcontextprotocol/server-threejs

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-threejs@240

@modelcontextprotocol/server-wiki-explorer

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-wiki-explorer@240

commit: 8de3789

@ochafik ochafik requested review from antonpk1 and idosal January 12, 2026 11:10
@ochafik ochafik changed the title feat: add transcript-server example with live speech transcription examples: add transcript-server example with live speech transcription Jan 12, 2026
@ochafik ochafik force-pushed the feat/transcript-server branch from 4eacc87 to 51c7bac Compare January 12, 2026 18:51
@ochafik ochafik force-pushed the feat/transcript-server branch from 51c7bac to 6f15b7b Compare January 12, 2026 19:19
@ochafik ochafik changed the base branch from main to ochafik/improve-host January 12, 2026 19:20
@ochafik ochafik force-pushed the feat/transcript-server branch from 6f15b7b to 6e49937 Compare January 12, 2026 19:27
@ochafik ochafik marked this pull request as ready for review January 12, 2026 19:31
ochafik and others added 13 commits January 12, 2026 19:43
- New example: transcript-server with Web Speech API transcription
  - Start/stop recording with timer display
  - Live transcript with interim results
  - Send button to send transcript as ui/message to host
  - Copy button to copy full transcript
  - Sent messages marked with divider, greyed out
  - Experimental ui/update-model-context for live context updates
  - Transparent background, respects host theme

- Enable microphone and clipboard in basic-host sandbox iframes
  - Add allow="microphone; clipboard-write" to both outer and inner iframes
  - Required for Web Speech API, audio capture, and clipboard access

- Add alert popup in basic-host for ui/message from apps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Removes ~50 lines of manually defined Web Speech API types in favor of
the community-maintained type package.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add formatEntry() helper that includes [timestamp] prefix
- Add formatEntries() helper that joins entries with newlines
- Use same format for clipboard copy, send message, and model context
- DRY up duplicate code in send button handler
- Add permissions parameter to loadSandboxProxy in implementation.ts
- Build iframe allow attribute from permissions instead of hardcoding
- Update sandbox.ts to not hardcode allow attribute (set via notification)
- Add microphone + clipboardWrite permissions to transcript-server resource
- Update README to reflect permission configuration via resource _meta.ui
- Remove min-height: 100vh and overflow-y: auto
- Let content grow naturally with autoResize (default)
- Remove unnecessary scrollTop calls
@ochafik ochafik force-pushed the feat/transcript-server branch from e27d06e to 88f4260 Compare January 12, 2026 19:44
@ochafik ochafik changed the base branch from ochafik/improve-host to main January 12, 2026 19:44
@ochafik ochafik merged commit 6bb63c3 into main Jan 12, 2026
19 checks passed
@ochafik ochafik deleted the feat/transcript-server branch January 12, 2026 21:28
@ochafik ochafik mentioned this pull request Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants