Podcast Generation
MicrosoftOfficialPublished by Microsoft
- Skill
- Workflow
- Automation and integrations
- MIT
- Low risk
- Has scripts
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a full-stack implementation for generating audio narratives from text content. It utilizes the Azure OpenAI GPT Realtime Mini model via WebSocket to stream audio and transcripts. The implementation includes a FastAPI backend service for orchestrating the generation process, handling PCM to WAV conversion, and managing database storage, alongside a React frontend component for audio playback.
What it does
- Connects to Azure OpenAI Realtime API via WebSockets
- Generates audio narratives from text inputs using various voice profiles
- Streams PCM audio chunks and transcripts from the model
- Converts raw PCM audio data into playable WAV format
- Provides a structured backend service for managing audio narrative requests
- Supports multiple narrative styles including podcast, summary, and lecture
When to use it
- When building text-to-speech features for web applications
- When creating automated audio summaries from existing text content
- When integrating Azure OpenAI Realtime API for audio output
- When implementing a full-stack workflow for generating and storing audio narratives
Advantages
- Enables real-time streaming of audio content
- Supports customizable narrative styles and voice options
- Includes logic for audio format conversion and duration calculation
- Provides a clear architectural pattern for full-stack integration
Usage examples
- “Generate a podcast-style audio narrative from my recent bookmarks.”
- The system fetches bookmark content, sends it to the Azure Realtime API with podcast-style instructions, and returns a playable WAV audio file.
- “Create an educational lecture audio for the selected tag.”
- The system compiles content associated with the tag, processes it through the Realtime API using the lecture style, and saves the resulting audio and transcript.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/microsoft/podcast-generation/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip podcast-generation.zip -d .agents/skills
Documentation by Microsoft
Podcast Generation with GPT Realtime Mini
Generate real audio narratives from text content using Azure OpenAI's Realtime API.
Quick Start
Configure environment variables for Realtime API
Connect via WebSocket to Azure OpenAI Realtime endpoint
Send text prompt, collect PCM audio chunks + transcript
Convert PCM to WAV format
Return base64-encoded audio to frontend for playback
Environment Configuration
AZURE_OPENAI_AUDIO_API_KEY=your_realtime_api_key
AZURE_OPENAI_AUDIO_ENDPOINT=https://your-resource.cognitiveservices.azure.com
AZURE_OPENAI_AUDIO_DEPLOYMENT=gpt-realtime-miniNote: Endpoint should NOT include /openai/v1/ - just the base URL.
Core Workflow
Backend Audio Generation
from openai import AsyncOpenAI
import base64
# Convert HTTPS endpoint to WebSocket URL
ws_url = endpoint.replace("https://", "wss://") + "/openai/v1"
client = AsyncOpenAI(
websocket_base_url=ws_url,
api_key=api_key
)
audio_chunks = []
transcript_parts = []
async with client.realtime.connect(model="gpt-realtime-mini") as conn:
# Configure for audio-only output
await conn.session.update(session={
"output_modalities": ["audio"],
"instructions": "You are a narrator. Speak naturally."
})
# Send text to narrate
await conn.conversation.item.create(item={
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": prompt}]
})
await conn.response.create()
# Collect streaming events
async for event in conn:
if event.type == "response.output_audio.delta":
audio_chunks.append(base64.b64decode(event.delta))
elif event.type == "response.output_audio_transcript.delta":
transcript_parts.append(event.delta)
elif event.type == "response.done":
break
# Convert PCM to WAV (see scripts/pcm_to_wav.py)
pcm_audio = b''.join(audio_chunks)
wav_audio = pcm_to_wav(pcm_audio, sample_rate=24000)Frontend Audio Playback
// Convert base64 WAV to playable blob
const base64ToBlob = (base64, mimeType) => {
const bytes = atob(base64);
const arr = new Uint8Array(bytes.length);
for (let i = 0; i < bytes.length; i++) arr[i] = bytes.charCodeAt(i);
return new Blob([arr], { type: mimeType });
};
const audioBlob = base64ToBlob(response.audio_data, 'audio/wav');
const audioUrl = URL.createObjectURL(audioBlob);
new Audio(audioUrl).play();Voice Options
| Voice | Character |
|---|---|
| alloy | Neutral |
| echo | Warm |
| fable | Expressive |
| onyx | Deep |
| nova | Friendly |
| shimmer | Clear |
Realtime API Events
response.output_audio.delta- Base64 audio chunkresponse.output_audio_transcript.delta- Transcript textresponse.done- Generation completeerror- Handle withevent.error.message
Audio Format
Input: Text prompt
Output: PCM audio (24kHz, 16-bit, mono)
Storage: Base64-encoded WAV
References
Full architecture: See references/architecture.md for complete stack design
Code examples: See references/code-examples.md for production patterns
PCM conversion: Use scripts/pcm_to_wav.py for audio format conversion
Files
- ATTRIBUTION.mdAdded by AutoGineer867 B
- LICENSELicence file added by AutoGineer1.1 KiB
- SKILL.md3.7 KiB
- references/architecture.md4.6 KiB
- references/code-examples.md6.0 KiB
- scripts/pcm_to_wav.pyScript2.4 KiB
references/code-examples.md
Code Examples
Complete Backend Service Method
async def generate_audio_narrative(
self,
source_type: str,
source_id: Optional[Union[int, str]] = None,
custom_query: Optional[str] = None,
voice_name: str = "alloy",
style: str = "podcast"
) -> Dict[str, Any]:
"""Generate podcast-style audio using gpt-realtime-mini"""
# 1. Validate configuration
if not settings.azure_openai_audio_api_key:
raise ValueError("AZURE_OPENAI_AUDIO_API_KEY not configured")
# 2. Gather content based on source
if source_type == "tag":
tag = await db.get_tag(source_id)
bookmarks = tag.bookmarks
title = f"Exploring {tag.name}"
elif source_type == "bookmark":
bookmark = await db.get_bookmark(source_id)
bookmarks = [bookmark]
title = f"Deep Dive: {bookmark.title}"
else: # custom
bookmarks = await db.get_recent_bookmarks(limit=10)
title = f"Research Summary: {custom_query[:50]}"
# 3. Build prompt with style
content = "\n".join([f"**{b.title}**\n{b.summary}" for b in bookmarks[:10]])
prompt = f"""Create a {style} narrative from these sources:
{content}
{STYLE_INSTRUCTIONS[style]}
Make it 1-2 minutes (150-250 words). Speak naturally."""
# 4. Connect to Realtime API
ws_url = settings.azure_openai_audio_endpoint.replace("https://", "wss://") + "/openai/v1"
client = AsyncOpenAI(websocket_base_url=ws_url, api_key=settings.azure_openai_audio_api_key)
audio_chunks, transcript_parts = [], []
async with client.realtime.connect(model=settings.azure_openai_audio_deployment) as conn:
await conn.session.update(session={
"output_modalities": ["audio"],
"instructions": f"Narrator creating {style}-style content. Speak naturally, don't ask questions."
})
await conn.conversation.item.create(item={
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": prompt}]
})
await conn.response.create()
async for event in conn:
if event.type == "response.output_audio.delta":
audio_chunks.append(base64.b64decode(event.delta))
elif event.type == "response.output_audio_transcript.delta":
transcript_parts.append(event.delta)
elif event.type == "response.done":
break
elif event.type == "error":
raise ValueError(f"Realtime API error: {event.error.message}")
# 5. Process audio
pcm_audio = b''.join(audio_chunks)
wav_audio = pcm_to_wav(pcm_audio, sample_rate=24000)
audio_base64 = base64.b64encode(wav_audio).decode('utf-8')
duration = len(pcm_audio) // (24000 * 2) # 24kHz, 16-bit
# 6. Save and return
narrative = AudioNarrative(
source_type=source_type,
title=title,
script=''.join(transcript_parts),
audio_data=audio_base64,
duration_seconds=duration,
voice_name=voice_name
)
db.add(narrative)
await db.commit()
return {
"id": narrative.id,
"title": title,
"script": narrative.script,
"audio_data": audio_base64,
"audio_url": f"/api/v1/ai/audio/{narrative.id}/stream",
"duration_seconds": duration,
"voice_name": voice_name
}FastAPI Endpoints
@router.post("/audio", response_model=AudioNarrativeResponse)
async def generate_audio(request: AudioNarrativeRequest, db: AsyncSession = Depends(get_db)):
ai_service = AIService(db)
result = await ai_service.generate_audio_narrative(
source_type=request.source_type,
source_id=request.source_id,
custom_query=request.custom_query,
voice_name=request.voice_name,
style=request.style
)
return AudioNarrativeResponse(**result)
@router.get("/audio/{narrative_id}/stream")
async def stream_audio(narrative_id: int, db: AsyncSession = Depends(get_db)):
ai_service = AIService(db)
narrative = await ai_service.get_audio_narrative_by_id(narrative_id)
if not narrative:
raise HTTPException(404, "Not found")
audio_bytes = base64.b64decode(narrative["audio_data"])
return Response(
content=audio_bytes,
media_type="audio/wav",
headers={"Content-Disposition": f'filename="narrative-{narrative_id}.wav"'}
)React Component Pattern
function AudioPlayer({ sourceType, sourceId }) {
const [narrative, setNarrative] = useState(null);
const [loading, setLoading] = useState(false);
const [isPlaying, setIsPlaying] = useState(false);
const audioRef = useRef(null);
const generate = async () => {
setLoading(true);
const response = await aiAPI.generateAudio(sourceType, sourceId, null, 'alloy', 'podcast');
setNarrative(response.data);
setLoading(false);
};
const play = () => {
if (!narrative?.audio_data) return;
const blob = base64ToBlob(narrative.audio_data, 'audio/wav');
const url = URL.createObjectURL(blob);
if (!audioRef.current) audioRef.current = new Audio();
audioRef.current.src = url;
audioRef.current.onended = () => setIsPlaying(false);
audioRef.current.play();
setIsPlaying(true);
};
return (
<div>
{!narrative && <button onClick={generate} disabled={loading}>Generate Podcast</button>}
{narrative && (
<>
<button onClick={play}>{isPlaying ? 'Pause' : 'Play'}</button>
<p>{narrative.script}</p>
</>
)}
</div>
);
}Frontend API Service
export const aiAPI = {
generateAudio: (sourceType, sourceId = null, customQuery = null,
voiceName = 'alloy', style = 'podcast') =>
api.post('/ai/audio', {
source_type: sourceType,
source_id: sourceId,
custom_query: customQuery,
voice_name: voiceName,
style
}),
listAudioNarratives: (limit = 10) => api.get(`/ai/audio?limit=${limit}`),
};Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Needs an API key, Needs Python
Security
What our scanner noticed· Low risk
- Scripts that your agent may run
This skill includes scripts that your agent may run. Read them before you use it.
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Published by
- Microsoft
- Publisher
- Microsoft (Official)
- Source
- microsoft/skills · .github/skills/podcast-generation · commit 0696600
- Licence
- MIT License (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by Microsoft.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download podcast-generation.zip