Introduction​

This article documents the implementation of an AI-powered job search agent using Google’s Agent Development Kit (ADK) with Google Search grounding. The agent demonstrates how to build intelligent search applications that combine the reasoning capabilities of large language models with real-time web search data.

What is Google ADK?​

Google’s Agent Development Kit (ADK) is a TypeScript framework for building AI agents powered by Gemini models. Released in late 2025, ADK provides:

• Agent orchestration — Define agents with specific tools and instructions
• Tool integration — Built-in tools like GOOGLE_SEARCH for grounding
• Session management — Persistent conversation state across interactions
• Event streaming — Real-time access to agent execution events
• Grounding metadata — Access to search sources, citations, and web queries

Key features​

import { LlmAgent, GOOGLE_SEARCH } from '@google/adk';

const agent = new LlmAgent({
  name: 'searchAgent',
  model: 'gemini-2.5-flash',
  tools: [GOOGLE_SEARCH],
  instruction: 'Your detailed instructions here...'
});

The ADK’s GOOGLE_SEARCH tool enables grounding—anchoring LLM responses in real web search results rather than relying solely on training data.

The search agent architecture​

High-level flow (event stream + grounding)​

Top-down diagrams​

Top-down architecture (layered)​

Top-down data flow (what moves where)​

Agent design​

Our job search agent is built with a clear purpose: find remote AI consulting opportunities using structured search strategies.

File: /src/google/search-agent.ts

export const searchAgent = new LlmAgent({
  name: 'jobSearchAgent',
  model: 'gemini-2.5-flash',
  tools: [GOOGLE_SEARCH],
  instruction: `You are an expert job search assistant specializing in AI/GenAI roles...

ALWAYS perform Google Search first to get real-time job listings.
Extract actual job data from search results...

Return structured JSON with job listings...`
});

Agent instructions​

The agent is instructed to:

1. Execute targeted searches — Use specific boolean search operators
2. Extract structured data — Parse job listings into JSON format
3. Apply filters — Focus on last 7 days, remote-only, consultancy roles
4. Include metadata — Company details, remote scope, EU eligibility

Search strategy​

The agent generates six complementary search queries:

const queries = [
  'fully remote "AI consultant" agency OR consultancy "client-facing" RAG OR LLM (last 7 days)',
  'fully remote "generative AI engineer" agency "client delivery" RAG OR agents (last 7 days)',
  'fully remote "LLM architect" consultancy "client engagement" RAG OR agents (last 7 days)',
  'fully remote "AI solutions architect" consultancy "client facing" (last 7 days)',
  'remote "AI delivery manager" consultancy LLM OR GenAI (last 7 days)',
  'fully remote "AI specialist" consulting "client projects" EU (last 7 days)'
];

This multi-query approach captures:

• Different role titles (consultant, engineer, architect, specialist)
• Various technical focuses (RAG, agents, LLM)
• Client-facing delivery emphasis
• Recent postings only (7-day window)

Implementation details​

Runner architecture​

File: /src/google/runner.ts

The runner manages agent execution and extracts grounding metadata:

export async function runSearchAgent(
  prompt: string,
  options: { saveToFile?: boolean; outputDir?: string } = {}
): Promise<void> {
  const sessionService = new InMemorySessionService();
  const sessionId = `session-${Date.now()}`;

  const runner = new Runner(rootAgent, sessionService);

  for await (const event of runner.runGenerateContent(sessionId, { text: prompt })) {
    if (isFinalResponse(event)) {
      const content = stringifyContent(event.content);
      const grounding = getGroundingArrays(event);

      // Process and save results
    }
  }
}

Grounding metadata extraction​

The critical feature is extracting grounding metadata—the search sources and citation mappings:

function getGroundingArrays(event: any): GroundingData {
  const metadata = event.groundingMetadata;

  return {
    groundingChunks: metadata?.groundingChunks || [],
    groundingSupports: metadata?.groundingSupports || [],
    webSearchQueries: metadata?.webSearchQueries || [],
    searchEntryPoint: metadata?.searchEntryPoint
  };
}

This provides:

• groundingChunks — Source URLs and titles
• groundingSupports — Citation mappings linking response segments to sources
• webSearchQueries — The actual search queries executed by Google
• searchEntryPoint — HTML content with Google Search suggestions

Citation integration​

The runner adds citation markers [1,2,3] to the agent’s response:

function addCitationMarkersToSegment(
  text: string,
  supports: any[]
): string {
  const indices = supports
    .flatMap(s => s.groundingChunkIndices || [])
    .filter((v, i, a) => a.indexOf(v) === i)
    .map(i => i + 1);

  return indices.length > 0
    ? `${text}[${indices.join(',')}]`
    : text;
}

JSON output​

Results are saved to timestamped JSON files:

const output = {
  timestamp: new Date().toISOString(),
  query: prompt,
  rawResponse: content,
  data: parsedData,
  summary: summary,
  webSearchQueries: grounding.webSearchQueries,
  sources: sources,
  renderedContent: grounding.searchEntryPoint?.renderedContent
};

fs.writeFileSync(
  path.join(outputDir, `job-search-results-${timestamp}.json`),
  JSON.stringify(output, null, 2)
);

Running the agent​

Execution script​

File: /scripts/run-google-search-agent.ts

import dotenv from 'dotenv';
import { runSearchAgent } from '../src/google/index';

// Load environment variables
dotenv.config({ path: '.env' });
dotenv.config({ path: '.env.local', override: true });

// Validate API key
if (!process.env.GEMINI_API_KEY) {
  console.error('❌ Error: GEMINI_API_KEY not found');
  process.exit(1);
}

// Execute search
const prompt = `Find 10 fully-remote AI / GenAI roles at agencies or consultancies.
Prefer client-facing delivery roles, especially those involving RAG, agents, or LLM implementation.
Include EU eligibility information.`;

runSearchAgent(prompt, { saveToFile: true, outputDir: './results' });

Running the search​

npx tsx scripts/run-google-search-agent.ts

Results and grounding analysis​

Search execution​

The agent successfully executed 6 web search queries and identified 10 distinct job sources:

1. remoterocketship.com — Remote-first job platform
2. pinpointhq.com — Tech recruitment
3. yutori.com — Remote opportunities
4. djinni.co — Tech talent marketplace
5. talent.com — Global job aggregator
6. shine.com — Job search platform
7. shiza.ai — AI-specialized job board
8. studysmarter.co.uk — Career resources (3 matches)

Grounding metadata​

The response included rich grounding data:

Web Search Queries Executed:

[
  "fully remote \"AI consultant\" agency OR consultancy OR \"professional services\" \"client-facing\" RAG OR LLM OR \"generative AI\" (last 7 days)",
  "fully remote \"generative AI engineer\" agency OR consultancy OR \"professional services\" \"client delivery\" RAG OR LLM OR agents (last 7 days)",
  "fully remote \"LLM architect\" consultancy \"client engagement\" RAG OR agents (last 7 days)",
  "fully remote \"AI solutions architect\" consultancy \"client facing\" (last 7 days)",
  "remote \"AI delivery manager\" consultancy LLM OR GenAI (last 7 days)",
  "fully remote \"AI specialist\" consulting \"client projects\" EU (last 7 days)"
]

Source Citations:

[
  {
    "index": 1,
    "title": "remoterocketship.com",
    "url": "https://vertexaisearch.cloud.google.com/grounding-api-redirect/..."
  }
]

Search Entry Point: The renderedContent field contains Google Search’s HTML with interactive search chips, providing users with clickable query refinements.

Technical challenges and lessons​

Challenge 1: Empty responses​

Problem: Initial implementation returned content: null despite successful execution.

Root cause: Using AgentTool to wrap another agent with GOOGLE_SEARCH created conflicts.

Solution: Use GOOGLE_SEARCH directly as the sole tool in the agent—ADK requires grounding tools to be the only tool configured.

Challenge 2: Structured data extraction​

Problem: Agent returned Google Search HTML suggestions instead of extracted job data.

Solution: Enhanced instructions with explicit output format requirements and processing rules:

instruction: `
Rules:
1. ALWAYS perform Google Search first
2. Extract actual job data from search results
3. Format as JSON with fields: title, company, remote_scope, url, posted_date, etc.
4. Search for jobs posted within the LAST 7 DAYS only
`

Challenge 3: API key configuration​

Problem: GEMINI_API_KEY not found in environment.

Solution: Support multiple .env files with precedence:

dotenv.config({ path: '.env' });
dotenv.config({ path: '.env.local', override: true });

Best practices​

1. Single tool limitation​

When using GOOGLE_SEARCH, it must be the only tool in the agent:

// ✅ Correct
const agent = new LlmAgent({
  tools: [GOOGLE_SEARCH]
});

// ❌ Incorrect - will cause issues
const agent = new LlmAgent({
  tools: [GOOGLE_SEARCH, CODE_EXECUTION, someCustomTool]
});

2. Explicit instructions​

Provide clear, structured instructions for the LLM:

instruction: `
Purpose: [What the agent does]

Process:
1. [Step 1]
2. [Step 2]

Output Format:
{
  "field": "description"
}

Rules:
- [Rule 1]
- [Rule 2]
`

3. Grounding metadata access​

Access grounding data from the final response event:

for await (const event of runner.runGenerateContent(sessionId, prompt)) {
  if (isFinalResponse(event)) {
    const grounding = event.groundingMetadata;
    const chunks = grounding?.groundingChunks || [];
  }
}

4. Event stream debugging​

Log event types to understand agent execution flow:

for await (const event of runner.runGenerateContent(...)) {
  console.log('Event:', {
    finishReason: event.finishReason,
    hasContent: !!event.content,
    hasGrounding: !!event.groundingMetadata
  });
}

Conclusion​

Google’s ADK provides a powerful framework for building grounded AI agents. Key takeaways:

• Grounding is essential — Real-time search data makes agents more accurate and current
• Citations add credibility — Extracted source URLs enable verification
• Event streams enable debugging — Real-time event monitoring aids development
• Instructions matter — Clear, structured prompts yield better results
• Multi-query strategies work — Complementary searches capture diverse results

The job search agent demonstrates how ADK can transform a simple search task into an intelligent, automated workflow with structured outputs and verifiable sources.

Technical stack​

• @google/adk v0.3.0 — Agent Development Kit
• gemini-2.5-flash — LLM model with grounding support
• TypeScript — Type-safe agent development
• Node.js 24+ — Runtime environment
• tsx — TypeScript execution
• dotenv — Environment configuration

Repository structure​

/src/google/
  search-agent.ts    # Agent definition with GOOGLE_SEARCH
  runner.ts          # Execution logic and grounding extraction
  index.ts           # Public exports
  README.md          # Setup and usage documentation

/scripts/
  run-google-search-agent.ts  # Executable runner script

/results/
  job-search-results-{timestamp}.json  # Output files

This article documents a production implementation of OpenAI's Text-to-Speech (TTS) API with automatic chunking for long-form content and seamless upload to Cloudflare R2 storage.

Architecture Overview​

The system provides two API entrypoints for audio generation:

1. GraphQL Mutation (generateOpenAIAudio) — used by the main app for story audio
2. REST API (/api/tts) — provides flexible streaming options and direct upload

Both endpoints support:

• Automatic text chunking for content exceeding 4000 characters
• Audio merging for seamless playback of long content
• Cloudflare R2 upload with public CDN URLs
• Base64 fallback for immediate playback while uploading
• Metadata tracking (duration, voice, model, etc.)

OpenAI TTS Integration​

Voice Selection​

Defaults to onyx but supports all OpenAI TTS voices:

• alloy, ash, ballad, coral, echo, fable
• onyx (default), nova, sage, shimmer
• verse, marin, cedar

Model Selection​

Supports three models:

• gpt-4o-mini-tts (default) — fast, efficient, high quality
• tts-1 — standard quality
• tts-1-hd — high definition audio

Audio Formats​

Supports multiple output formats:

• mp3 (default) — best compatibility
• opus, aac, flac, wav, pcm

Text Chunking for Long Content​

OpenAI TTS has a 4096 character limit. This implementation uses Mastra RAG’s recursive chunking strategy to intelligently split long text.

import { MDocument } from "@mastra/rag";

const MAX_CHARS = 4000; // Buffer below OpenAI's 4096 limit

async function chunkTextForSpeech(text: string): Promise<string[]> {
  const doc = MDocument.fromText(text);

  const chunks = await doc.chunk({
    strategy: "recursive",
    maxSize: MAX_CHARS,
    overlap: 50,
    separators: ["\n\n", "\n", ". ", "! ", "? "],
  });

  return chunks.map((chunk) => chunk.text);
}

Key features:

• Respects paragraph breaks (\n\n)
• Falls back to sentence boundaries (., !, ?)
• 50-character overlap prevents awkward breaks
• Maintains narrative flow across chunks

Audio Merging​

When text is chunked, each piece is converted to audio separately, then merged into a single file:

// Generate audio for each chunk
const audioChunks: Buffer[] = [];

for (const chunk of chunks) {
  const response = await openai.audio.speech.create({
    model: "gpt-4o-mini-tts",
    voice: "onyx",
    input: chunk,
    response_format: "mp3",
    speed: 0.9,
  });

  const buffer = Buffer.from(await response.arrayBuffer());
  audioChunks.push(buffer);
}

// Combine into single audio file
const combined = Buffer.concat(audioChunks);

Why merge?

• Single file = simpler playback
• No gaps between chunks
• Easier to upload and share
• Better browser compatibility

Cloudflare R2 Upload​

R2 Client Setup​

Uses AWS SDK v3 with Cloudflare R2 endpoints:

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const r2Client = new S3Client({
  region: "auto",
  endpoint: `https://${R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
  credentials: {
    accessKeyId: R2_ACCESS_KEY_ID,
    secretAccessKey: R2_SECRET_ACCESS_KEY,
  },
});

Upload Function​

export async function uploadToR2(options: {
  key: string;
  body: Buffer;
  contentType?: string;
  metadata?: Record<string, string>;
}): Promise<{
  key: string;
  publicUrl: string | null;
  bucket: string;
  sizeBytes: number;
}> {
  const { key, body, contentType = "audio/mpeg", metadata = {} } = options;

  await r2Client.send(
    new PutObjectCommand({
      Bucket: R2_BUCKET_NAME,
      Key: key,
      Body: body,
      ContentType: contentType,
      Metadata: metadata,
    }),
  );

  const publicUrl = R2_PUBLIC_DOMAIN ? `${R2_PUBLIC_DOMAIN}/${key}` : null;

  return {
    key,
    publicUrl,
    bucket: R2_BUCKET_NAME,
    sizeBytes: body.length,
  };
}

Key Generation​

Unique keys with timestamps prevent collisions:

export function generateAudioKey(prefix?: string): string {
  const timestamp = Date.now();
  const random = Math.random().toString(36).substring(2, 15);
  return `${prefix ? `${prefix}/` : ""}audio-${timestamp}-${random}.mp3`;
}

Example output: graphql-tts/audio-1707839472156-9k2jd8x4.mp3

Metadata Extraction​

The system calculates audio duration using music-metadata:

import { parseBuffer } from "music-metadata";

let duration: number | null = null;
try {
  const metadata = await parseBuffer(audioBuffer, {
    mimeType: "audio/mp3",
  });
  duration = metadata.format.duration || null;
} catch (error) {
  console.warn("Failed to parse audio duration:", error);
}

Metadata stored with upload:

• voice — TTS voice used
• model — OpenAI model
• textLength — original text length
• chunks — number of chunks (if split)
• generatedBy — user email
• instructions — custom TTS instructions (optional)

GraphQL Implementation​

Schema Definition​

input GenerateOpenAIAudioInput {
  text: String!
  storyId: Int
  voice: OpenAITTSVoice
  model: OpenAITTSModel
  speed: Float
  responseFormat: OpenAIAudioFormat
  uploadToCloud: Boolean
  instructions: String
}

type GenerateOpenAIAudioResult {
  success: Boolean!
  message: String
  audioBuffer: String
  audioUrl: String
  key: String
  sizeBytes: Int
  duration: Float
}

type Mutation {
  generateOpenAIAudio(input: GenerateOpenAIAudioInput!): GenerateOpenAIAudioResult!
}

Resolver Implementation​

export const generateOpenAIAudio: MutationResolvers["generateOpenAIAudio"] = async (
  _parent,
  args,
  ctx,
) => {
  const userEmail = ctx.userEmail;
  if (!userEmail) {
    throw new Error("Authentication required");
  }

  const {
    text,
    storyId,
    voice = "ONYX",
    model = "GPT_4O_MINI_TTS",
    speed = 0.9,
    responseFormat = "MP3",
    uploadToCloud,
  } = args.input;

  // Map GraphQL enums to OpenAI API values
  const openAIVoice = voice.toLowerCase();
  const openAIModel = model === "GPT_4O_MINI_TTS" ? "gpt-4o-mini-tts" : "tts-1";
  const format = responseFormat.toLowerCase();

  // Handle chunking if needed
  if (text.length > MAX_CHARS) {
    const chunks = await chunkTextForSpeech(text);
    const audioChunks: Buffer[] = [];

    for (const chunk of chunks) {
      const response = await openai.audio.speech.create({
        model: openAIModel,
        voice: openAIVoice,
        input: chunk,
        response_format: format,
        speed,
      });

      audioChunks.push(Buffer.from(await response.arrayBuffer()));
    }

    const combined = Buffer.concat(audioChunks);

    // Upload to R2
    if (uploadToCloud) {
      const key = generateAudioKey("graphql-tts");
      const result = await uploadToR2({
        key,
        body: combined,
        contentType: `audio/${format}`,
        metadata: {
          voice: openAIVoice,
          model: openAIModel,
          textLength: text.length.toString(),
          chunks: chunks.length.toString(),
          generatedBy: userEmail,
        },
      });

      // Save to story if provided
      if (storyId) {
        await saveAudioToStory(storyId, result.key, result.publicUrl, userEmail);
      }

      return {
        success: true,
        message: `Audio generated from ${chunks.length} chunks and uploaded to R2`,
        audioBuffer: combined.toString("base64"),
        audioUrl: result.publicUrl,
        key: result.key,
        sizeBytes: result.sizeBytes,
        duration: audioDuration,
      };
    }

    return {
      success: true,
      message: `Audio generated from ${chunks.length} chunks`,
      audioBuffer: combined.toString("base64"),
      audioUrl: null,
      key: null,
      sizeBytes: combined.length,
      duration: audioDuration,
    };
  }

  // ... handle short text similarly
};

Client Usage​

import { useGenerateOpenAiAudioMutation } from "@/app/__generated__/hooks";

const [generateAudio, { loading }] = useGenerateOpenAiAudioMutation();

async function handleTextToSpeech() {
  const result = await generateAudio({
    variables: {
      input: {
        text: storyContent,
        storyId,
        voice: OpenAittsVoice.Onyx,
        model: OpenAittsModel.Gpt_4OMiniTts,
        speed: 0.9,
        responseFormat: OpenAiAudioFormat.Mp3,
        uploadToCloud: true,
      },
    },
  });

  const payload = result.data?.generateOpenAIAudio;
  if (payload?.success && payload.audioUrl) {
    // Use public R2 URL
    setAudioSrc(payload.audioUrl);
  } else if (payload?.audioBuffer) {
    // Fallback to base64
    const blob = base64ToBlob(payload.audioBuffer);
    setAudioSrc(blob);
  }
}

REST API Implementation​

Streaming Options​

The /api/tts endpoint supports multiple streaming modes:

export async function POST(request: NextRequest) {
  const {
    text,
    voice = "alloy",
    uploadToCloud,
    streamFormat, // "sse" for Server-Sent Events
    instructions,
  } = await request.json();

  // For short text with SSE streaming
  if (streamFormat === "sse" && !uploadToCloud) {
    const response = await openai.audio.speech.create({
      model: "gpt-4o-mini-tts",
      voice,
      input: text,
      response_format: "mp3",
      speed: 0.9,
      stream_format: "sse",
    });

    // Create SSE stream
    const stream = new ReadableStream({
      async start(controller) {
        const reader = response.body?.getReader();
        if (!reader) {
          controller.close();
          return;
        }

        while (true) {
          const { done, value } = await reader.read();
          if (done) break;
          controller.enqueue(value);
        }
        controller.close();
      },
    });

    return new NextResponse(stream, {
      headers: {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache",
        Connection: "keep-alive",
      },
    });
  }

  // Standard audio stream
  const response = await openai.audio.speech.create({
    model: "gpt-4o-mini-tts",
    voice,
    input: text,
    response_format: "mp3",
    speed: 0.9,
  });

  return new NextResponse(response.body, {
    headers: {
      "Content-Type": "audio/mp3",
      "Cache-Control": "no-cache",
      "Transfer-Encoding": "chunked",
    },
  });
}

Client Usage (Fetch API)​

// Upload to R2
const response = await fetch("/api/tts", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    text: storyContent,
    voice: "onyx",
    uploadToCloud: true,
    storyId: 123,
    userEmail: "user@example.com",
  }),
});

const data = await response.json();
// { success: true, audioUrl: "https://tts.yourdomain.com/audio-123456.mp3", key: "...", sizeBytes: 234567 }

// Direct streaming
const response2 = await fetch("/api/tts", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    text: "Hello world",
    voice: "onyx",
  }),
});

const blob = await response2.blob();
const audioUrl = URL.createObjectURL(blob);
audioElement.src = audioUrl;

Database Integration​

Audio metadata is saved to the database when a storyId is provided:

async function saveAudioToStory(
  storyId: number,
  audioKey: string,
  audioUrl: string | null,
  userEmail: string,
): Promise<void> {
  const now = new Date().toISOString();
  await turso.execute({
    sql: `UPDATE stories
          SET audio_key = ?, audio_url = ?, audio_generated_at = ?, updated_at = ?
          WHERE id = ? AND user_id = ?`,
    args: [audioKey, audioUrl || "", now, now, storyId, userEmail],
  });
}

Database schema (SQLite):

CREATE TABLE stories (
  id INTEGER PRIMARY KEY,
  goal_id INTEGER NOT NULL,
  user_id TEXT NOT NULL,
  content TEXT NOT NULL,
  audio_key TEXT,
  audio_url TEXT,
  audio_generated_at TEXT,
  created_at TEXT NOT NULL,
  updated_at TEXT NOT NULL
);

Configuration​

Environment Variables​

# OpenAI
OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Cloudflare R2
R2_ACCOUNT_ID=your-account-id-here
R2_ACCESS_KEY_ID=your-access-key-id-here
R2_SECRET_ACCESS_KEY=your-secret-access-key-here
R2_BUCKET_NAME=longform-tts
R2_PUBLIC_DOMAIN=https://tts.yourdomain.com

R2 Bucket Setup​

1. Create R2 Bucket: In Cloudflare dashboard, create bucket named longform-tts

2. Generate API Token: Create an API token with R2 read/write permissions

3. Configure Custom Domain (optional):

   • Add custom domain in R2 bucket settings
   • Point DNS to R2 bucket URL
   • Enable public access for the bucket

4. CORS Configuration (if accessing from browser):

{
  "AllowedOrigins": ["https://your-app.com"],
  "AllowedMethods": ["GET", "HEAD"],
  "AllowedHeaders": ["*"],
  "ExposeHeaders": ["Content-Length", "Content-Type"],
  "MaxAgeSeconds": 3600
}

Performance Considerations​

Text Chunking​

• 4000 char limit: safe buffer below OpenAI’s 4096
• 50 char overlap: prevents awkward sentence breaks
• Recursive splitting: maintains natural paragraph/sentence flow

Audio Merging​

• In-memory concat: fast for typical story lengths (< 50KB per chunk)
• Buffer pooling: efficient memory usage with Buffer.concat()
• No intermediate files: everything happens in memory

R2 Upload​

• Direct buffer upload: no filesystem writes
• Parallel processing: upload doesn’t block audio generation
• Public CDN URLs: instant global availability
• Cost: ~$0.015/GB storage,$0.00/GB egress (first 10GB/month)

Metadata Parsing​

• Optional duration: parse only if needed
• Non-blocking: failures logged but don’t break flow
• File size tracking: useful for analytics and storage optimization

Error Handling​

try {
  const result = await generateOpenAIAudio({ ... });

  if (!result.success) {
    // Handle OpenAI API errors
    console.error(result.message);
    return;
  }

  if (result.audioUrl) {
    // Use R2 URL (preferred)
    setAudioSrc(result.audioUrl);
  } else if (result.audioBuffer) {
    // Fallback to base64
    const blob = base64ToBlob(result.audioBuffer);
    setAudioSrc(URL.createObjectURL(blob));
  }
} catch (error) {
  // Network or authentication errors
  console.error("TTS Error:", error);
}

Common errors:

• 401 Unauthorized: invalid OpenAI API key
• 429 Rate Limited: too many requests (use exponential backoff)
• 413 Payload Too Large: text exceeds chunking limits
• 500 R2 Upload Failed: check R2 credentials and bucket config

Production Checklist​

• OpenAI API key configured
• R2 credentials set up
• R2 bucket created and public domain configured
• CORS configured for R2 bucket
• Database schema includes audio columns
• Rate limiting implemented for TTS endpoints
• Error tracking (Sentry, LogRocket, etc.)
• Audio duration parsing tested
• Chunking tested with various text lengths
• Fallback to base64 tested when R2 unavailable

Cost Estimation​

OpenAI TTS Pricing (as of 2024):

• gpt-4o-mini-tts: ~$1 per 10,000 characters
• tts-1: ~$15 per 1M characters
• tts-1-hd: ~$30 per 1M characters

Cloudflare R2 Pricing:

• Storage: $0.015/GB/month
• Class A operations (writes): $4.50 per million
• Class B operations (reads): $0.36 per million
• Egress: Free (no bandwidth charges)

Example: 1000 stories × 2000 chars each

• OpenAI cost: 2M chars × $1/10K =$200
• R2 storage: ~200MB × $0.015 =$0.003/month
• R2 writes: 1000 × $4.50/1M =$0.0045

Conclusion​

This implementation provides a robust, production-ready solution for converting long-form text to audio with automatic cloud storage. Key benefits:

1. Handles long content: automatic chunking + merging
2. Multiple interfaces: GraphQL and REST for flexibility
3. Dual fallback: R2 URLs + base64 for reliability
4. Rich metadata: duration, voice, chunks tracked
5. Cost-effective: Cloudflare R2 eliminates egress fees
6. Developer-friendly: type-safe GraphQL, clean API

The pattern can be adapted for other use cases like podcast generation, audiobook creation, or voice-enabled content platforms.

Resources​

• OpenAI TTS API Documentation
• Cloudflare R2 Documentation
• AWS SDK v3 for JavaScript
• Mastra RAG Documentation
• music-metadata npm package

Live Implementation​

This implementation is used in production at:

• Research Thera - AI-powered therapeutic audio platform
• GitHub Repository - Open source implementation

In the rapidly evolving landscape of Large Language Model (LLM) applications, prompt engineering has emerged as a critical discipline. As teams scale their AI applications, managing prompts across different versions, environments, and use cases becomes increasingly complex. This is where LangSmith's prompt management capabilities shine.

LangSmith, developed by LangChain, provides a comprehensive platform for managing, versioning, and collaborating on prompts—effectively bringing software engineering best practices to the world of prompt engineering.

The Challenge of Prompt Management​

Why Prompt Management Matters​

Prompts are the primary interface between your application and LLMs. As your AI application grows, you'll face several challenges:

1. Version Control: Tracking changes to prompts over time and understanding what worked and what didn't
2. Collaboration: Multiple team members working on prompts simultaneously
3. Environment Management: Different prompts for development, staging, and production
4. Performance Tracking: Understanding which prompt variations perform best
5. Reusability: Sharing successful prompts across different projects and teams
6. Rollback Safety: Ability to quickly revert to previous versions when something breaks

Without proper prompt management, these challenges can lead to:

• Lost productivity from recreating prompts that were deleted or modified
• Difficulty debugging when prompt changes cause unexpected behavior
• Lack of visibility into what prompts are being used in production
• Inability to A/B test different prompt variations systematically

LangSmith Prompt Management Features​

1. Git-Like Version Control​

LangSmith treats prompts as versioned artifacts, similar to how Git manages code. Each change to a prompt creates a new commit with a unique hash.

Key Features:

• Commit History: Every modification is tracked with metadata
• Diff Viewing: Compare versions to see exactly what changed
• Rollback: Revert to any previous version instantly
• Branching: Work on experimental prompts without affecting production

// Example from the codebase
export async function fetchLangSmithPromptCommit(
  promptIdentifier: string,
  options?: { includeModel?: boolean },
): Promise<LangSmithPromptCommit> {
  const client = getLangSmithClient();

  const commit = await client.pullPromptCommit(promptIdentifier, options);

  return {
    owner: commit.owner,
    promptName: commit.repo,
    commitHash: commit.commit_hash,
    manifest: commit.manifest,
    examples: commit.examples,
  };
}

2. Collaborative Prompt Repositories​

LangSmith organizes prompts into repositories, enabling team collaboration:

• Public Prompts: Share prompts with the broader community
• Private Prompts: Keep proprietary prompts within your organization
• Social Features: Like, download, and fork popular prompts
• Discovery: Search and explore prompts created by others

// Listing prompts with filters
export async function listLangSmithPrompts(options?: {
  isPublic?: boolean;
  isArchived?: boolean;
  query?: string;
}): Promise<LangSmithPrompt[]> {
  const client = getLangSmithClient();
  const prompts: LangSmithPrompt[] = [];

  for await (const prompt of client.listPrompts(options)) {
    prompts.push({
      id: prompt.id,
      fullName: prompt.full_name,
      isPublic: prompt.is_public,
      tags: prompt.tags,
      numLikes: prompt.num_likes,
      numDownloads: prompt.num_downloads,
      // ... other metadata
    });
  }

  return prompts;
}

3. Rich Metadata and Organization​

Prompts in LangSmith can be enriched with metadata:

• Tags: Categorize prompts by use case, domain, or team
• Descriptions: Document the purpose and usage
• README: Provide comprehensive documentation
• Examples: Include sample inputs and outputs

type LangSmithPrompt {
  id: String!
  promptHandle: String!
  fullName: String!
  description: String
  readme: String
  tags: [String!]!
  numCommits: Int!
  lastCommitHash: String
  createdAt: String!
  updatedAt: String!
}

4. User Ownership and Access Control​

LangSmith implements robust ownership and permission models:

export function ensureUserPromptIdentifier(
  promptIdentifier: string,
  userEmail: string,
): string {
  const userHandle = toUserHandle(userEmail);

  // Prompts are namespaced by owner
  if (promptIdentifier.includes("/")) {
    const [owner, ...rest] = promptIdentifier.split("/");

    // Validate owner matches user
    if (owner !== userHandle) {
      throw new Error(`Prompt identifier owner does not match your handle`);
    }
    return promptIdentifier;
  }

  // Add user's handle as prefix
  return `${userHandle}/${promptIdentifier}`;
}

5. Integration with LLM Workflows​

LangSmith prompts integrate seamlessly into your application:

// Create a prompt
const prompt = await createLangSmithPrompt("user/customer-support-classifier", {
  description: "Classifies customer support tickets",
  tags: ["support", "classification", "production"],
  isPublic: false,
});

// Push a new version
await pushLangSmithPrompt("user/customer-support-classifier", {
  object: {
    template: "Classify this ticket: {ticket_text}",
    model: "gpt-4",
    temperature: 0.2,
  },
  description: "Improved classification accuracy",
});

Best Practices for Prompt Management​

1. Version Your Prompts Deliberately​

Treat prompt changes with the same rigor as code changes:

• Semantic Versioning: Use meaningful version identifiers
• Commit Messages: Write clear descriptions of what changed and why
• Test Before Merging: Validate prompts in development before promoting to production

2. Use Tags Strategically​

Tags are powerful for organization and filtering:

const productionTags = [
  "env:production",
  "use-case:classification",
  "model:gpt-4",
  "team:support",
  "user:alice@example.com",
];

Recommended Tag Categories:

• Environment: env:dev, env:staging, env:prod
• Use Case: use-case:summarization, use-case:extraction
• Model: model:gpt-4, model:claude-3
• Owner: owner:team-name, user:email@domain.com
• Status: status:experimental, status:stable

3. Document Your Prompts​

Good documentation is essential for team collaboration:

await createLangSmithPrompt("user/email-generator", {
  description: "Generates professional email responses",
  readme: `
# Email Generator Prompt

## Purpose
Generates professional, context-aware email responses for customer inquiries.

## Input Format
- customer_name: String
- inquiry_type: "support" | "sales" | "billing"
- context: String (previous conversation)

## Output Format
Professional email in plain text

## Performance Notes
- Works best with GPT-4
- Temperature: 0.7 for natural variation
- Max tokens: 500

## Examples
See attached examples for common scenarios.
  `,
  tags: ["communication", "email", "production"],
});

4. Implement User-Based Filtering​

When building multi-tenant applications, filter prompts by user:

// From the GraphQL resolver
const userPrompts = allPrompts.filter((prompt) => {
  // Check user tags
  const hasUserTag = prompt.tags.some(
    (tag) =>
      tag.includes(`user:${context.userEmail}`) ||
      tag.includes(`owner:${context.userEmail}`),
  );

  // Check if owner matches
  const isOwner = prompt.owner === context.userEmail;

  return hasUserTag || isOwner;
});

5. Handle Permissions Gracefully​

LangSmith requires specific API permissions. Handle errors clearly:

export async function createLangSmithPrompt(
  promptIdentifier: string,
  input?: CreateLangSmithPromptInput,
): Promise<LangSmithPrompt> {
  try {
    return await client.createPrompt(promptIdentifier, input);
  } catch (error: any) {
    if (
      error?.message?.includes("403") ||
      error?.message?.includes("Forbidden")
    ) {
      throw new Error(
        "LangSmith API key lacks 'Prompt Engineering' permissions. " +
          "Please generate a new API key with Read, Write, AND " +
          "Prompt Engineering scopes.",
      );
    }
    throw error;
  }
}

6. Use the Hub for Reusable Prompts​

LangSmith Hub allows sharing prompts across projects:

• Fork Popular Prompts: Start with community-tested templates
• Share Successful Patterns: Contribute back to the community
• Cross-Project Reuse: Reference the same prompt from multiple applications

Architectural Patterns​

Pattern 1: GraphQL Wrapper for Type Safety​

Wrap the LangSmith SDK with a GraphQL layer for type-safe client access:

extend type Mutation {
  pushLangSmithPrompt(
    promptIdentifier: String!
    input: PushLangSmithPromptInput
  ): String!
}

input PushLangSmithPromptInput {
  object: JSON
  parentCommitHash: String
  description: String
  tags: [String!]
  isPublic: Boolean
}

Benefits:

• Type safety across frontend and backend
• Centralized permission checking
• Consistent error handling
• Easy to mock for testing

Pattern 2: Singleton Client Pattern​

Use a singleton to manage the LangSmith client:

let singleton: Client | null = null;

export function getLangSmithClient(): Client {
  if (!singleton) {
    const apiKey = process.env.LANGSMITH_API_KEY;
    if (!apiKey) {
      throw new Error("LANGSMITH_API_KEY required");
    }
    singleton = new Client({ apiKey });
  }
  return singleton;
}

Benefits:

• Single point of configuration
• Connection pooling
• Consistent client instance across requests

Pattern 3: User Namespace Enforcement​

Automatically namespace prompts by user to prevent conflicts:

export function toUserHandle(userEmail: string): string {
  return userEmail
    .trim()
    .toLowerCase()
    .replace(/[^a-z0-9@._-]+/g, "-")
    .replace(/@/g, "-at-")
    .replace(/\./g, "-");
}

// alice@company.com -> alice-at-company-com/my-prompt
const identifier = `${toUserHandle(userEmail)}/${promptName}`;

Integration with Your Application​

Step 1: Environment Setup​

# .env.local
LANGSMITH_API_KEY=lsv2_pt_...
LANGSMITH_API_URL=https://api.smith.langchain.com

Step 2: Client Initialization​

import { Client } from "langsmith";

const client = new Client({
  apiKey: process.env.LANGSMITH_API_KEY,
  apiUrl: process.env.LANGSMITH_API_URL,
});

Step 3: Create and Version Prompts​

// Initial creation
await client.pushPrompt("my-org/summarizer", {
  object: {
    _type: "prompt",
    input_variables: ["text"],
    template: "Summarize: {text}",
  },
  tags: ["v1", "production"],
  isPublic: false,
});

// Update with new version
await client.pushPrompt("my-org/summarizer", {
  object: {
    _type: "prompt",
    input_variables: ["text", "style"],
    template: "Summarize in {style} style: {text}",
  },
  tags: ["v2", "production"],
  isPublic: false,
});

Step 4: Retrieve and Use Prompts​

// Get latest version
const commit = await client.pullPromptCommit("my-org/summarizer");

// Use the prompt
const template = commit.manifest.template;
const rendered = template
  .replace("{text}", article)
  .replace("{style}", "concise");

Monitoring and Analytics​

LangSmith provides built-in analytics for prompts:

• Usage Tracking: See how often each prompt is used
• Performance Metrics: Track latency and success rates
• Version Comparison: Compare metrics across versions
• Cost Analysis: Monitor token usage per prompt

interface LangSmithPrompt {
  numViews: number;
  numDownloads: number;
  numLikes: number;
  numCommits: number;
  lastUsedAt?: string;
}

Common Pitfalls and Solutions​

Pitfall 1: Missing Permissions​

Problem: API key doesn't have "Prompt Engineering" scope

Solution:

// Always check permissions and provide clear error messages
if (error?.message?.includes("403")) {
  throw new Error(
    "Generate API key with Prompt Engineering permissions at " +
      "https://smith.langchain.com/settings",
  );
}

Pitfall 2: Unbounded Listing​

Problem: Listing all prompts can be slow or timeout

Solution:

// Limit results and provide pagination
const MAX_PROMPTS = 100;
for await (const prompt of client.listPrompts(options)) {
  prompts.push(prompt);
  if (prompts.length >= MAX_PROMPTS) break;
}

Pitfall 3: Naming Conflicts​

Problem: Multiple users trying to create prompts with same name

Solution:

// Always namespace by user/organization
const namespaced = `${organization}/${promptName}`;
await client.createPrompt(namespaced, options);

Advanced Features​

Prompt Playground Integration​

LangSmith provides a visual playground for testing prompts:

1. Edit prompt templates interactively
2. Test with sample inputs
3. Compare outputs across models
4. Iterate quickly without code changes
5. Save successful variations as new commits

Example Sets​

Attach example inputs/outputs to prompts:

await client.pushPrompt("classifier", {
  object: promptTemplate,
  examples: [
    {
      inputs: { text: "I love this product!" },
      outputs: { sentiment: "positive" },
    },
    {
      inputs: { text: "Terrible experience" },
      outputs: { sentiment: "negative" },
    },
  ],
});