Enhance Your Applications with venkr/whisperx-diarization: A Developer's Guide to Transcribing and Diarizing Audio

25 Apr 2025
Enhance Your Applications with venkr/whisperx-diarization: A Developer's Guide to Transcribing and Diarizing Audio

In the realm of audio processing, the venkr/whisperx-diarization offers powerful Cognitive Actions that leverage advanced AI technologies for tasks such as audio transcription and speaker diarization. This guide will help developers understand how to integrate the "Transcribe and Diarize Audio" action into their applications, enabling them to efficiently convert spoken content into written text while identifying individual speakers.

Introduction

The venkr/whisperx-diarization API provides developers with sophisticated tools for transforming audio recordings into actionable text data. By utilizing the latest advancements in Whisper-Large-V2 and Pyannote 3.0, these Cognitive Actions enhance transcription accuracy and improve speaker identification. The benefits of using this API include saving time on manual transcription, improving accessibility, and enabling content analysis.

Prerequisites

Before getting started, ensure you have the following:

  • An API key for the Cognitive Actions platform.
  • Basic knowledge of JSON and HTTP requests.

Authentication typically involves including your API key in the request headers. Here’s a conceptual overview of how authentication works:

headers = {
    "Authorization": "Bearer YOUR_COGNITIVE_ACTIONS_API_KEY",
    "Content-Type": "application/json"
}

Cognitive Actions Overview

Transcribe and Diarize Audio

Description: This action utilizes Whisper-Large-V2 with Pyannote 3.0 for audio transcription and speaker diarization. The enhancements include improved speaker identification and transcription accuracy.

Category: Speech-to-Text

Input

The input schema requires the following fields:

  • audio (required): The URI of the audio file to be processed. It must point to a valid audio file format (.mp3, .wav, etc.).
  • debug (optional): A boolean to enable debugging information during processing. Defaults to false.
  • diarize (optional): A boolean that determines whether speaker diarization is included in the output. Defaults to true.
  • onlyText (optional): A boolean that specifies if only transcribed text should be returned. Defaults to false.
  • batchSize (optional): An integer that defines the number of audio samples processed concurrently for transcription. Defaults to 32.

Example Input:

{
  "audio": "https://replicate.delivery/pbxt/JuhRLXsJ6GoX6VJMPHZ5GTAdFixXCQgfzpJppOHvc4vUv46O/3.mp3",
  "debug": true,
  "diarize": false,
  "onlyText": false,
  "batchSize": 32
}

Output

The action returns a list of transcribed segments, each containing:

  • start: The start time of the segment in seconds.
  • end: The end time of the segment in seconds.
  • text: The transcribed text of that segment.

Example Output:

[
  {
    "start": 1.425,
    "end": 31.408,
    "text": "Hey guys, this week's episode of The Read is being brought to you by Talkspace..."
  },
  {
    "start": 52.381,
    "end": 80.469,
    "text": "Okay, are you ready? As I'll ever be..."
  }
]

Conceptual Usage Example (Python)

Here's how a developer might call the Transcribe and Diarize Audio action using the Cognitive Actions API:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "232749ad-68aa-428d-8791-1574e31b8409"  # Action ID for Transcribe and Diarize Audio

# Construct the input payload based on the action's requirements
payload = {
    "audio": "https://replicate.delivery/pbxt/JuhRLXsJ6GoX6VJMPHZ5GTAdFixXCQgfzpJppOHvc4vUv46O/3.mp3",
    "debug": true,
    "diarize": false,
    "onlyText": false,
    "batchSize": 32
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

This Python snippet demonstrates how to structure a request to the Cognitive Actions API, passing in the required input JSON payload. The focus is on how to set up the request and manage the response effectively.

Conclusion

The venkr/whisperx-diarization Cognitive Action for transcribing and diarizing audio provides developers with a robust solution for converting audio content into organized text format. By leveraging this technology, applications can enhance accessibility, automate transcription tasks, and improve user experiences. As you integrate these actions, consider how you can further utilize the data extracted to drive insights and innovations in your projects.

← Back to All Articles