Transform Your Applications with Voice Synthesis Using CosyVoice 2.0 Actions

23 Apr 2025
Transform Your Applications with Voice Synthesis Using CosyVoice 2.0 Actions

In the rapidly evolving world of AI, text-to-speech technology has become a cornerstone for building engaging and interactive applications. The CosyVoice 2.0 actions offer developers a powerful solution for scalable streaming speech synthesis. With ultra-low latency, high accuracy, and multilingual support, this upgraded API enhances pronunciation and prosody, reducing character errors by up to 50%. In this article, we will explore how to effectively integrate the CosyVoice 2.0 actions into your applications.

Prerequisites

Before diving into the integration of CosyVoice 2.0 actions, ensure you have:

  • An API key for accessing the Cognitive Actions platform.
  • Familiarity with JSON format, as the input and output will be structured in JSON.
  • A basic understanding of making HTTP requests in Python or your preferred programming language.

For authentication, you will typically pass your API key in the headers of your request to securely access the API endpoints.

Cognitive Actions Overview

Generate Voice with CosyVoice 2.0

The Generate Voice with CosyVoice 2.0 action enables you to create natural-sounding speech from written text using voice cloning technologies. This action supports various tasks, including zero-shot voice cloning and instructed voice generation, making it flexible for multiple use cases.

Input:

The input for this action is structured as follows:

  • task (string, required): Specifies the type of voice processing task, with options including:
    • "zero-shot voice clone"
    • "cross-lingual voice clone"
    • "Instructed Voice Generation" (default: "zero-shot voice clone")
  • instruction (string, optional): Specific instructions for the "Instructed Voice Generation" task. This field can be left empty if not applicable.
  • sourceAudio (string, required): A URI pointing to the source audio file that serves as the reference for the voice cloning process.
  • sourceTranscript (string, required): A textual transcript of the source audio. This is crucial for tasks requiring audio analysis.
  • textToSpeechText (string, required): The text intended for audio generation that will be converted into synthesized speech.

Example Input:

{
  "task": "zero-shot voice clone",
  "instruction": "",
  "sourceAudio": "https://replicate.delivery/pbxt/MCyjoMjdC1WlvhMHzNhylKOrz97Vy0dFRM8ciNtq5siWG3pj/En_3_prompt.wav",
  "sourceTranscript": "I'm so happy I got to do this. I really wanted to work with Tom Hooper. I know that he records live and he films and records your vocals live. It's such an interesting thing to me and I wanted to see him work. I had actually done screen tests for Les Mis.",
  "textToSpeechText": "Every stage is a fresh adventure, and as the lights ignite, it's an unspoken pact between me and the audience, weaving unforgettable nights where dreams meet reality."
}

Output:

The action typically returns a URL linking to the generated audio file, which is a WAV file containing the synthesized speech.

Example Output:

https://assets.cognitiveactions.com/invocations/ceebc846-d84d-453a-b8b3-a09bdb4f320a/e6d62f9b-a5ef-4e3a-9cc6-553abc5f8c13.wav

Conceptual Usage Example (Python):

Here’s how you might call the Generate Voice with CosyVoice 2.0 action using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "41e7c6f0-f544-48a5-a409-7b14eb84f946"  # Action ID for Generate Voice with CosyVoice 2.0

# Construct the input payload based on the action's requirements
payload = {
    "task": "zero-shot voice clone",
    "instruction": "",
    "sourceAudio": "https://replicate.delivery/pbxt/MCyjoMjdC1WlvhMHzNhylKOrz97Vy0dFRM8ciNtq5siWG3pj/En_3_prompt.wav",
    "sourceTranscript": "I'm so happy I got to do this. I really wanted to work with Tom Hooper. I know that he records live and he films and records your vocals live. It's such an interesting thing to me and I wanted to see him work. I had actually done screen tests for Les Mis.",
    "textToSpeechText": "Every stage is a fresh adventure, and as the lights ignite, it's an unspoken pact between me and the audience, weaving unforgettable nights where dreams meet reality."
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this example, replace the YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key. The payload is built according to the required input schema for the action. The URL and request structure are illustrative, so ensure you adapt them according to the actual API specifications.

Conclusion

The CosyVoice 2.0 actions significantly enhance the capabilities of your applications by providing high-quality voice synthesis. By integrating these actions, developers can create more engaging user experiences through natural-sounding speech. Whether you are building applications for entertainment, education, or accessibility, the versatility of CosyVoice 2.0 opens up new possibilities for innovation. Start exploring these actions today and bring your applications to life!

← Back to All Articles