voice-cloning

Create Multilingual Voice Clones Effortlessly with Xtts V2

27 Apr 2025
Create Multilingual Voice Clones Effortlessly with Xtts V2

In a world that increasingly values personalization and accessibility, the ability to generate high-quality voice clones in multiple languages has become a game changer. Xtts V2 offers developers powerful Cognitive Actions that utilize advanced text-to-speech technology to create multilingual voice clones from just a short audio sample. This capability not only enhances user experience but also opens up new avenues for applications in various industries, including entertainment, education, and customer service.

Imagine being able to create a unique voice for your application or service that resonates with users in their preferred language. With Xtts V2, this is made possible. By inputting a short audio clip along with the desired text and language, developers can generate a seamless and natural-sounding voice clone that can articulate content in multiple languages, including English, Spanish, French, and more.

Prerequisites

To get started with Xtts V2, you'll need a Cognitive Actions API key and a basic understanding of making API calls. This will ensure you can easily integrate the voice-cloning functionalities into your applications.

Generate Multilingual Voice Clone

The Generate Multilingual Voice Clone action allows you to create a voice clone that can speak in different languages based on a provided audio sample. This action leverages the Coqui XTTS-v2 model to synthesize speech that closely mimics the original speaker's voice, making it ideal for applications requiring localized content delivery.

Input Requirements

The input for this action requires a structured object that includes:

  • text: The textual content you want to convert to speech. It’s essential to ensure clarity in the text to improve the synthesis output.
  • speaker: A URI pointing to the original speaker's audio file in supported formats such as wav, mp3, m4a, ogg, and flv.
  • language: A language code that specifies the language for the synthesized speech output, with options including English, Spanish, French, German, and more.
  • cleanupVoice: A boolean indicating whether to apply denoising to enhance audio quality; defaults to false.

Expected Output

The output from this action is a URI linking to the generated voice clone audio file, which can be used in your applications for playback or other functionalities.

Use Cases for this Action

  • Personalized User Experiences: Create unique voice interactions in applications, enhancing user engagement by allowing users to hear content in a voice they're familiar with.
  • Multilingual Support: Develop applications that cater to a global audience by providing localized voice options, making content more accessible.
  • Content Creation: Use voice clones for audio books, podcasts, or any content that benefits from a natural-sounding voice, saving time and resources in the production process.
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "c1eda2cc-570a-4f4e-8c50-3944e61163ff" # Action ID for: Generate Multilingual Voice Clone

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "text": "Hi there, I'm your new voice clone. Try your best to upload quality audio",
  "speaker": "https://replicate.delivery/pbxt/Jt79w0xsT64R1JsiJ0LQRL8UcWspg5J4RFrU6YwEKpOT1ukS/male.wav",
  "language": "en",
  "cleanupVoice": false
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")

Conclusion

Xtts V2's voice cloning capabilities empower developers to create highly personalized and multilingual audio experiences. With just a short audio sample, you can generate realistic voice clones that enhance user interaction and broaden the reach of your applications. Whether for entertainment, education, or customer service, the ability to synthesize speech in various languages opens up new possibilities. Start integrating Xtts V2 into your projects today and elevate your applications to the next level!

← Back to All Articles