Mastering Audio Source Separation with Soykertje/Spleeter Cognitive Actions

In the world of audio processing, separating different components of a mixed audio file can be crucial for various applications, such as music production, remixing, or even karaoke. The Soykertje/Spleeter Cognitive Actions offer developers a powerful tool to achieve this by utilizing Spleeter, a source separation library by Deezer. This library leverages pretrained models with TensorFlow support to effortlessly separate audio into distinct components like vocals and accompaniment. In this article, we’ll explore how to integrate this Cognitive Action into your applications.

Prerequisites

Before you start using the Soykertje/Spleeter Cognitive Actions, ensure you have the following:

• An API key for the Cognitive Actions platform, which you’ll include in your request headers for authentication.
• A valid audio file URI that you want to process. Make sure the audio file is accessible via a public URL.

In your API requests, you’ll typically pass the API key in the headers like this:

Authorization: Bearer YOUR_COGNITIVE_ACTIONS_API_KEY
Content-Type: application/json

Cognitive Actions Overview

Perform Audio Source Separation

The Perform Audio Source Separation action allows you to use Spleeter to separate audio files into their constituent parts. This is particularly useful for isolating vocals from music, allowing for enhanced audio manipulation and editing.

• Category: Audio Processing

Input

The input schema for this action requires a single field:

• audio (string, required): The URI of the audio file you wish to process. It must be in a valid URI format.

Example Input:

{
  "audio": "https://replicate.delivery/pbxt/JQ8v4dbyBga3g3ZFXP8t73Ioz97xpbdwwJWFR93N9Hvhdb0i/Nettie%20-%20Type%20O%20Negative.mp3"
}

Output

Upon successful execution, the action returns a JSON object containing two links:

• vocals (string): A URI pointing to the separated vocal track.
• accompaniment (string): A URI pointing to the separated accompaniment track.

Example Output:

{
  "vocals": "https://assets.cognitiveactions.com/invocations/8f63e814-0123-4b53-8b3a-652ef93c26b2/55706c2e-fbe5-45cb-822a-92da1f70ab56.wav",
  "accompaniment": "https://assets.cognitiveactions.com/invocations/8f63e814-0123-4b53-8b3a-652ef93c26b2/20c8cf26-9864-4eb0-bc4b-82d85f218d73.wav"
}

Conceptual Usage Example (Python)

Here’s how you might call this action using Python. This example constructs the necessary input payload based on the action's requirements and sends a request to the Cognitive Actions execution endpoint.

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "eb273287-51b0-4af0-9cb9-870812c5a55a"  # Action ID for Perform Audio Source Separation

# Construct the input payload based on the action's requirements
payload = {
    "audio": "https://replicate.delivery/pbxt/JQ8v4dbyBga3g3ZFXP8t73Ioz97xpbdwwJWFR93N9Hvhdb0i/Nettie%20-%20Type%20O%20Negative.mp3"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this example, replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key and ensure that the endpoint URL corresponds to your Cognitive Actions service. The action ID is specified, and the input payload is constructed according to the required schema.

Conclusion

The Soykertje/Spleeter Cognitive Actions provide a straightforward way to achieve audio source separation, enabling developers to enhance audio applications significantly. By integrating these actions, you can separate vocals from music tracks, thereby unlocking a range of creative possibilities. Consider exploring this functionality in your next audio project, and enjoy the benefits of efficient audio processing!