audio-processing

Effortlessly Extract Sounds with Zero Shot Audio Source Separation

25 Apr 2025
Effortlessly Extract Sounds with Zero Shot Audio Source Separation

In the world of audio processing, the ability to isolate specific sounds from complex mixes can significantly enhance production quality and creativity. The Zero Shot Audio Source Separation service empowers developers to achieve this by allowing for the separation of any audio source from a sound track using query samples. Leveraging advanced zero-shot learning techniques, this service improves flexibility and accuracy in audio extraction tasks. Whether you're a music producer looking to isolate an instrument or a sound designer needing specific sound effects from a mix, this tool simplifies your workflow and enhances your creative possibilities.

Prerequisites

To get started with Zero Shot Audio Source Separation, you'll need an API key for the Cognitive Actions service and a basic understanding of making API calls.

Perform Audio Source Separation with Query Samples

This action allows you to separate any audio source from a sound track by providing a query sample. It excels in tasks such as extracting specific instruments or sounds from complex audio mixes, making it an invaluable tool for audio engineers and developers alike.

Input Requirements

To use this action, you need to provide the following inputs:

  • mixFile: A URI pointing to the reference sound file from which the source is to be extracted. For example, a link to a mixed audio track.
  • queryFile: A URI pointing to the query sound file that you want to search for and extract from the mix file. This could be a sample of the instrument or sound you're interested in.

Example input:

{
  "mixFile": "https://replicate.delivery/mgxm/37dba301-948f-493d-89d6-7adacb8160ad/SevenNationArmy_trimmed.mp3",
  "queryFile": "https://replicate.delivery/mgxm/2362d82d-d016-445d-b9e2-b99f3e3f70ac/bass.wav"
}

Expected Output

The output will be a URI pointing to the extracted audio source, allowing you to access the isolated sound. For example, the output could be a link to a WAV file containing just the bass extracted from the mixed track.

Example output:

https://assets.cognitiveactions.com/invocations/99350286-f2e7-48b6-9406-834d619189c2/6274d381-8d1c-4951-a404-1d3d07c2b1ab.wav

Use Cases for this Specific Action

  • Music Production: Producers can extract specific instruments from a mix, enabling them to remix or enhance tracks without starting from scratch.
  • Sound Design: Designers can isolate sound effects from background audio, providing more control over the soundscapes they create.
  • Educational Purposes: Music educators can use this tool to demonstrate instrument sounds or teach mixing techniques by isolating specific components of a track.
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "ad56987c-f2ed-44f6-87ee-d20f0f5f70d7" # Action ID for: Perform Audio Source Separation with Query Samples

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "mixFile": "https://replicate.delivery/mgxm/37dba301-948f-493d-89d6-7adacb8160ad/SevenNationArmy_trimmed.mp3",
  "queryFile": "https://replicate.delivery/mgxm/2362d82d-d016-445d-b9e2-b99f3e3f70ac/bass.wav"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")

Conclusion

The Zero Shot Audio Source Separation service provides a powerful and flexible solution for audio processing needs. By allowing developers to separate sounds from complex mixes using query samples, it opens up a wide range of creative and practical applications. Whether you're looking to improve your music production, sound design, or educational tools, this service can streamline your workflow and enhance your projects. Start integrating this action into your applications today and unlock new possibilities in audio manipulation!

← Back to All Articles