Analyze Music Structures Effectively with the All-In-One Music Structure Analyzer

21 Apr 2025
Analyze Music Structures Effectively with the All-In-One Music Structure Analyzer

In today's digital landscape, understanding music structure is vital for applications in music analytics, production, and automated composition. The All-In-One Music Structure Analyzer by mir-aidj offers developers a powerful toolset for analyzing music tracks, enabling deep insights into tempo, beats, downbeats, and functional segments such as intro, verse, chorus, bridge, and outro. This blog post will delve into the capabilities of this integration, showcasing how you can leverage its Cognitive Actions to enhance your applications.

Prerequisites

Before you start using the Cognitive Actions provided by the All-In-One Music Structure Analyzer, you will need to ensure you have the following:

  • An API key for accessing the Cognitive Actions platform.
  • Basic understanding of RESTful API calls and JSON data format.

Authentication typically involves passing the API key in the request headers, allowing your application to securely interact with the service.

Cognitive Actions Overview

Perform Music Structure Analysis

The Perform Music Structure Analysis action is designed to analyze the structural aspects of music tracks, offering detailed insights into various segments of a song. This includes identifying the tempo, beats, downbeats, and functional labels for different parts of the music.

Input

The input for this action consists of several properties that dictate how the analysis will be performed:

{
  "model": "harmonix-all",
  "sonify": true,
  "visualize": true,
  "musicInput": "https://replicate.delivery/pbxt/K3iP4RhDPayT24NMYswahQ7kYfG1NS4vhNaF3PZVSZoLaSSY/x2mate.com%20-%20Sean%20Lennon.%20Parachute%20%28128%20kbps%29.mp3",
  "saveEmbeddings": false,
  "saveActivations": false,
  "includeEmbeddings": false,
  "includeActivations": false
}
  • model (string): Specifies which pretrained model to use. Default is harmonix-all.
  • sonify (boolean): Indicates whether to save sonifications. Default is false.
  • visualize (boolean): Indicates whether to save visualizations. Default is false.
  • musicInput (string): URI of the audio file to be analyzed. Must be a valid URI format.
  • saveEmbeddings (boolean): Indicates whether to save frame-level embeddings. Default is false.
  • saveActivations (boolean): Indicates whether to save frame-level raw activations. Default is false.
  • includeEmbeddings (boolean): Determines if embeddings should be included in the results. Default is false.
  • includeActivations (boolean): Determines if activations should be included in the results. Default is false.

Output

Upon successful execution, the action typically returns an array of URLs pointing to various outputs, including:

  1. A JSON file with detailed analysis results.
  2. A visualization image of the music structure.
  3. A sonified audio file if the sonification option was enabled.

For example, the output could look like this:

[
  "https://assets.cognitiveactions.com/invocations/ff5f3c14-2f30-4ccc-b3de-07939d337937/6d966c48-4e21-417d-becb-4d781d31a9f8.json",
  "https://assets.cognitiveactions.com/invocations/ff5f3c14-2f30-4ccc-b3de-07939d337937/7dd0e133-5552-46fa-8129-fb2f7aa59b21.png",
  "https://assets.cognitiveactions.com/invocations/ff5f3c14-2f30-4ccc-b3de-07939d337937/ca5f6a1e-ebcc-4a4b-8f2d-c312a87f03ee.mp3"
]

Conceptual Usage Example (Python)

Here’s how you might interact with the Cognitive Actions API in Python to execute the music structure analysis:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "8bca5624-4fbf-4a87-bf75-66b533ad4ea8"  # Action ID for Perform Music Structure Analysis

# Construct the input payload based on the action's requirements
payload = {
    "model": "harmonix-all",
    "sonify": True,
    "visualize": True,
    "musicInput": "https://replicate.delivery/pbxt/K3iP4RhDPayT24NMYswahQ7kYfG1NS4vhNaF3PZVSZoLaSSY/x2mate.com%20-%20Sean%20Lennon.%20Parachute%20%28128%20kbps%29.mp3",
    "saveEmbeddings": False,
    "saveActivations": False,
    "includeEmbeddings": False,
    "includeActivations": False
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this Python snippet, make sure to replace the COGNITIVE_ACTIONS_API_KEY and the endpoint URL with your actual credentials. The action ID and the input payload are structured correctly to call the music structure analysis.

Conclusion

The All-In-One Music Structure Analyzer offers powerful tools for developers looking to add music analysis capabilities to their applications. By leveraging the Perform Music Structure Analysis action, you can gain detailed insights into the structure of music tracks, enhancing your application's functionality. Consider exploring additional use cases such as automated music tagging, music education tools, or building interactive music experiences. Start integrating these Cognitive Actions today and unlock the potential of your music applications!

← Back to All Articles