Enhance Your Video Content with Audio Using MMAudio Cognitive Actions

24 Apr 2025
Enhance Your Video Content with Audio Using MMAudio Cognitive Actions

In today's digital landscape, engaging video content is essential for capturing audience attention. The MMAudio Cognitive Actions enable developers to enhance their video projects by seamlessly adding contextually appropriate audio. By utilizing advanced AI models, these actions provide high-quality audio generation that synchronizes perfectly with video actions and environments. In this article, we'll explore how to leverage these capabilities to elevate your multimedia applications.

Prerequisites

Before you can utilize the MMAudio Cognitive Actions, ensure you have the necessary setup in place:

  1. API Key: You will need an API key to authenticate your requests. This key will be passed in the headers of your HTTP requests.
  2. Endpoint: Familiarize yourself with the endpoint for executing Cognitive Actions (the exact URL will depend on your service provider).

Authentication typically involves including your API key in the request headers like so:

headers = {
    "Authorization": f"Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}

Cognitive Actions Overview

Add Audio to Video

The Add Audio to Video action transforms video content into high-quality audio using the MMAudio Video-to-Audio Synthesis Model. This AI model generates audio that is tailored to the video's context, ensuring that sounds align perfectly with the visual elements.

Input

The input for this action consists of several parameters, allowing for flexible audio generation:

  • seed (integer, optional): A random seed for generating consistent results. Use -1 for completely random behavior. Defaults to 0.
  • video (string, optional): A URI link to the video file that will be used for audio generation. The file must be accessible online.
  • prompt (string, optional): A text prompt guiding the content and context of the generated audio. Defaults to an empty string.
  • duration (number, optional): Length of the generated audio output in seconds. Defaults to 8 seconds.
  • numberOfSteps (integer, optional): The number of steps used during the inference process to generate audio. Defaults to 25 steps.
  • negativePrompt (string, optional): A text prompt specifying sounds to avoid in the generated audio. Defaults to "music".
  • configurationStrength (number, optional): Controls the guidance strength (CFG) during audio generation. A higher value results in closer adherence to the prompt. Defaults to 4.5.

Example Input:

{
  "seed": -1,
  "video": "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_galloping.mp4",
  "prompt": "galloping",
  "duration": 8,
  "numberOfSteps": 25,
  "negativePrompt": "music",
  "configurationStrength": 4.5
}

Output

The typical output of this action is a URI link to the generated audio file, which will be contextually appropriate based on the input video and prompts.

Example Output:

https://assets.cognitiveactions.com/invocations/4ae693fa-da2a-44bd-aaef-d0cd6d3adf43/3a686e74-0b9f-4937-ae9c-1d78cd5f8a08.mp4

Conceptual Usage Example (Python)

Here’s how you might call the Add Audio to Video action using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "b20eade5-1305-4c4a-918c-d1afea5034b7" # Action ID for Add Audio to Video

# Construct the input payload based on the action's requirements
payload = {
    "seed": -1,
    "video": "https://huggingface.co/hkchengrex/MMAudio/resolve/main/examples/sora_galloping.mp4",
    "prompt": "galloping",
    "duration": 8,
    "numberOfSteps": 25,
    "negativePrompt": "music",
    "configurationStrength": 4.5
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet, replace the placeholder for COGNITIVE_ACTIONS_API_KEY with your actual API key. The payload is constructed based on the required input schema, ensuring the parameters align with the expected format.

Conclusion

The MMAudio Cognitive Actions provide an innovative way to enrich your video content with contextually relevant audio. By leveraging the Add Audio to Video action, developers can create immersive experiences that captivate audiences. Consider exploring further use cases, such as integrating this action into video editing applications or enhancing multimedia presentations. Start experimenting today and unlock the full potential of your video content!

← Back to All Articles