Create Stunning Animations with the AniPortrait Cognitive Actions

25 Apr 2025
Create Stunning Animations with the AniPortrait Cognitive Actions

Creating engaging and animated content can significantly enhance user experiences in applications, especially those leveraging multimedia elements. The AniPortrait framework under the spec cjwbw/aniportrait-audio2vid offers a powerful Cognitive Action that enables developers to generate high-quality photorealistic portrait animations from audio and reference images. This blog post will guide you through this action, demonstrating its capabilities and how you can integrate it into your applications.

Prerequisites

Before diving into the Cognitive Actions, ensure you have:

  • An API key for the Cognitive Actions platform.
  • Basic knowledge of working with REST APIs.

For authentication, you typically include your API key in the request headers, allowing you to securely access the functionality provided by the Cognitive Actions.

Cognitive Actions Overview

Generate AniPortrait

The Generate AniPortrait action allows you to create stunning portrait animations driven by audio input and reference images. This action supports face reenactment from a video and offers configurable parameters to customize the output video dimensions, frame rate, and guidance scale.

Input

The input for this action requires the following fields:

  • audio (string, required): URI of the input audio file. This file is essential for processing.
  • image (string, required): URI of the input image file. This file is also essential for processing.
  • seed (integer, optional): A random seed for generating the output. If not provided, a random seed is used.
  • steps (integer, optional): Number of inference steps for the process. Default is 25.
  • width (integer, optional): Width of the output video in pixels. Default is 512 pixels.
  • height (integer, optional): Height of the output video in pixels. Default is 512 pixels.
  • guidanceScale (number, optional): Scale factor for classifier-free guidance. Default is 3.5.
  • framesPerSecond (integer, optional): Frame rate of the output video in frames per second. Default is 30 FPS.

Example Input:

{
  "audio": "https://replicate.delivery/pbxt/KfVpX7wBikBZbAqVyur6eBPFPzTeDExcl12VGYEnJgvecHSU/lyl.wav",
  "image": "https://replicate.delivery/pbxt/KfVpX606yiO1dn0ZDR8LAPcFsMFBmynKD5IEXWy2CFZnmzel/lyl.png",
  "steps": 25,
  "width": 512,
  "height": 512,
  "guidanceScale": 3.5,
  "framesPerSecond": 30
}

Output

The output of this action typically includes:

  • pose (string): The URI of the generated pose video.
  • video (string): The URI of the generated animation video.

Example Output:

{
  "pose": "https://assets.cognitiveactions.com/invocations/b2259abc-0b94-444d-8ace-09be8ab5ce2f/03af97ab-b3b9-46c4-937a-93fd4d23a9c0.mp4",
  "video": "https://assets.cognitiveactions.com/invocations/b2259abc-0b94-444d-8ace-09be8ab5ce2f/63c05f2b-b3f5-4ec1-91af-3975199527f1.mp4"
}

Conceptual Usage Example (Python)

Here's a conceptual Python code snippet demonstrating how you might call the Generate AniPortrait action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "a3aba99c-8959-40e4-b62c-d7bb4dd5858c" # Action ID for Generate AniPortrait

# Construct the input payload based on the action's requirements
payload = {
    "audio": "https://replicate.delivery/pbxt/KfVpX7wBikBZbAqVyur6eBPFPzTeDExcl12VGYEnJgvecHSU/lyl.wav",
    "image": "https://replicate.delivery/pbxt/KfVpX606yiO1dn0ZDR8LAPcFsMFBmynKD5IEXWy2CFZnmzel/lyl.png",
    "steps": 25,
    "width": 512,
    "height": 512,
    "guidanceScale": 3.5,
    "framesPerSecond": 30
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this snippet, replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key. The payload is structured to match the input requirements of the Generate AniPortrait action. The endpoint URL and request structure are illustrative and should be adjusted according to the actual API documentation.

Conclusion

The Generate AniPortrait action provides a robust solution for creating captivating animated portraits driven by audio and images. By leveraging this Cognitive Action, developers can enhance their applications with engaging multimedia content, opening doors to numerous creative possibilities.

As you experiment with these capabilities, consider exploring different audio and image combinations to see how the output varies. Happy coding!

← Back to All Articles