Create Stunning Animated Portraits with the Live Portrait Cognitive Actions

21 Apr 2025
Create Stunning Animated Portraits with the Live Portrait Cognitive Actions

In the world of digital creativity, the ability to animate static images can open up new avenues for engagement and storytelling. The fofr/live-portrait spec provides developers with powerful Cognitive Actions that let you create animated portraits using a driving video as the source for facial movements. With efficient stitching and retargeting techniques, these actions allow for seamless animations that can enhance applications in gaming, social media, and virtual interactions.

Prerequisites

Before diving into the integration of the Cognitive Actions, you need to ensure you have the following in place:

  • An API key for the Cognitive Actions platform.
  • Basic knowledge of how to make HTTP requests to RESTful APIs.

Authentication typically involves passing your API key in the headers of your HTTP requests.

Cognitive Actions Overview

Animate Portrait Using Driving Video

The Animate Portrait Using Driving Video action allows you to create dynamic animated portraits by utilizing a driving video that controls facial movements. This action falls under the image-animation category.

Input

The input for this action is defined by the following schema:

{
  "faceImage": "string (uri)",
  "drivingVideo": "string (uri)",
  "videoFrameLoadCap": "integer",
  "livePortraitOutputSize": "integer",
  "livePortraitScaleFactor": "number",
  "videoSelectEveryNthFrame": "integer",
  "livePortraitLipZeroEnabled": "boolean",
  "livePortraitStitchingEnabled": "boolean",
  "livePortraitVerticalShiftRatio": "number",
  "livePortraitRelativePositioning": "boolean",
  "livePortraitHorizontalShiftRatio": "number",
  "livePortraitEyeRetargetingEnabled": "boolean",
  "livePortraitLipRetargetingEnabled": "boolean",
  "livePortraitLipRetargetingMultiplier": "number",
  "livePortraitEyesRetargetingMultiplier": "number"
}

Example Input

Here’s an example of a JSON payload you would use to invoke this action:

{
  "faceImage": "https://replicate.delivery/pbxt/L0gy7uyLE5UP0uz12cndDdSOIgw5R3rV5N6G2pbt7kEK9dCr/0_3.webp",
  "drivingVideo": "https://replicate.delivery/pbxt/LEQxLFMUNZMiKt5PWjyMJIbTdvKAb5j3f0spuiEwt9TEbo8B/d0.mp4",
  "videoFrameLoadCap": 128,
  "livePortraitOutputSize": 512,
  "livePortraitScaleFactor": 2.3,
  "videoSelectEveryNthFrame": 1,
  "livePortraitLipZeroEnabled": true,
  "livePortraitStitchingEnabled": true,
  "livePortraitVerticalShiftRatio": -0.12,
  "livePortraitRelativePositioning": true,
  "livePortraitHorizontalShiftRatio": 0,
  "livePortraitEyeRetargetingEnabled": false,
  "livePortraitLipRetargetingEnabled": false,
  "livePortraitLipRetargetingMultiplier": 1,
  "livePortraitEyesRetargetingMultiplier": 1
}

Output

The action typically returns a URL to the generated animated portrait as a response. Here’s an example of the expected output:

[
  "https://assets.cognitiveactions.com/invocations/5458ce79-86cb-41e8-865d-228c5ad86441/f940e92e-443d-4543-a377-6439269f8fd8.mp4"
]

Conceptual Usage Example (Python)

Here’s a conceptual Python code snippet that demonstrates how to call the action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "71b9c75a-c907-4730-ac26-7145f13f599b"  # Action ID for Animate Portrait Using Driving Video

# Construct the input payload based on the action's requirements
payload = {
    "faceImage": "https://replicate.delivery/pbxt/L0gy7uyLE5UP0uz12cndDdSOIgw5R3rV5N6G2pbt7kEK9dCr/0_3.webp",
    "drivingVideo": "https://replicate.delivery/pbxt/LEQxLFMUNZMiKt5PWjyMJIbTdvKAb5j3f0spuiEwt9TEbo8B/d0.mp4",
    "videoFrameLoadCap": 128,
    "livePortraitOutputSize": 512,
    "livePortraitScaleFactor": 2.3,
    "videoSelectEveryNthFrame": 1,
    "livePortraitLipZeroEnabled": true,
    "livePortraitStitchingEnabled": true,
    "livePortraitVerticalShiftRatio": -0.12,
    "livePortraitRelativePositioning": true,
    "livePortraitHorizontalShiftRatio": 0,
    "livePortraitEyeRetargetingEnabled": false,
    "livePortraitLipRetargetingEnabled": false,
    "livePortraitLipRetargetingMultiplier": 1,
    "livePortraitEyesRetargetingMultiplier": 1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this snippet, you need to replace the placeholders with your actual API key and ensure the action ID and payload are structured as required. The endpoint URL and request structure are illustrative, so adapt them according to your API specifications.

Conclusion

The fofr/live-portrait Cognitive Actions offer developers a unique opportunity to create engaging animated portraits that can significantly enhance user experiences. By leveraging the capabilities of the Animate Portrait Using Driving Video action, you can bring static images to life, making your applications more interactive and visually appealing. Consider experimenting with different driving videos and settings to create unique animations tailored to your audience's preferences. Happy coding!

← Back to All Articles