Create Lifelike Animations with the Live Portrait Cognitive Actions

23 Apr 2025
Create Lifelike Animations with the Live Portrait Cognitive Actions

The Live Portrait Cognitive Actions from the lightsketch-ai/live-portrait spec allow developers to breathe life into static images by animating them using a driving video. This innovative technology enhances user experiences by creating dynamic and engaging visual content. By utilizing these pre-built actions, developers can easily integrate advanced image animation capabilities into their applications without delving deep into complex algorithms.

Prerequisites

To get started with the Live Portrait Cognitive Actions, you'll need the following:

  • An API key for the Cognitive Actions platform to authenticate your requests.
  • Basic knowledge of how to make HTTP requests and handle JSON data.

Authentication typically involves passing your API key in the headers of your requests to ensure secure access to the Action's capabilities.

Cognitive Actions Overview

Animate Face with Driving Video

The Animate Face with Driving Video action generates a live portrait by animating a static face image using a driving video. This action utilizes techniques like lip position zeroing and frame stitching to enhance the coherence and realism of the animation.

Input

The input for this action is structured as follows:

{
  "faceImage": "https://replicate.delivery/pbxt/L0gy7uyLE5UP0uz12cndDdSOIgw5R3rV5N6G2pbt7kEK9dCr/0_3.webp",
  "drivingVideo": "https://replicate.delivery/pbxt/LEQxLFMUNZMiKt5PWjyMJIbTdvKAb5j3f0spuiEwt9TEbo8B/d0.mp4",
  "enableLipZero": true,
  "enableStitching": true,
  "maxFramesToLoad": 128,
  "outputImageSize": 512,
  "faceScalingFactor": 2.3,
  "verticalShiftRatio": -0.12,
  "selectEveryNthFrame": 1,
  "enableEyeRetargeting": false,
  "enableLipRetargeting": false,
  "horizontalShiftRatio": 0,
  "useRelativePositioning": true,
  "eyeRetargetingMultiplier": 1,
  "lipRetargetingMultiplier": 1
}

Required Fields:

  • faceImage: The URI of an image containing a human face, which serves as the base for animation.
  • drivingVideo: The URI of the video that drives the facial animation.

Optional Fields:

  • enableLipZero: Enables lip position zeroing for reducing lip motion (default: true).
  • enableStitching: Enables stitching of frames to enhance animation coherence (default: true).
  • maxFramesToLoad: The maximum number of frames to load from the video (default: 128).
  • outputImageSize: Dimensions of the output image in pixels (default: 512).
  • faceScalingFactor: Scaling factor for the face during animation (default: 2.3).
  • verticalShiftRatio and horizontalShiftRatio: Ratios for shifting the animated face vertically and horizontally (default: -0.12 and 0, respectively).
  • selectEveryNthFrame: Determines the frame selection from the driving video (default: 1).
  • enableEyeRetargeting: Enables eye retargeting to improve realism (default: false).
  • enableLipRetargeting: Enables lip retargeting for enhanced lip movement accuracy (default: false).
  • useRelativePositioning: Positions the face relative to initial frame position (default: true).
  • eyeRetargetingMultiplier and lipRetargetingMultiplier: Multipliers to adjust eye and lip retargeting effects (default: 1).

Output

The action typically returns a URL pointing to the generated animated portrait video:

[
  "https://assets.cognitiveactions.com/invocations/16ee742c-66ed-4cf0-bdea-dea4f4bfd84c/7a64197e-5bdf-4f95-9f5e-102bae96cf2e.mp4"
]

Conceptual Usage Example (Python)

Here’s a conceptual example of how you might call the Animate Face with Driving Video action using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "c7b6cf27-c9c5-42d0-bf16-ce5eb7f73e10" # Action ID for Animate Face with Driving Video

# Construct the input payload based on the action's requirements
payload = {
    "faceImage": "https://replicate.delivery/pbxt/L0gy7uyLE5UP0uz12cndDdSOIgw5R3rV5N6G2pbt7kEK9dCr/0_3.webp",
    "drivingVideo": "https://replicate.delivery/pbxt/LEQxLFMUNZMiKt5PWjyMJIbTdvKAb5j3f0spuiEwt9TEbo8B/d0.mp4",
    "enableLipZero": True,
    "enableStitching": True,
    "maxFramesToLoad": 128,
    "outputImageSize": 512,
    "faceScalingFactor": 2.3,
    "verticalShiftRatio": -0.12,
    "selectEveryNthFrame": 1,
    "enableEyeRetargeting": False,
    "enableLipRetargeting": False,
    "horizontalShiftRatio": 0,
    "useRelativePositioning": True,
    "eyeRetargetingMultiplier": 1,
    "lipRetargetingMultiplier": 1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet:

  • You replace the API key and endpoint with actual values.
  • The action ID and the input payload are structured according to the requirements of the Animate Face with Driving Video action.
  • The result will include a URL for the generated animated video.

Conclusion

The Live Portrait Cognitive Actions offer an exciting way to animate static images, creating a more engaging user experience. By leveraging the Animate Face with Driving Video action, developers can easily integrate sophisticated image animation functionalities into their applications. Explore these capabilities further and consider how you can enhance your own projects with dynamic visual content!

← Back to All Articles