Create Engaging Talking Face Animations with cjwbw/sadtalker Cognitive Actions

23 Apr 2025
Create Engaging Talking Face Animations with cjwbw/sadtalker Cognitive Actions

In the world of digital content creation, adding a human touch to animations can significantly enhance user engagement. The cjwbw/sadtalker API offers a powerful Cognitive Action that allows developers to create talking head video animations from a single portrait image and audio input. This feature leverages the SadTalker model to generate realistic facial movements synchronized with audio, giving life to still images in an innovative way.

Prerequisites

Before you start integrating the Cognitive Actions, ensure you have the following:

  • API Key: You will need an API key for authenticating your requests to the Cognitive Actions platform. This key should be passed in the headers of your API calls.
  • Audio and Image Files: Make sure to have the audio file (in .wav or .mp4 format) and the image file (in .png format) ready to use.

Cognitive Actions Overview

Generate Talking Face Animation

Description:
This action generates a talking head video animation from a portrait image and audio input. The animation is driven by the audio, stylizing facial movements accordingly. It also supports various processing methods and enhancement options for an optimal output.

Category: Video Generation

Input

The following fields are required to invoke the action:

  • drivenAudio (string): URI pointing to the uploaded audio file.
  • sourceImage (string): URI pointing to the uploaded image file.

Optional fields include:

  • poseStyle (integer): An index representing the style of pose (default is 0, valid range is 0-45).
  • stillMode (boolean): Enables less head motion when true (default is true).
  • useEnhancer (boolean): Activates the GFPGAN face enhancer (default is false).
  • useEyeblink (boolean): Enables eye blinking in the animation (default is true).
  • expressionScale (number): Controls the strength of facial expressions (default is 1).
  • imageResolution (integer): Sets the resolution of the face model (default is 256, available options: 256, 512).
  • faceRenderMethod (string): Specifies the method for face rendering (default is "facevid2vid", options: "facevid2vid", "pirender").
  • preprocessMethod (string): Determines the image preprocessing technique (default is "crop", options: "crop", "resize", "full", "extcrop", "extfull").

Example Input:

{
  "poseStyle": 0,
  "stillMode": true,
  "drivenAudio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
  "sourceImage": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
  "useEnhancer": true,
  "useEyeblink": true,
  "expressionScale": 1,
  "imageResolution": 256,
  "faceRenderMethod": "facevid2vid",
  "preprocessMethod": "crop"
}

Output

The output of this action will be a URI pointing to the generated video animation.

Example Output:

https://assets.cognitiveactions.com/invocations/07f5cbc8-66ac-4066-b71f-03a7330fd7aa/7050a45b-fca6-4ee0-9b7f-6010ee4b8860.mp4

Conceptual Usage Example (Python)

Here’s how you might structure a request to execute the "Generate Talking Face Animation" action using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "e03c9a70-8a6c-49ec-85e2-42e2eaf2f6c1" # Action ID for Generate Talking Face Animation

# Construct the input payload based on the action's requirements
payload = {
    "poseStyle": 0,
    "stillMode": True,
    "drivenAudio": "https://replicate.delivery/pbxt/IkgWA4bLoXpk5NwVsfOBzHh7MswfNLTgtf44Qr2gdOTOWvSX/japanese.wav",
    "sourceImage": "https://replicate.delivery/pbxt/IkgW9tngATq608Qf6haUXDpg81s5YBJfS9GaBiCFjdKXk4F5/art_1.png",
    "useEnhancer": True,
    "useEyeblink": True,
    "expressionScale": 1,
    "imageResolution": 256,
    "faceRenderMethod": "facevid2vid",
    "preprocessMethod": "crop"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet, you'll need to replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key. The action_id corresponds to the ID of the action you want to execute. The payload is constructed using the required input fields, and the request is sent to the hypothetical Cognitive Actions execution endpoint.

Conclusion

Integrating the cjwbw/sadtalker Cognitive Actions into your applications opens up exciting possibilities for creating dynamic and engaging content. Whether you're developing educational tools, enhancing video content, or creating interactive experiences, the ability to generate talking face animations can significantly enrich your projects. Explore the capabilities of this Cognitive Action today and transform your static images into captivating animations!

← Back to All Articles