Transform Your Videos with Motion Transfer Using lucataco/diffusion-motion-transfer Actions

22 Apr 2025
Transform Your Videos with Motion Transfer Using lucataco/diffusion-motion-transfer Actions

In the realm of video processing, the ability to transfer motion across objects can open up a world of creative possibilities. The lucataco/diffusion-motion-transfer API offers developers a powerful toolset to synthesize videos by utilizing space-time diffusion features. With the Cognitive Actions provided in this spec, you can retain the motion and scene layout of an input video while transforming it based on a target text prompt. This pre-built functionality not only accelerates video editing processes but also enhances creativity by allowing for zero-shot text-driven motion transfer.

Prerequisites

Before diving into the integration of these Cognitive Actions, ensure you have the following:

  • An API key for the Cognitive Actions platform, which will be used for authentication.
  • A basic understanding of how to make HTTP requests, particularly POST requests that include JSON payloads.

For authentication, you will typically include your API key in the request headers. Here’s a conceptual overview of how this might look:

headers = {
    "Authorization": f"Bearer YOUR_COGNITIVE_ACTIONS_API_KEY",
    "Content-Type": "application/json"
}

Cognitive Actions Overview

Transfer Motion Across Video Objects

The Transfer Motion Across Video Objects action enables you to synthesize videos by utilizing space-time diffusion features. It allows you to maintain the original video’s motion and scene layout while adhering to a specified target prompt that describes the desired transformation.

Input

The following fields are required to invoke this action:

  • video: A URI pointing to the input video file. (Required)
  • videoPrompt: A prompt describing the input video, guiding the enhancement process. (Optional, defaults to "Amazing quality, masterpiece, A locomotive rides in a forest")
  • targetPrompt: A prompt detailing the desired output video's theme. (Optional, defaults to "Amazing quality, masterpiece, A motorbike driving in a forest")
  • negativePrompt: A prompt listing characteristics to avoid during processing. (Optional, defaults to "bad quality, distortions, unrealistic, distorted image, watermark, signature")
  • numberOfTimesteps: The number of preprocessing steps applied to the video. (Optional, defaults to 999)

Example Input:

{
  "video": "https://replicate.delivery/pbxt/K7P4ZdV95HbR4aEof8ihr2j7UVWeFxHfoMWTG7H16gCy62Jf/demo.mp4",
  "videoPrompt": "Amazing quality, masterpiece, A locomotive rides in a forest",
  "targetPrompt": "Amazing quality, masterpiece, A motorbike driving in a forest",
  "negativePrompt": "bad quality, distortions, unrealistic, distorted image, watermark, signature",
  "numberOfTimesteps": 900
}

Output

Upon successful execution, the action returns a URI pointing to the synthesized output video.

Example Output:

https://assets.cognitiveactions.com/invocations/76a3905d-ea4f-416b-a3a1-2ea339220b25/b9636d5c-6809-4b4e-8b0c-282b5aa83c2f.mp4

Conceptual Usage Example (Python)

Here is a conceptual Python code snippet illustrating how you might call this action:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "8d33cd90-bb61-4a7a-bd01-af63af7f0e9a"  # Action ID for Transfer Motion Across Video Objects

# Construct the input payload based on the action's requirements
payload = {
    "video": "https://replicate.delivery/pbxt/K7P4ZdV95HbR4aEof8ihr2j7UVWeFxHfoMWTG7H16gCy62Jf/demo.mp4",
    "videoPrompt": "Amazing quality, masterpiece, A locomotive rides in a forest",
    "targetPrompt": "Amazing quality, masterpiece, A motorbike driving in a forest",
    "negativePrompt": "bad quality, distortions, unrealistic, distorted image, watermark, signature",
    "numberOfTimesteps": 900
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code, replace YOUR_COGNITIVE_ACTIONS_API_KEY with your actual API key. Notice how the input payload is structured according to the action's requirements, and the action ID is specified for the correct execution.

Conclusion

The lucataco/diffusion-motion-transfer Cognitive Actions provide developers with an innovative way to transform video content creatively. By leveraging the ability to transfer motion across video objects based on text prompts, you can enhance your video applications in unique ways. Start experimenting with these actions and explore the potential of video synthesis in your projects!

← Back to All Articles