Create Long-Form Videos from Short Clips with xiankgx/short-to-long-video-diffusion Actions

25 Apr 2025
Create Long-Form Videos from Short Clips with xiankgx/short-to-long-video-diffusion Actions

In the evolving landscape of video content creation, the xiankgx/short-to-long-video-diffusion API provides developers with powerful Cognitive Actions that streamline the generation of long-form videos from short inputs. Utilizing advanced models like SEINE, these actions enable the creation of visually appealing and engaging videos with minimal effort. This post will guide you through the capabilities of the provided actions and how to integrate them into your applications.

Prerequisites

Before diving into the integration of Cognitive Actions, ensure you have the following:

  • An API key for the Cognitive Actions platform, which will be used for authentication when making requests.
  • Familiarity with handling JSON data, as the input and output of the actions will be in this format.

Typically, authentication involves including the API key in the headers of your request. Here’s how you might structure it conceptually:

headers = {
    "Authorization": "Bearer YOUR_COGNITIVE_ACTIONS_API_KEY",
    "Content-Type": "application/json"
}

Cognitive Actions Overview

Generate Video with SEINE Model

The Generate Video with SEINE Model action allows developers to create long-form videos by generating smooth transitions and predictive scenes based on a provided prompt and a short video input. It is categorized under video-generation and leverages the SEINE model's capabilities to produce high-quality video content.

Input

The input for this action requires several fields, both required and optional:

  • inputPath (string, required): A URI pointing to the input image used as a base or reference for generation.
  • prompt (string, required): A textual description of the scene or content to be generated by the AI.
  • seed (integer, optional): A seed number for reproducibility. Use -1 for a random seed.
  • width (integer, optional): Specifies the width in pixels of the generated video (default is 560).
  • height (integer, optional): Specifies the height in pixels of the generated video (default is 240).
  • guidanceLevel (number, optional): The scale of guidance affecting output fidelity (default is 8).
  • numberOfFrames (integer, optional): Total number of frames in the video (default is 16).
  • samplingMethod (string, optional): Method used for sampling ('ddpm' or 'ddim', default is 'ddpm').
  • extraDescription (string, optional): Additional prompts to append to the main prompt (e.g., ', slow motion.').
  • negativeDescription (string, optional): Descriptions of elements to omit from the content.

Here’s an example input JSON structure for this action:

{
  "seed": -1,
  "width": 560,
  "height": 240,
  "prompt": "Close up essence is poured from bottle Kodak Vision",
  "inputPath": "https://replicate.delivery/pbxt/JxvgKVWtUA4htUESCsMrkOmDCjIg6UA6MJIDoAKPVumHatOu/Close-up_essence_is_poured_from_bottleKodak_Vision.png",
  "guidanceLevel": 8,
  "numberOfFrames": 16,
  "samplingMethod": "ddpm",
  "extraDescription": ", slow motion.",
  "negativeDescription": ""
}

Output

Upon successful execution, this action typically returns a URI pointing to the generated video. For example:

https://assets.cognitiveactions.com/invocations/d9705294-9aac-4312-9e9c-a4d8fedb4734/e88d2793-63ba-4371-8c86-40380d98fa76.mp4

Conceptual Usage Example (Python)

Here’s how you might call the Generate Video with SEINE Model action using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute" # Hypothetical endpoint

action_id = "cf9aab20-ded5-43a8-aee5-7378e97232cc" # Action ID for Generate Video with SEINE Model

# Construct the input payload based on the action's requirements
payload = {
    "seed": -1,
    "width": 560,
    "height": 240,
    "prompt": "Close up essence is poured from bottle Kodak Vision",
    "inputPath": "https://replicate.delivery/pbxt/JxvgKVWtUA4htUESCsMrkOmDCjIg6UA6MJIDoAKPVumHatOu/Close-up_essence_is_poured_from_bottleKodak_Vision.png",
    "guidanceLevel": 8,
    "numberOfFrames": 16,
    "samplingMethod": "ddpm",
    "extraDescription": ", slow motion.",
    "negativeDescription": ""
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload} # Hypothetical structure
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this example, the action ID and input payload are structured accurately to align with the required input schema. The endpoint URL and exact request structure are illustrative, showcasing how you can interact with the Cognitive Actions API.

Conclusion

The xiankgx/short-to-long-video-diffusion Cognitive Action for generating videos with the SEINE model opens up exciting possibilities for developers looking to enhance their applications with advanced video content generation. By understanding the input requirements and output results, you can easily integrate this functionality into your projects. Explore potential use cases, such as creating engaging marketing videos or enhancing storytelling through dynamic content. Happy coding!

← Back to All Articles