video-generation

Create Engaging Videos from Text Prompts with Controlvideo

26 Apr 2025
Create Engaging Videos from Text Prompts with Controlvideo

In the world of video content creation, the ability to swiftly transform text prompts into engaging videos can significantly enhance productivity and creativity. Controlvideo offers a powerful Cognitive Action that allows developers to generate videos from text descriptions without the need for extensive training. By leveraging the ControlVideo framework and ControlNet, this action produces high-quality, consistent videos that can be tailored to various specifications. Whether you’re looking to create animations, promotional content, or educational materials, this service simplifies the video generation process and opens up a realm of possibilities for developers.

Use Cases

Imagine a scenario where a marketing team needs to produce a series of promotional videos quickly. With Controlvideo, they can input descriptive text prompts, specify conditions for video generation, and receive high-quality videos in a matter of minutes. Similarly, educators can create visually appealing instructional videos by simply providing text prompts that outline the subject matter. The versatility of this action makes it ideal for content creators, advertisers, and educators alike, enabling them to bring their ideas to life without the overhead of traditional video production methods.

Prerequisites

To get started with Controlvideo, you'll need a Cognitive Actions API key and a basic understanding of API calls to integrate this functionality seamlessly into your applications.

Generate Controllable Video from Text

This operation allows for the generation of videos from text prompts without the need for training. Utilizing the ControlVideo framework, it leverages ControlNet to produce high-quality and consistent videos. The operation supports various structuring methods such as depth, canny, and pose, along with customizable video lengths and other synthesis parameters.

Input Requirements

The input for this action requires a valid URI of the source video, along with several optional parameters that can refine the video generation process:

  • prompt: A descriptive text prompt guiding the transformation or elements of the target video.
  • condition: Specifies the method for structuring the sequence, with options such as 'depth', 'canny', or 'pose'.
  • videoPath: The URI of the source video to be transformed.
  • isLongVideo: A boolean indicating whether to use a hierarchical sampler for longer videos.
  • videoLength: The desired length of the synthesized video in seconds.
  • guidanceScale: A numeric scale for classifier-free guidance.
  • smootherSteps: A list of timestep indices for applying the interleaved-frame smoother.
  • numInferenceSteps: The total number of denoising inference steps for video quality.

Expected Output

The output is a URI link to the generated video, ready for viewing or further use in your projects.

Use Cases for this specific action

  • Marketing Campaigns: Quickly generate promotional videos based on text descriptions, allowing for rapid content iteration and testing.
  • Educational Tools: Create instructional videos that visually explain concepts, enhancing the learning experience for students.
  • Social Media Content: Develop eye-catching videos for social media platforms using creative prompts, increasing engagement and reach.
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "b881df9a-cddc-42a4-9fba-a0df672c3782" # Action ID for: Generate Controllable Video from Text

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "prompt": "James bond moonwalk on the beach, animation style.",
  "condition": "pose",
  "videoPath": "https://replicate.delivery/pbxt/Itp2gAJAkjTrIpLElNLmXjCtwLv5LnQTR2qWE15Qav0eyktR/moonwalk.mp4",
  "isLongVideo": true,
  "videoLength": 15,
  "guidanceScale": 12.5,
  "smootherSteps": "19, 20",
  "numInferenceSteps": 50
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")

Conclusion

The Controlvideo service provides a unique opportunity for developers to harness the power of AI-driven video generation from text prompts. Its flexibility and speed empower content creators across various industries to produce high-quality videos without the complexities of traditional video production. By integrating this Cognitive Action, you can enhance your applications and deliver engaging content that resonates with your audience. Explore the possibilities with Controlvideo and transform your creative ideas into captivating visual stories.

← Back to All Articles