3d-reconstruction

Transform Text and Images into 3D Holograms with DreamGaussian

27 Apr 2025
Transform Text and Images into 3D Holograms with DreamGaussian

In an era where visual content is king, the ability to create engaging and interactive 3D holograms from simple text and images can significantly enhance user experiences across various applications. DreamGaussian offers a powerful Cognitive Action that leverages Generative Gaussian Splatting technology to transform your 2D inputs into stunning 3D content. This service not only simplifies the 3D modeling process but also optimizes the output for holographic displays, making it an ideal solution for developers looking to innovate in fields such as gaming, virtual reality, and advertising.

Imagine being able to take a static image or a piece of text and convert it into a dynamic 3D object. Whether you're designing a virtual character for a game, creating a unique product display for an e-commerce site, or developing immersive educational tools, DreamGaussian provides the flexibility and efficiency needed to bring your ideas to life. By utilizing parameters like elevation angle and image size, developers can fine-tune the output to meet their specific needs, ensuring that the final product is both visually appealing and functionally effective.

Prerequisites

Before diving into the integration of DreamGaussian's Cognitive Actions, ensure you have a valid API key and a basic understanding of making API calls.

Generate 3D Content with DreamGaussian

The Generate 3D Content with DreamGaussian action is specifically designed to create efficient 3D hologram content. This action allows you to input both text and image prompts, which are then processed to generate realistic 3D objects optimized for holographic displays.

Purpose

This action solves the problem of creating 3D content quickly and efficiently from 2D sources. It empowers developers to easily produce interactive visuals that can enhance user engagement and experience.

Input Requirements

To use this action, you'll need to provide the following inputs:

  • Text: A string that serves as a prompt for generating the 3D object.
  • Image: A URI pointing to the input image that will be transformed into a 3D object.
  • Elevation: An integer for the elevation angle of the input image, ranging from -90 to 90 degrees (default is 0).
  • Image Size: An integer that specifies the target size for the preprocessed input image (default is 256).
  • Num Steps: An integer defining the total number of iterations for the process (default is 500).
  • Num Point Samples: An integer specifying the number of points sampled for Gaussian Splatting, ranging from 200 to 10,000 (default is 1000).
  • Num Refinement Steps: An integer indicating the number of refinement iterations (default is 50).

Example Input

{
  "image": "https://replicate.delivery/pbxt/KJ8DW4s2VG9W9dN2MgG16HtbOxccxqdYDtN478M6IuCwYtBO/doraemon-mascot.webp",
  "elevation": 0,
  "imageSize": 256,
  "numberOfSteps": 500,
  "numberOfPointSamples": 1000,
  "numberOfRefinementSteps": 50
}

Expected Output

The expected output will be a 3D model that can be utilized in various applications, although there may be error messages indicating issues during processing, such as command execution failures.

Example Output

Error: Command '['python', 'optimize_raw.py', '--config', 'configs/image.yaml', 'input=/src/image_rgba.png', 'save_path=image', 'ref_size=256', 'elevation=0', 'iters=500', 'num_pts=1000', 'force_cuda_rast=True']' returned non-zero exit status 1.

Use Cases for this Action

  • Gaming: Developers can create unique characters or environments that enhance gameplay.
  • E-commerce: Retailers can showcase products in 3D, allowing customers to see them from all angles before making a purchase.
  • Education: Educators can develop interactive learning tools that provide 3D visualizations of complex concepts.
  • Advertising: Marketers can produce eye-catching ads that include 3D representations of products, attracting more attention.

```python
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "8d453a0e-e62d-4465-9f70-c8b03191d133" # Action ID for: Generate 3D Content with DreamGaussian

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "image": "https://replicate.delivery/pbxt/KJ8DW4s2VG9W9dN2MgG16HtbOxccxqdYDtN478M6IuCwYtBO/doraemon-mascot.webp",
  "elevation": 0,
  "imageSize": 256,
  "numberOfSteps": 500,
  "numberOfPointSamples": 1000,
  "numberOfRefinementSteps": 50
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")



## Conclusion
DreamGaussian's ability to generate 3D holograms from text and images opens up new avenues for innovation in various sectors. By simplifying the 3D content creation process, it allows developers to focus on enhancing user experiences rather than getting bogged down by technical complexities. As you explore the possibilities of integrating DreamGaussian into your applications, consider the numerous ways this technology can elevate your projects—whether it's through immersive experiences in gaming, interactive product displays, or engaging educational tools. Start experimenting with DreamGaussian today and transform your creative ideas into reality!
← Back to All Articles