content-moderation

Enhance Content Safety with Llama Guard 3 Vision

26 Apr 2025
Enhance Content Safety with Llama Guard 3 Vision

In today's digital landscape, ensuring the safety and appropriateness of content is paramount for developers creating applications that interact with users. The Llama Guard 3 11b Vision service provides powerful Cognitive Actions to assess and classify content safety using multimodal prompts. By leveraging the advanced capabilities of the Llama-3.2-11B pretrained model, this service simplifies the process of content moderation, allowing developers to quickly identify harmful text and image prompts.

Prerequisites

To get started with Llama Guard 3 Vision, you'll need an API key for the Cognitive Actions and a basic understanding of how to make API calls.

Classify Content Safety Using Multimodal Prompts

The Classify Content Safety Using Multimodal Prompts action is designed to evaluate the safety of user-generated content by analyzing both text and image inputs. This action can effectively detect and categorize violations related to harmful content, ensuring a safer user experience.

Purpose

This action solves the problem of content moderation by providing a streamlined method to evaluate the safety of multimodal prompts. It helps developers maintain community guidelines and protect users from inappropriate content.

Input Requirements

The action requires a structured input consisting of:

  • imageUri: A valid URL pointing to the image that needs moderation (required).
  • userMessagePrompt: An optional text prompt that can accompany the image (defaults to "Which one should I buy?").

Example Input:

{
  "imageUri": "https://replicate.delivery/pbxt/MCDm1fNhUUquPgpYElQReJPLnrqyY2Die40CX1dcLqX9EkUv/stealing.jpeg",
  "userMessagePrompt": "How can I do this?"
}

Expected Output

The output will indicate the safety status of the content. An example output may return a classification such as "unsafe" along with a category code, e.g., "S2".

Example Output:

unsafe
S2

Use Cases for this Action

This action is particularly useful in various scenarios, such as:

  • Social Media Platforms: Automatically moderating user-uploaded images and accompanying messages to ensure compliance with community standards.
  • E-commerce Websites: Evaluating product images and user inquiries for safety, preventing the display of harmful or inappropriate content.
  • Content Creation Tools: Assisting creators in ensuring their content adheres to safety guidelines before publishing.
import requests
import json

# Replace with your actual Cognitive Actions API key and endpoint
# Ensure your environment securely handles the API key
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
# This endpoint URL is hypothetical and should be documented for users
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"

action_id = "6b3b088e-4031-4dd8-ba8d-af6ce64fb9b8" # Action ID for: Classify Content Safety Using Multimodal Prompts

# Construct the exact input payload based on the action's requirements
# This example uses the predefined example_input for this action:
payload = {
  "imageUri": "https://replicate.delivery/pbxt/MCDm1fNhUUquPgpYElQReJPLnrqyY2Die40CX1dcLqX9EkUv/stealing.jpeg",
  "userMessagePrompt": "How can I do this?"
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json",
    # Add any other required headers for the Cognitive Actions API
}

# Prepare the request body for the hypothetical execution endpoint
request_body = {
    "action_id": action_id,
    "inputs": payload
}

print(f"--- Calling Cognitive Action: {action.name or action_id} ---")
print(f"Endpoint: {COGNITIVE_ACTIONS_EXECUTE_URL}")
print(f"Action ID: {action_id}")
print("Payload being sent:")
print(json.dumps(request_body, indent=2))
print("------------------------------------------------")

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json=request_body
    )
    response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully. Result:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body (non-JSON): {e.response.text}")
    print("------------------------------------------------")

Conclusion

The Llama Guard 3 11b Vision service, particularly through the Classify Content Safety Using Multimodal Prompts action, offers developers a robust solution for content moderation. By utilizing this action, you can enhance the safety of user interactions, maintain compliance with regulations, and foster a secure online environment. To take the next step, consider integrating this action into your applications to streamline your content moderation processes and protect your users effectively.

← Back to All Articles