Enhance Your Application with Text Prediction Using Airoboros Llama 2 Actions

22 Apr 2025
Enhance Your Application with Text Prediction Using Airoboros Llama 2 Actions

Integrating advanced text processing capabilities into applications has never been easier with the uwulewd/airoboros-llama-2-70b Cognitive Actions. This powerful set of actions allows developers to leverage the Airoboros L2 70B 2.1 - GPTQ model, optimized for high accuracy text predictions through Low-Rank Adaptation (LoRA). By utilizing these pre-built actions, you can efficiently generate text based on prompts, control output randomness, and customize token generation, greatly enhancing user interactions in your applications.

Prerequisites

Before diving into the implementation of the Cognitive Actions, ensure you have the following:

  • An API key for accessing the Cognitive Actions platform.
  • Basic understanding of JSON and API calls.
  • Familiarity with Python for crafting requests.

Authentication typically involves passing the API key in the request headers. This ensures secure access to the Cognitive Actions you wish to utilize.

Cognitive Actions Overview

Perform Text Prediction Using Airoboros

Description:
This action allows for inference using the Airoboros L2 70B 2.1 - GPTQ model with ExLlama, enabling highly accurate and efficient text predictions. Users can control the randomness and output length through adjustable parameters.

Category: text-processing

Input

The input schema for this action is structured as follows:

{
  "seed": -1,
  "prompt": "USER: Hello, please code a python function that calculates cosine similarity between 2 vectors, remember to do error checks.\nASSISTANT:",
  "useLora": false,
  "maxTokens": 1024,
  "minTokens": 1,
  "temperature": 0.5,
  "topCandidates": 20,
  "repetitionPenalty": 1,
  "cumulativeProbabilityThreshold": 1
}

Fields:

  • seed (integer): A value for reproducibility of results. Use -1 for a random seed. Range: -2147483648 to 2147483647.
  • prompt (string): The input text that guides the model's response. Be clear and distinct in roles like USER and ASSISTANT.
  • useLora (boolean): Use Low-Rank Adaptation (LoRA) for predictions. Defaults to false.
  • maxTokens (integer): Maximum number of tokens in the output. Valid range: 1 to 4096. Default is 1024.
  • minTokens (integer): Minimum number of tokens to generate. Ranges from 0 to 4096. Default is 1.
  • temperature (number): Controls randomness. Values closer to 0.01 make outputs more deterministic, while values closer to 2 make them more random. Default is 0.5.
  • topCandidates (integer): Number of top candidate predictions to consider. Valid values: 1 to 100. Default is 20.
  • repetitionPenalty (number): Penalty for repeated tokens in the output. Range: 1 to 1.5. Default is 1.
  • cumulativeProbabilityThreshold (number): Filters candidate tokens using nucleus sampling. Range: 0.01 to 1. Default is 1.

Output

The output typically consists of a list of tokenized strings representing the generated text. For example:

[
  " Here",
  "'",
  "s",
  " a",
  " Python",
  " function",
  " that",
  " calcul",
  "ates",
  " the",
  " cos",
  "ine",
  " similarity",
  " between",
  " two",
  " vectors",
  ":",
  "...",
]

This output can be processed further to reconstruct the generated text as needed.

Conceptual Usage Example (Python)

Here’s a conceptual example of how you might call the Cognitive Actions execution endpoint using Python:

import requests
import json

# Replace with your Cognitive Actions API key and endpoint
COGNITIVE_ACTIONS_API_KEY = "YOUR_COGNITIVE_ACTIONS_API_KEY"
COGNITIVE_ACTIONS_EXECUTE_URL = "https://api.cognitiveactions.com/actions/execute"  # Hypothetical endpoint

action_id = "af952d51-c820-4859-95ff-5ed1fe63620e"  # Action ID for Perform Text Prediction Using Airoboros

# Construct the input payload based on the action's requirements
payload = {
    "seed": -1,
    "prompt": "USER: Hello, please code a python function that calculates cosine similarity between 2 vectors, remember to do error checks.\nASSISTANT:",
    "useLora": False,
    "maxTokens": 1024,
    "minTokens": 1,
    "temperature": 0.5,
    "topCandidates": 20,
    "repetitionPenalty": 1,
    "cumulativeProbabilityThreshold": 1
}

headers = {
    "Authorization": f"Bearer {COGNITIVE_ACTIONS_API_KEY}",
    "Content-Type": "application/json"
}

try:
    response = requests.post(
        COGNITIVE_ACTIONS_EXECUTE_URL,
        headers=headers,
        json={"action_id": action_id, "inputs": payload}  # Hypothetical structure
    )
    response.raise_for_status()  # Raise an exception for bad status codes (4xx or 5xx)

    result = response.json()
    print("Action executed successfully:")
    print(json.dumps(result, indent=2))

except requests.exceptions.RequestException as e:
    print(f"Error executing action {action_id}: {e}")
    if e.response is not None:
        print(f"Response status: {e.response.status_code}")
        try:
            print(f"Response body: {e.response.json()}")
        except json.JSONDecodeError:
            print(f"Response body: {e.response.text}")

In this code snippet, replace the API key and endpoint with your details. The action ID and input payload are structured to match the requirements specified. The endpoint URL and request structure are illustrative and should be adapted based on the actual API documentation.

Conclusion

By leveraging the uwulewd/airoboros-llama-2-70b Cognitive Actions, developers can easily integrate powerful text prediction capabilities into their applications. The ability to customize parameters such as randomness and output length allows for nuanced and tailored user interactions. Consider exploring additional use cases or integrating other actions to further enhance your application's capabilities. Happy coding!

← Back to All Articles