Foundational

The Foundational Analytic Engine harnesses the capabilities of LLMs to perform sophisticated analysis, reasoning, and evaluation of conversational content.

Foundational Analytic Engine

• Use Case: Complex analysis and reasoning about content
• Technology: LLMs
• Valid Inputs: Text conversations (prompts and completions)
• Available Evaluators:
  • PromptInjection-Foundational
  • ToxicLanguage-Foundational
• Last Engine Update: 2025-03-03
• Dependencies: Locally hosted foundational model inside the ThirdLaw VPC

Detailed Description

The Foundational Analytic Engine employs LLMs to perform nuanced analysis of conversational content that requires complex reasoning, contextual understanding, and specialized evaluation criteria. This engine excels at tasks that require deep understanding of language, context, and human communication patterns.

How It Works

The Foundational Analytic Engine input text to perform sophisticated analysis that requires human-like reasoning capabilities. When anevaluatorusing this engine is initialized, it connects to a locally hosted foundational model inside the ThirdLaw VPC and loads specialized prompt templates designed for specific analytical tasks. During analysis, the engine prepares the input by integrating the conversation content with the specialized prompt templates. The engine then submits this enhanced prompt to the model.

The LLM performs the requested analysis based on the guidance in the prompt template, applying its broad knowledge and reasoning capabilities to evaluate the content according to the specified criteria. Once the LLM generates its analysis, the engine parses the response into a structured Finding format, extracting key judgments, confidence scores, and justifications from the LLM's output. This approach makes the Foundational Analytic Engine particularly effective for complex evaluations that require nuanced judgment, contextual understanding, and specialized domain knowledge that would be difficult to implement with more traditional pattern matching or data-driven classification or semantic similarity approaches.

Configuration Options

The Foundational Analytic Engine supports the following configuration parameters:

Parameter	Description	Default
api_key	API key for the LLM provider	Required
model	LLM model to use for analysis	llama3-70b-8192
prompt	Path to the prompt template file	Required
temperature	Temperature setting for LLM generation	0.0
tokens	Token replacements for prompt templates

Finding Structure

A genericevaluatorbased on the Foundational Analytic Engine returns a Finding with the following structure.

Finding Structure

{
    "EvaluatorName-Foundational": True/False,
    "EvaluatorName-Foundational.confidence": [0-1],
    "EvaluatorName-Foundational.other_properties": string,
    "EvaluatorName-Foundational.justification": string,
}

Available Evaluators

The following table lists common Evaluators that can be created using the Foundational Analytic Engine:

Evaluator Name	Description	Common Use Cases
ToxicLanguage-Foundational	Identifies harmful or unsafe content across multiple hazard categories	Content moderation, safety compliance

Dependencies

• Locally hosted foundational model: Primary LLM provider integrated inside the ThirdLaw VPC

Revision History

• 2025-03-03: Initial documentation creation