Skip to main content

ThirdLaw Collect

Collect the data needed to identify security, compliance, performance, end-user experience, and reliability issues on a per-interaction basis for LLM-connected systems using various detections and best-practice recommendations.

Key Capabilities​

ThirdLaw Collect gathers and processes conversation data from various LLM systems, creating a standardized format for analysis. It captures prompts, completions, metadata, and contextual information needed for comprehensive evaluation.

  • Collection: Collect from source locations of LLM input/output data.
  • Ingestion and Storage: Reliably store raw, intermediate process data, calculated data and final processed data to both serve as an audit record, activity log record, downstream feed, and application interface for dependent services such as Evaluate, Intervene, and Investigate.
  • Data Serving: Appropriate latency and cost data query and serving support for services including downstream exports (e.g. Splunk), dependent services (e.g. Evaluate) and any other data requests of ThirdLaw.
  • Administration: Administer users, configuration and the behavior of ThirdLaw. This includes managing; all user and role management; data management including indexes and retention policies; ThirdLaw system configuration for ingestion and export; ThirdLaw platform monitoring, ThirdLaw security management including encryption and audit logging; ThirdLaw services management including licensing, and configuration of applications including Evaluate, Intervene, and Investigate.

Built for IT and Security Teams​

Designed with IT and Security teams in mind, ThirdLaw Collect is a data pipeline that provides the critical data needed to ensure compliance, manage risk, and maintain control over the interactions between your applications and LLMs without relying on software engineering support. It is a platform that allows IT and Security teams to act quickly, ensuring that compliance, security, and governance requirements are met.

Why ThirdLaw Collect?​

  • Comprehensive Data Capture: Collects full bodies and headers of calls to LLMs, across all data formats and types.
  • Seamless Integration: Works across gateways, agent environments, SDKs, and OpenTelemetry-instrumented apps.
  • Scalable Data Routing: Route your data to ThirdLaw or your preferred downstream observability stack or data lake for analysis and storage.
  • Enterprise-Ready: Tailored for IT and Security teams to ensure LLM observability and risk management.