ZENETICS Testing SDK

The ZENETICS SDK is a powerful Python library that provides seamless integration between your LLM applications and the ZENETICS Platform for Quality Management. With a simple command-line interface and flexible configuration options, you can run comprehensive test suites on your local development machine or integrate them into your CI/CD pipeline.

Why Use the ZENETICS SDK?

The ZENETICS SDK bridges the gap between your LLM application and rigorous quality testing, enabling you to:

• Run tests locally during development with immediate feedback
• Integrate seamlessly into CI/CD pipelines for automated testing
• Monitor quality metrics across different model configurations
• Track performance with detailed token usage and metadata
• Support RAG applications with retrieval context evaluation

Getting ready for testing requires only 3 simple steps, which are documented in our comprehensive Testing Quickstart Guide:

1. Install the ZENETICS SDK
2. Configure your ZENETICS API key
3. Implement a generate function to connect your application

Installation

Install the ZENETICS SDK using your preferred Python package manager:

Using pip

pip install zenetics

Using Poetry

poetry add zenetics

Using pipenv

pipenv install zenetics

The ZENETICS SDK is available on PyPI (opens in a new tab) and supports Python 3.8+.

Configuration & Setup

API Key Configuration

The SDK offers flexible configuration options for your ZENETICS API key:

Option 1: .env File (Recommended)

ZENETICS_API_KEY=your-api-key-here

Option 2: Environment Variable

export ZENETICS_API_KEY=your-api-key-here

Option 3: CI/CD Environment

env:
    ZENETICS_API_KEY: ${{ secrets.ZENETICS_API_KEY }}

Verification

Verify your setup with the built-in configuration tools:

# Check configuration
zenetics config show
 
# Test connectivity
zenetics check --verbose

API Access Management

Access your ZENETICS Testing API keys through the ZENETICS Portal (opens in a new tab):

1. Navigate to your application settings
2. Find "API Key - Test Runners" section
3. Create and manage application-specific API keys
4. Control access across your portfolio of applications

The Generate Function

The heart of ZENETICS testing is the generate function - a simple interface that connects your application to the testing framework.

Basic Structure

# Example: ./tests/zenetics/generate.py
 
from zenetics.models.generation import Generation, TokenUsage, ModelMetadata
 
def generate(input: str) -> Generation:
    """
    Connect your LLM application to ZENETICS testing.
 
    Args:
        input: Test case input from your ZENETICS test suite
 
    Returns:
        Generation object with results and optional metadata
    """
 
    # Call your application to generate results
    output, retrieval_context = YOUR_APPLICATION.generate(input)
 
    # Optional: Capture performance metrics
    token_usage = TokenUsage(
        input_tokens=864,
        completion_tokens=567,
        total_tokens=1431
    )
 
    # Optional: Include model configuration
    metadata = ModelMetadata(
        provider="OpenAI",
        model="gpt-3.5-turbo",
        parameters={
            "max_tokens": 1024,
            "temperature": 0.7
        }
    )
 
    return Generation(
        output=output,                        # Required: Your application's response
        retrieval_context=retrieval_context,  # For RAG applications
        token_usage=token_usage,              # Performance tracking
        metadata=metadata                     # Model configuration
    )

Generation Object Components

• output (required): The response generated by your LLM application
• retrieval_context (optional): Retrieved documents for RAG evaluation
• token_usage (optional): Token consumption metrics for cost tracking
• metadata (optional): Model configuration for result correlation

Running Tests

Execute your test suites with the intuitive command-line interface:

Basic Usage

zenetics run --test-suite-id 123 --generator-file ./tests/generate.py

Short Form

zenetics run -t 123 -g ./tests/generate.py

Advanced Options

zenetics run -t 123 -g ./tests/generate.py \
  --output-dir ./test-results \
  --verbose \
  --local-only

Command Options

CLI Commands Overview

Configuration Management

# View current configuration
zenetics config show
 
# Get help
zenetics config --help

Test Suite Management

# List available test suites
zenetics testsuite list
 
# Get help
zenetics testsuite --help

System Verification

# Check connectivity
zenetics check
 
# Verbose connectivity check
zenetics check --verbose

Version Information

zenetics version

Development Workflow

Local Development

# 1. Verify configuration
zenetics config show
 
# 2. Check connectivity
zenetics check --verbose
 
# 3. List available test suites
zenetics testsuite list
 
# 4. Run tests with verbose output
zenetics run -t 123 -g ./tests/generate.py --verbose
 
# 5. Local-only testing (no API calls)
zenetics run -t 123 -g ./tests/generate.py --local-only

CI/CD Integration

GitHub Actions Example

name: ZENETICS Quality Tests
 
on: [push, pull_request]
 
jobs:
    test:
        runs-on: ubuntu-latest
        steps:
            - uses: actions/checkout@v3
 
            - name: Set up Python
              uses: actions/setup-python@v4
              with:
                  python-version: '3.9'
 
            - name: Install dependencies
              run: |
                  pip install zenetics
                  pip install -r requirements.txt
 
            - name: Verify ZENETICS setup
              env:
                  ZENETICS_API_KEY: ${{ secrets.ZENETICS_API_KEY }}
              run: |
                  zenetics config show
                  zenetics check --verbose
 
            - name: Run quality tests
              env:
                  ZENETICS_API_KEY: ${{ secrets.ZENETICS_API_KEY }}
              run: |
                  zenetics run -t 123 -g ./tests/generate.py --verbose

Environment Support

The ZENETICS SDK supports multiple environments for different stages of your development lifecycle:

• Local: http://localhost:8080 (for local ZENETICS instances)
• Development: https://dev.api.zenetics.io
• Staging: https://staging.api.zenetics.io
• Production: https://api.zenetics.io (default)

Configure your environment using the ENV variable:

ENV=dev
ZENETICS_API_KEY=your-dev-api-key

Best Practices

Generator Function Design

• Keep your generate() function focused and simple
• Handle errors gracefully within the function
• Include meaningful metadata when available
• Use type hints for better code clarity

Configuration Management

• Use .env files for project-specific configurations
• Never commit API keys to version control
• Add .env to your .gitignore file
• Use different configurations for different environments

Testing Strategy

• Group related test cases into logical test suites
• Use descriptive names for your generator files
• Organize generator files in a dedicated directory structure
• Run tests locally before pushing to CI/CD

Performance Optimization

• Monitor token usage to control costs
• Use --local-only for development and debugging
• Organize test suites for efficient execution

Troubleshooting

The SDK includes comprehensive troubleshooting tools:

# Check complete configuration
zenetics config show
 
# Verify API connectivity
zenetics check --verbose
 
# List available test suites
zenetics testsuite list

Common issues and solutions are covered in our Troubleshooting Guide.

Next Steps

Ready to get started? Follow our step-by-step Testing Quickstart Guide to:

1. Set up your ZENETICS account
2. Install and configure the SDK
3. Create your first generate function
4. Run your first quality test

For advanced usage, explore our API Reference and Integration Examples.

Need help? Contact our support team at contact@zenetics.io or visit our Documentation Portal (opens in a new tab) for comprehensive guides and examples.