Classifier Ensembles for Production Content Moderation

A single content classifier has a characteristic failure profile. The OpenAI Moderation API is fast but misses encoding-based obfuscation. Llama Guard is strong on standard categories but has high false positive rates on educational content. Perspective API is excellent on toxicity but not designed for instruction-following safety.

Each classifier covers the space of harmful content differently. An ensemble that combines multiple classifiers can cover more of the space while managing the false positive burden intelligently.

This is the practical architecture for production deployments where neither coverage gaps nor false positive rates are acceptable.

Why ensembles work

The key insight from ensemble learning: classifiers that make errors on different inputs can be combined to produce a system with lower overall error than any individual classifier.

This applies to content moderation because different classifiers:

• Are trained on different data distributions
• Use different feature representations (some operate on token sequences, some on embeddings, some on natural language)
• Have different coverage on different content types and languages

The OpenAI Moderation API might miss a base64-encoded harmful request that Llama Guard catches. Llama Guard might flag an educational medical passage that OpenAI Moderation correctly allows. The ensemble sees both signals.

Ensemble architectures for content moderation

Parallel evaluation, OR aggregation (high recall): Flag if any classifier flags. Maximizes recall (catches the most harmful content) at the cost of higher false positive rates. Appropriate for the highest-severity categories where false negatives are very costly.

Parallel evaluation, AND aggregation (high precision): Flag only if all classifiers agree. Minimizes false positives at the cost of lower recall. Appropriate for categories where false positives are costly (medical content, legal discussion, education).

Parallel evaluation, weighted score (tunable): Combine classifier scores with learned weights. A logistic regression on top of the classifier scores, trained on a labeled set of your production traffic, gives you a tunable operating point.

class ContentModerationEnsemble:
    def __init__(self, classifiers, weights=None):
        self.classifiers = classifiers
        self.weights = weights or [1.0] * len(classifiers)
    
    def classify(self, text: str) -> dict:
        scores = []
        for classifier, weight in zip(self.classifiers, self.weights):
            result = classifier.score(text)
            scores.append({
                'score': result.score,
                'categories': result.categories,
                'weight': weight,
                'classifier': classifier.name
            })
        
        # Weighted average score per category
        weighted_score = self._aggregate(scores)
        
        return {
            'flagged': weighted_score > self.threshold,
            'score': weighted_score,
            'component_scores': scores,
            'explanation': self._explain(scores)
        }
    
    def _aggregate(self, scores):
        total_weight = sum(s['weight'] for s in scores)
        return sum(s['score'] * s['weight'] for s in scores) / total_weight

Sequential evaluation with early exit (low latency): Run a fast, lightweight classifier first. If it scores clearly safe or clearly unsafe, return immediately. Only for ambiguous inputs, run the slower, more thorough classifier.

This is the right architecture for latency-sensitive applications. The fast classifier (OpenAI Moderation API, ~20ms) handles the clear cases. The slow classifier (Llama Guard, ~150ms) only runs on borderline inputs that represent a fraction of total volume.

Implementation: sequential with early exit

class SequentialEnsemble:
    def __init__(self, fast_classifier, slow_classifier, 
                 safe_threshold=0.1, unsafe_threshold=0.8):
        self.fast = fast_classifier
        self.slow = slow_classifier
        self.safe_threshold = safe_threshold
        self.unsafe_threshold = unsafe_threshold
    
    async def classify(self, text: str) -> dict:
        # Phase 1: fast classifier
        fast_result = await self.fast.classify_async(text)
        
        # Clear cases: exit early
        if fast_result.score < self.safe_threshold:
            return {'flagged': False, 'fast_only': True, 'score': fast_result.score}
        if fast_result.score > self.unsafe_threshold:
            return {'flagged': True, 'fast_only': True, 'score': fast_result.score}
        
        # Borderline: run slow classifier
        slow_result = await self.slow.classify_async(text)
        
        # Combine with weighted average
        combined = (fast_result.score * 0.4) + (slow_result.score * 0.6)
        
        return {
            'flagged': combined > 0.5,
            'fast_only': False,
            'score': combined,
            'fast_score': fast_result.score,
            'slow_score': slow_result.score
        }

In production, this architecture gives you:

• Fast path (~20ms) for clear safe and clearly unsafe inputs (~70-80% of traffic)
• Slow path (~150ms) for ambiguous inputs (~20-30% of traffic)
• Average latency ~40-60ms rather than 150ms for every request

Operating an ensemble

Monitor per-classifier performance. If one classifier in the ensemble starts degrading — a provider API changes behavior, a deployed model updates — you want to detect this before it affects ensemble output. Track each classifier’s disagree rate with the ensemble as a whole.

Track disagreement patterns. When classifiers disagree, save a sample to a review queue. The disagreements are the most informative cases for understanding where each classifier’s coverage gaps are.

Calibrate weights on production data. The initial weights are a guess. After 30 days of production data, you have labeled disagreements to train the weighting function. Calibrate quarterly.

Keep the ensemble auditable. Regulators and users sometimes need explanations for moderation decisions. The component_scores field in the output above is essential: “Flagged by Llama Guard (score: 0.87) due to violence category” is more defensible than “flagged by the moderation system.”

The comparative performance data for different classifier combinations is published at bestaisecuritytools.com ↗.