Audit Logging

Build comprehensive audit trails for ML systems that satisfy regulatory requirements, enable incident investigation, and provide accountability for AI-driven decisions.

What to Log in ML Systems

ML audit logging extends beyond traditional application logging. You need to capture the full lifecycle of data, models, and predictions:

Prediction Auditing

For high-stakes AI decisions (credit, healthcare, hiring), you must be able to explain why a specific prediction was made:

import json, uuid, time

class PredictionAuditLogger:
    def log_prediction(self, request, response, model_info):
        audit_record = {
            "prediction_id": str(uuid.uuid4()),
            "timestamp": time.time(),
            "model_id": model_info["id"],
            "model_version": model_info["version"],
            "model_hash": model_info["sha256"],
            "input_hash": hash_input(request),
            "output": response["prediction"],
            "confidence": response["confidence"],
            "feature_importance": response.get("explanations"),
            "requester": request.authenticated_user,
            "latency_ms": response["latency"],
        }

        # Write to append-only, tamper-evident log
        self.audit_store.append(audit_record)

        return audit_record["prediction_id"]

Data Lineage Tracking

Data lineage traces how data flows from raw sources through transformations to model training and predictions:

• Source tracking: Record where every piece of training data originated and when it was collected.
• Transformation logging: Log every data transformation, cleaning step, and feature engineering operation.
• Version control: Track dataset versions alongside model versions. Know exactly which data trained which model.
• Deletion compliance: When data subjects exercise deletion rights (GDPR), lineage tracking identifies all models trained on that data.

Regulatory Requirements