Secure Deployment

Learn to deploy ML models securely with cryptographic signing, encrypted inference, hardened containers, and proper secrets management.

Model Signing

Model signing ensures that deployed models are exactly the ones that were trained and approved. Any tampering with model weights, architecture, or metadata invalidates the signature.

import hashlib
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding

def sign_model(model_path, private_key):
    """Sign a model file with RSA private key."""
    # Hash the model file
    model_hash = hashlib.sha256(
        open(model_path, 'rb').read()
    ).digest()

    # Sign the hash
    signature = private_key.sign(
        model_hash,
        padding.PSS(
            mgf=padding.MGF1(hashes.SHA256()),
            salt_length=padding.PSS.MAX_LENGTH
        ),
        hashes.SHA256()
    )
    return signature

def verify_model(model_path, signature, public_key):
    """Verify model signature before loading."""
    model_hash = hashlib.sha256(
        open(model_path, 'rb').read()
    ).digest()

    try:
        public_key.verify(signature, model_hash,
            padding.PSS(mgf=padding.MGF1(hashes.SHA256()),
                        salt_length=padding.PSS.MAX_LENGTH),
            hashes.SHA256())
        return True
    except InvalidSignature:
        raise SecurityError("Model signature verification failed!")

Encrypted Inference

Encrypted inference protects both the model (intellectual property) and the input data (user privacy) during prediction:

Container Hardening

ML serving containers require specific hardening beyond standard practices:

• Minimal base images: Use distroless or slim images. Remove shells, package managers, and debugging tools from production containers.
• Read-only root filesystem: Mount the model and code as read-only. Only temporary directories (for caching) should be writable.
• Non-root execution: Run the inference process as a non-root user with minimal capabilities.
• Resource limits: Set CPU, memory, and GPU limits to prevent resource exhaustion attacks.
• Network policies: Restrict container networking to only allow necessary ingress (inference requests) and egress (monitoring, logging).

Secrets Management

ML deployments involve multiple secrets: API keys, database credentials, model signing keys, and encryption keys. Never embed these in containers or configuration files.