Random Variables Intermediate

Random variables are mathematical objects that assign numerical values to random outcomes. They let us formalize uncertainty, compute expected values, measure spread (variance), and quantify relationships between variables (covariance). These concepts appear everywhere in ML - from loss functions to batch statistics.

Expectation (Mean)

The expected value E[X] is the weighted average of all possible outcomes, weighted by their probabilities. In ML, expectations appear in loss functions, Monte Carlo methods, and reinforcement learning:

Python 
import numpy as np

# Expected value from samples (Monte Carlo estimate)
samples = np.random.normal(5, 2, size=10000)
print("E[X] ~=", np.mean(samples))  # ~5.0

# Expected loss over a dataset
# E[L] = (1/N) * sum(L(y_i, pred_i))
losses = np.array([0.5, 0.3, 0.8, 0.2, 0.4])
expected_loss = np.mean(losses)
print("Expected loss:", expected_loss)

Variance and Standard Deviation

Variance Var(X) measures how spread out a distribution is. It equals the expected squared deviation from the mean:

Python 
# Var(X) = E[(X - E[X])^2]
samples = np.random.normal(0, 3, size=10000)
print("Variance:", np.var(samples))   # ~9.0 (sigma^2)
print("Std dev:", np.std(samples))     # ~3.0 (sigma)

# In batch normalization: normalize using batch mean and variance
batch = np.random.randn(32, 64)  # 32 samples, 64 features
batch_mean = batch.mean(axis=0)
batch_var = batch.var(axis=0)
normalized = (batch - batch_mean) / np.sqrt(batch_var + 1e-8)

Covariance and Correlation

Covariance measures how two variables change together. The covariance matrix captures all pairwise relationships and is fundamental to PCA and multivariate Gaussians:

Python 
# Generate correlated data
mean = [0, 0]
cov = [[1, 0.8],
       [0.8, 1]]  # Positive correlation
X = np.random.multivariate_normal(mean, cov, size=1000)

# Compute covariance matrix from data
cov_matrix = np.cov(X.T)
print("Covariance matrix:\n", cov_matrix)

# Correlation: normalized covariance (-1 to 1)
corr_matrix = np.corrcoef(X.T)
print("Correlation:\n", corr_matrix)
Law of Large Numbers: As you collect more data, the sample mean converges to the true expected value. This is why more training data generally leads to better models - our estimates of the true distribution become more accurate.

Next Up: MLE / MAP

Learn how to estimate model parameters from data using Maximum Likelihood and Maximum A Posteriori estimation.

Next: MLE / MAP →
← Bayes Theorem MLE / MAP →

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka
📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp
🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX