Deep Learning for Sentiment Analysis Intermediate

Transformer models like BERT have set new benchmarks for sentiment analysis, achieving 93%+ accuracy on standard datasets. This lesson covers using pretrained sentiment models, fine-tuning BERT on your own data, and deploying with the Hugging Face pipeline API.

Quick Start: Pretrained Pipeline

The fastest way to get state-of-the-art sentiment analysis is using a pretrained Hugging Face pipeline:

Python

from transformers import pipeline

# Load pretrained sentiment model
sentiment = pipeline("sentiment-analysis")

# Analyze sentiment
results = sentiment([
    "I absolutely love this product!",
    "Terrible quality, waste of money.",
    "It's okay, nothing special.",
])

for r in results:
    print(f"{r['label']}: {r['score']:.3f}")

# Use a more specific model
sentiment_5star = pipeline(
    "sentiment-analysis",
    model="nlptown/bert-base-multilingual-uncased-sentiment"
)
result = sentiment_5star("Pretty good but could be better")
print(result)  # {'label': '3 stars', 'score': 0.45}

Fine-Tuning BERT for Sentiment

Python

from transformers import (
    AutoTokenizer, AutoModelForSequenceClassification,
    Trainer, TrainingArguments
)
from datasets import load_dataset
import numpy as np
from sklearn.metrics import accuracy_score, f1_score

# Load IMDB dataset from Hugging Face
dataset = load_dataset("imdb")

# Load tokenizer and model
model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
    model_name, num_labels=2
)

# Tokenize dataset
def tokenize(examples):
    return tokenizer(
        examples["text"], truncation=True,
        padding="max_length", max_length=256
    )

tokenized = dataset.map(tokenize, batched=True)

# Metrics function
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    return {
        "accuracy": accuracy_score(labels, preds),
        "f1": f1_score(labels, preds, average="weighted"),
    }

# Training configuration
training_args = TrainingArguments(
    output_dir="./sentiment-bert",
    num_train_epochs=3,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    learning_rate=2e-5,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    fp16=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized["train"],
    eval_dataset=tokenized["test"],
    compute_metrics=compute_metrics,
)

trainer.train()

Using DistilBERT for Faster Inference

DistilBERT is 60% faster than BERT with 97% of its accuracy:

Python

# Just change the model name - everything else stays the same
model_name = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
    model_name, num_labels=2
)

Model Comparison

Model	IMDB Accuracy	Speed	Size
TF-IDF + LR	~88%	Very fast	~50 MB
DistilBERT	~92%	Fast	~250 MB
BERT-base	~93%	Moderate	~420 MB
RoBERTa-large	~95%	Slow	~1.3 GB

Choosing a model: For production with latency constraints, use DistilBERT. For maximum accuracy, fine-tune RoBERTa-large. For quick prototyping, use the pretrained pipeline with no fine-tuning. If you cannot use a GPU, stick with the ML-based approach from the previous lesson.

Inference in Production

Python

from transformers import pipeline

# Load your fine-tuned model
sentiment = pipeline(
    "sentiment-analysis",
    model="./sentiment-bert/checkpoint-best",
    device=0  # GPU index, -1 for CPU
)

# Batch inference for efficiency
texts = ["Great product!", "Terrible service.", ...]
results = sentiment(texts, batch_size=32)

for text, result in zip(texts, results):
    print(f"{result['label']} ({result['score']:.3f}): {text}")

Try It Yourself

Use the Hugging Face pipeline to analyze sentiment on 100 product reviews, then fine-tune DistilBERT on a subset. Compare accuracy before and after fine-tuning.

Next: Aspect-Based SA →

← ML-Based Methods Aspect-Based SA →

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka

📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp

🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX