Intermediate

AV Simulation

Test autonomous driving algorithms in photorealistic virtual environments with realistic traffic, weather, and sensor simulation.

Why Simulation is Essential

Autonomous vehicles cannot be safely tested at scale on public roads. Simulation enables millions of miles of virtual testing, including rare and dangerous scenarios that would be impossible to reproduce safely in the real world. Companies like Waymo simulate billions of miles annually.

AV Simulators

🏙

CARLA

Open-source simulator built on Unreal Engine. Photorealistic rendering, diverse maps, weather control, traffic scenarios, and full sensor suite simulation.

💻

NVIDIA DRIVE Sim

Enterprise-grade simulator on Omniverse. Physically accurate sensor models, digital twin capability, and cloud-scale testing.

🚀

AirSim

Microsoft's open-source platform for drones and cars. Unreal Engine rendering with APIs for Python and C++. Great for RL research.

SUMO

Open-source traffic simulation for large-scale traffic flow modeling. Often used alongside 3D simulators for realistic traffic patterns.

Getting Started with CARLA

import carla
import random

# Connect to CARLA server
client = carla.Client('localhost', 2000)
world = client.get_world()

# Spawn an ego vehicle
bp_lib = world.get_blueprint_library()
vehicle_bp = bp_lib.find('vehicle.tesla.model3')
spawn_point = random.choice(world.get_map().get_spawn_points())
vehicle = world.spawn_actor(vehicle_bp, spawn_point)

# Attach sensors
camera_bp = bp_lib.find('sensor.camera.rgb')
camera_bp.set_attribute('image_size_x', '1920')
camera_bp.set_attribute('image_size_y', '1080')
camera_transform = carla.Transform(carla.Location(x=1.5, z=2.4))
camera = world.spawn_actor(camera_bp, camera_transform, attach_to=vehicle)

# Add LiDAR
lidar_bp = bp_lib.find('sensor.lidar.ray_cast')
lidar_bp.set_attribute('channels', '64')
lidar_bp.set_attribute('range', '100')
lidar = world.spawn_actor(lidar_bp, camera_transform, attach_to=vehicle)

# Enable autopilot for traffic
vehicle.set_autopilot(True)

Scenario Testing

Effective AV simulation requires testing across diverse scenarios:

CategoryScenariosPurpose
Normal drivingHighway, urban, suburbanBaseline performance validation
WeatherRain, fog, snow, night, glareSensor robustness testing
TrafficCongestion, merging, roundaboutsPlanning and behavior testing
Edge casesConstruction zones, emergency vehiclesRare event handling
AdversarialAggressive drivers, jaywalkersSafety margin validation

Metrics for Evaluation

  • Safety: Collision rate, time-to-collision, minimum distance to obstacles
  • Comfort: Jerk (rate of acceleration change), lateral acceleration, steering smoothness
  • Efficiency: Trip time, distance traveled, fuel/energy consumption
  • Rule compliance: Speed limit adherence, traffic signal compliance, lane violations
Key takeaway: CARLA is the best starting point for AV simulation research - it's free, open-source, and has an active community. Build a comprehensive scenario library and automate testing to catch regressions early. Remember that simulation is necessary but not sufficient - real-world testing is still required.

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.