GPT-2

Latency

Latency Per Batch

GPT-2 Latency BS=1 GPT-2 Latency BS=2 GPT-2 Latency BS=4 GPT-2 Latency BS=8 GPT-2 Latency BS=16 GPT-2 Latency BS=32 GPT-2 Latency BS=64 GPT-2 Latency BS=128 GPT-2 Latency BS=256 GPT-2 Latency BS=512 GPT-2 Latency BS=1024

Latency Per Optimization

GPT-2 Latency Opt=None GPT-2 Latency Opt=Script

Throughput

Throughput Per Batch

GPT-2 Throughput BS=1 GPT-2 Throughput BS=2 GPT-2 Throughput BS=4 GPT-2 Throughput BS=8 GPT-2 Throughput BS=16 GPT-2 Throughput BS=32 GPT-2 Throughput BS=64 GPT-2 Throughput BS=128 GPT-2 Throughput BS=256 GPT-2 Throughput BS=512 GPT-2 Throughput BS=1024

Throughput Per Optimization

GPT-2 Latency Opt=None GPT-2 Latency Opt=Script

Speedup

GPT-2 Latency Opt=None GPT-2 Latency Opt=Script

Throughput Per Batch

GPT-2 Throughput per BS

NUMA-Effects

% Latency Degradation HeatMap

GPT-2 Latency Degradation

% Throughput Degradation HeatMap

GPT-2 Latency Degradation

Latency Per Batch

GPT-2 Latency BS=1 GPT-2 Latency BS=2 GPT-2 Latency BS=4 GPT-2 Latency BS=8 GPT-2 Latency BS=16 GPT-2 Latency BS=32 GPT-2 Latency BS=64 GPT-2 Latency BS=128 GPT-2 Latency BS=256 GPT-2 Latency BS=512 GPT-2 Latency BS=1024

% Latency Loss Per Batch

GPT-2 Latency BS=1 GPT-2 Latency BS=2 GPT-2 Latency BS=4 GPT-2 Latency BS=8 GPT-2 Latency BS=16 GPT-2 Latency BS=32 GPT-2 Latency BS=64 GPT-2 Latency BS=128 GPT-2 Latency BS=256 GPT-2 Latency BS=512 GPT-2 Latency BS=1024

Throughput Per Batch

GPT-2 Throughput BS=1 GPT-2 Throughput BS=2 GPT-2 Throughput BS=4 GPT-2 Throughput BS=8 GPT-2 Throughput BS=16 GPT-2 Throughput BS=32 GPT-2 Throughput BS=64 GPT-2 Throughput BS=128 GPT-2 Throughput BS=256 GPT-2 Throughput BS=512 GPT-2 Throughput BS=1024

% Throughput Loss Per Batch

GPT-2 Throughput BS=1 GPT-2 Throughput BS=2 GPT-2 Throughput BS=4 GPT-2 Throughput BS=8 GPT-2 Throughput BS=16 GPT-2 Throughput BS=32 GPT-2 Throughput BS=64 GPT-2 Throughput BS=128 GPT-2 Throughput BS=256 GPT-2 Throughput BS=512 GPT-2 Throughput BS=1024