Latency

Latency Per Batch

gpt2 Latency BS=1 gpt2 Latency BS=8 gpt2 Latency BS=16 gpt2 Latency BS=32

Latency Per Optimization

gpt2 Latency Opt=None gpt2 Latency Opt=Script

Throughput

Throughput Per Batch

gpt2 Throughput BS=1 gpt2 Throughput BS=8 gpt2 Throughput BS=16 gpt2 Throughput BS=32

Throughput Per Optimization

gpt2 Latency Opt=None gpt2 Latency Opt=Script

Speedup

gpt2 Latency Opt=None gpt2 Latency Opt=Script