Latency (Sequential)

Latency Per Batch

GPT2 Latency BS=1 GPT2 Latency BS=2 GPT2 Latency BS=3 GPT2 Latency BS=4 GPT2 Latency BS=5 GPT2 Latency BS=6 GPT2 Latency BS=7 GPT2 Latency BS=8 GPT2 Latency BS=9 GPT2 Latency BS=10 GPT2 Latency BS=11 GPT2 Latency BS=12 GPT2 Latency BS=13 GPT2 Latency BS=14 GPT2 Latency BS=15 GPT2 Latency BS=16 GPT2 Latency BS=17 GPT2 Latency BS=18 GPT2 Latency BS=19 GPT2 Latency BS=20 GPT2 Latency BS=21 GPT2 Latency BS=22 GPT2 Latency BS=23 GPT2 Latency BS=24 GPT2 Latency BS=25 GPT2 Latency BS=26 GPT2 Latency BS=27 GPT2 Latency BS=28 GPT2 Latency BS=29 GPT2 Latency BS=30 GPT2 Latency BS=31 GPT2 Latency BS=32 GPT2 Latency BS=64 GPT2 Latency BS=128 GPT2 Latency BS=256 GPT2 Latency BS=512 GPT2 Latency BS=1024

Latency Per Optimization

GPT2 Latency Opt=Script

Throughput (Sequential)

Throughput Per Batch

GPT2 Throughput BS=1 GPT2 Throughput BS=2 GPT2 Throughput BS=3 GPT2 Throughput BS=4 GPT2 Throughput BS=5 GPT2 Throughput BS=6 GPT2 Throughput BS=7 GPT2 Throughput BS=8 GPT2 Throughput BS=9 GPT2 Throughput BS=10 GPT2 Throughput BS=11 GPT2 Throughput BS=12 GPT2 Throughput BS=13 GPT2 Throughput BS=14 GPT2 Throughput BS=15 GPT2 Throughput BS=16 GPT2 Throughput BS=17 GPT2 Throughput BS=18 GPT2 Throughput BS=19 GPT2 Throughput BS=20 GPT2 Throughput BS=21 GPT2 Throughput BS=22 GPT2 Throughput BS=23 GPT2 Throughput BS=24 GPT2 Throughput BS=25 GPT2 Throughput BS=26 GPT2 Throughput BS=27 GPT2 Throughput BS=28 GPT2 Throughput BS=29 GPT2 Throughput BS=30 GPT2 Throughput BS=31 GPT2 Throughput BS=32 GPT2 Throughput BS=64 GPT2 Throughput BS=128 GPT2 Throughput BS=256 GPT2 Throughput BS=512 GPT2 Throughput BS=1024

Throughput Per Optimization

GPT2 Throughput Opt=Script

Throughput Per Batch

GPT2 Throughput per BS