Latency (Sequential)

Latency Per Batch

Bert Latency BS=1 Bert Latency BS=2 Bert Latency BS=4 Bert Latency BS=8 Bert Latency BS=16 Bert Latency BS=32 Bert Latency BS=64 Bert Latency BS=128 Bert Latency BS=256 Bert Latency BS=512 Bert Latency BS=1024

Latency Per Optimization

Bert Latency Opt=None Bert Latency Opt=Script

Throughput (Sequential)

Throughput Per Batch

Bert Throughput BS=1 Bert Throughput BS=2 Bert Throughput BS=4 Bert Throughput BS=8 Bert Throughput BS=16 Bert Throughput BS=32 Bert Throughput BS=64 Bert Throughput BS=128 Bert Throughput BS=256 Bert Throughput BS=512 Bert Throughput BS=1024

Throughput Per Optimization

Bert Latency Opt=None Bert Latency Opt=Script

Speedup

Bert Latency Opt=Script

Throughput Per Batch

Bert Throughput per BS