Latency (Sequential)

Latency Per Batch

BERT Latency BS=1 BERT Latency BS=2 BERT Latency BS=4 BERT Latency BS=8 BERT Latency BS=16 BERT Latency BS=32 BERT Latency BS=64 BERT Latency BS=128 BERT Latency BS=256 BERT Latency BS=512 BERT Latency BS=1024

Latency Per Optimization

BERT Latency HT=False

BERT Latency HT=True

Throughput (Sequential)

Throughput Per Batch

BERT Throughput BS=1 BERT Throughput BS=2 BERT Throughput BS=4 BERT Throughput BS=8 BERT Throughput BS=16 BERT Throughput BS=32 BERT Throughput BS=64 BERT Throughput BS=128 BERT Throughput BS=256 BERT Throughput BS=512 BERT Throughput BS=1024

Throughput Per Optimization

BERT Latency HT=False

BERT Latency HT=True