README.md

---
tags:
- optimum
datasets:
- banking77
metrics:
- accuracy
model-index:
- name: quantized-distilbert-banking77
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: banking77
      type: banking77
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.9224
---


# Quantized-distilbert-banking77

This model is a statically quantized version of [optimum/distilbert-base-uncased-finetuned-banking77](https://huggingface.co/optimum/distilbert-base-uncased-finetuned-banking77) on the `banking77` dataset.

The model was created using the [optimum-static-quantization](https://github.com/philschmid/optimum-static-quantization) notebook.

It achieves the following results on the evaluation set:

**Accuracy**

- Vanilla model: 92.5%
- Quantized model: 92.24%  

> The quantized model achieves 99.72% accuracy of the fp32 model

**Latency**

Payload sequence length: 128  
Instance type: AWS c6i.xlarge  

| latency | vanilla transformers | quantized optimum model | improvement |
|---------|----------------------|-------------------------|-------------|
| p95     | 75.69ms              | 26.75ms                 | 2.83x       |
| avg     | 57.52ms              | 24.86ms                 | 2.31x       |

## How to use

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained("philschmid/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("philschmid/quantized-distilbert-banking77")

remote_clx = pipeline("text-classification",model=model, tokenizer=tokenizer)

remote_clx("What is the exchange rate like on this app?")
```