Instructions to use mlx-community/embeddinggemma-300m-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use mlx-community/embeddinggemma-300m-4bit with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("mlx-community/embeddinggemma-300m-4bit") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - MLX
How to use mlx-community/embeddinggemma-300m-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir embeddinggemma-300m-4bit mlx-community/embeddinggemma-300m-4bit
- Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
Update README.md
Browse files
README.md
CHANGED
|
@@ -30,8 +30,20 @@ import mlx.core as mx
|
|
| 30 |
|
| 31 |
model, tokenizer = load("mlx-community/embeddinggemma-300m-4bit")
|
| 32 |
|
| 33 |
-
# For text
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
embeddings = output.text_embeds # Normalized embeddings
|
| 36 |
|
| 37 |
# Compute dot product between normalized embeddings
|
|
@@ -41,4 +53,22 @@ print("Similarity matrix between texts:")
|
|
| 41 |
print(similarity_matrix)
|
| 42 |
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
```
|
|
|
|
| 30 |
|
| 31 |
model, tokenizer = load("mlx-community/embeddinggemma-300m-4bit")
|
| 32 |
|
| 33 |
+
# For text embedding
|
| 34 |
+
sentences = [
|
| 35 |
+
"task: sentence similarity | query: Nothing really matters.",
|
| 36 |
+
"task: sentence similarity | query: The dog is barking.",
|
| 37 |
+
"task: sentence similarity | query: The dog is barking.",
|
| 38 |
+
]
|
| 39 |
+
|
| 40 |
+
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='mlx')
|
| 41 |
+
|
| 42 |
+
# Compute token embeddings
|
| 43 |
+
input_ids = encoded_input['input_ids']
|
| 44 |
+
attention_mask = encoded_input['attention_mask']
|
| 45 |
+
output = model(input_ids, attention_mask)
|
| 46 |
+
|
| 47 |
embeddings = output.text_embeds # Normalized embeddings
|
| 48 |
|
| 49 |
# Compute dot product between normalized embeddings
|
|
|
|
| 53 |
print(similarity_matrix)
|
| 54 |
|
| 55 |
|
| 56 |
+
# You can use these task-specific prefixes for different tasks
|
| 57 |
+
task_prefixes = {
|
| 58 |
+
"BitextMining": "task: search result | query: ",
|
| 59 |
+
"Clustering": "task: clustering | query: ",
|
| 60 |
+
"Classification": "task: classification | query: ",
|
| 61 |
+
"MultilabelClassification": "task: classification | query: ",
|
| 62 |
+
"PairClassification": "task: sentence similarity | query: ",
|
| 63 |
+
"InstructionRetrieval": "task: code retrieval | query: ",
|
| 64 |
+
"Reranking": "task: search result | query: ",
|
| 65 |
+
"Retrieval": "task: search result | query: ",
|
| 66 |
+
"Retrieval-query": "task: search result | query: ",
|
| 67 |
+
"Retrieval-document": "title: none | text: ",
|
| 68 |
+
"STS": "task: sentence similarity | query: ",
|
| 69 |
+
"Summarization": "task: summarization | query: ",
|
| 70 |
+
"document": "title: none | text: "
|
| 71 |
+
}
|
| 72 |
+
|
| 73 |
+
|
| 74 |
```
|