AtmicQuoterv2

Fine-tuned from SriRamanaAtmic/AtmicQuoterv1 (itself a fine-tune of BAAI/bge-small-en-v1.5) on an expert-vetted Q&A set.

Training data: mined from expert_pass.csv (expert-vetted {Question, Response} pairs). For each question, the expert Response was used as an anchor and searched against the 2,875-passage Sri Ramana Maharshi citation corpus using a weighted, per-query min-max-normalized blend of three retrieval signals:

• AtmicQuoterv1 cosine similarity — weight 0.25
• AtmicEmbeddingv3 cosine similarity — weight 0.45
• BM25 (lexical) — weight 0.30

The top 4 distinct-source-family matches per question were kept as positives (4 separate training rows, one positive each), sharing a pool of 10 mined hard negatives that excludes all 4 positives' source families. 445 questions -> 1,780 rows -> 1,424 train / 355 val (query-level split, 0 leakage).

Benchmark: closed-pool citation retrieval (355 val queries, 2,857-passage pool)

Full results, all four models scored on the same val set / 2,857-passage closed pool (dense-only, each model's own embedding space):

metric	baseline (bge-small)	bge-m3	v1	v2
accuracy@1	0.0254	0.0254	0.0451	0.0507
recall@3	0.0648	0.0648	0.0930	0.1127
recall@5	0.0901	0.0986	0.1239	0.1606
recall@10	0.1324	0.1408	0.1859	0.2620
mrr@3	0.0427	0.0432	0.0657	0.0770
mrr@10	0.0544	0.0560	0.0805	0.1008
ndcg@3	0.0484	0.0488	0.0727	0.0861
ndcg@10	0.0727	0.0758	0.1049	0.1380
map@100	0.0610	0.0626	0.0892	0.1115

v2 wins outright on every metric, clearly ahead of bge-m3 despite bge-m3 being a much larger general-purpose multilingual model. v1 also beats both untrained baselines, and bge-m3 roughly ties stock bge-small — domain fine-tuning (v1->v2) matters far more than model scale for this task.

Benchmark: production-shaped pipeline (dense top-20 -> monoBERT rerank -> top-4)

This mimics how the model is actually meant to be served, rather than raw closed-pool ranking: for each of 89 unique held-out questions (88 with 4 mined valid citations, 1 with 3), the quoter model dense-retrieves the top 20 candidates from the full 2,857-passage corpus (pure query-mode, the standard BGE-instruction retrieval), castorini/monobert-large-msmarco reranks those 20, and the top 4 are what would be shown to a user.

metric	base (bge-small)	v1	v2
stage1_recall@20	0.1751	0.2575	0.3502
precision@4	0.0627	0.0852	0.1002
hit_rate@4 (>=1 of 4 found)	0.2360	0.3034	0.3258

stage1_recall@20 — of a question's true citations, the fraction that even made the top-20 retrieval pool (the ceiling the reranker can't exceed). precision@4 — of the 4 citations served, the fraction that are correct. hit_rate@4 — the fraction of questions where at least one of the 4 served citations is correct.

In production, the app will show at least one correct citation for about 32.6% of queries with AtmicQuoterv2 — versus 23.6% for stock bge-small and 30.3% for AtmicQuoterv1. The gain traces back to stage 1: v2's retriever gets far more of the true citations into the top-20 pool in the first place (35.0% vs. 17.5%/25.8%), which the reranker then has more to work with — retrieval recall is the ceiling here, not the reranker, and v2 raises that ceiling the most of the three.

Absolute scores are low in every configuration tested — this is a genuinely hard closed-pool task (natural questions against ~2,857 short, often mutually confusable quote fragments) — but AtmicQuoterv2 consistently raises the retrieval ceiling the most, which is what any downstream reranking or serving strategy is bounded by.

---

SentenceTransformer based on SriRamanaAtmic/AtmicQuoterv1

This is a sentence-transformers model finetuned from SriRamanaAtmic/AtmicQuoterv1. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: SriRamanaAtmic/AtmicQuoterv1
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'Represent this sentence for searching relevant passages: What were his final days like, and what did he teach about his own passing?',
]
documents = [
    'During his last illness, when devotees expressed grief at his approaching departure, Bhagavan repeatedly assured them, "They say that I am dying, but I am not going away. I am here". He emphasized that they attached too much importance to the body and that he would remain as the Inner Guru.',
    'Bhagavan placed his hand over the heart of Griddalur Satyanarayana Rao, who was dying of terminal cancer. After a night of internal physical purging, the devotee was cured the next day.',
    'Even if you give up the physical body, the suffering associated with the body cannot be avoided',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.6090, 0.5375, 0.1721]])

Evaluation

Metrics

Information Retrieval

• Dataset: atmic-val
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0507
cosine_accuracy@3	0.1127
cosine_accuracy@5	0.1606
cosine_accuracy@10	0.262
cosine_precision@1	0.0507
cosine_precision@3	0.0376
cosine_precision@5	0.0321
cosine_precision@10	0.0262
cosine_recall@1	0.0507
cosine_recall@3	0.1127
cosine_recall@5	0.1606
cosine_recall@10	0.262
cosine_ndcg@3	0.0861
cosine_ndcg@10	0.138
cosine_mrr@3	0.077
cosine_mrr@10	0.1008
cosine_map@100	0.1115

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,424 training samples
• Columns: anchor, positive, negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, negative_7, negative_8, negative_9, and negative_10
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10
  type	string	string	string	string	string	string	string	string	string	string	string	string
  modality	text	text	text	text	text	text	text	text	text	text	text	text
  details	min: 14 tokens mean: 26.58 tokens max: 40 tokens	min: 9 tokens mean: 38.71 tokens max: 127 tokens	min: 10 tokens mean: 43.5 tokens max: 128 tokens	min: 8 tokens mean: 45.81 tokens max: 128 tokens	min: 10 tokens mean: 40.0 tokens max: 107 tokens	min: 7 tokens mean: 42.92 tokens max: 128 tokens	min: 11 tokens mean: 37.92 tokens max: 108 tokens	min: 11 tokens mean: 39.12 tokens max: 118 tokens	min: 8 tokens mean: 38.81 tokens max: 89 tokens	min: 10 tokens mean: 43.81 tokens max: 87 tokens	min: 11 tokens mean: 37.85 tokens max: 99 tokens	min: 12 tokens mean: 36.85 tokens max: 98 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10
  Represent this sentence for searching relevant passages: How do good actions purify the mind? (Chitta-shuddhi)	"If you are so anxious for trance any narcotic will bring it about. Drug-habit will be the result and not liberation. There are vasanas in the latent state even in trance. The vasanas must be destroyed."	If the mind, which is the cause of all thoughts and activities, vanishes, the external objects will vanish	Give up all efforts and surrender. Let the ‘I’, that wants the Divine World die, and the Divine in you will be realised	Get to business on the agreed point, namely that the ego must be got rid of	Drawing in the thoughts, restraining them and preventing them from going outwards is called vairagya	The mind should be Made to merge into the Self; the practice must be long because it is slow	Unless you have obtained the grace / Of the good guru who has subsumed / All triads in the One, you can / Have no abiding place / In the infinite bliss of moksha, / The ultimate goal and good	Win the state of Deliverance about the time they cease to be boys; and they do so with little or no effort	If the objects have an independent existence... it may be possible for you to go away from them. But they don’t exist apart from you; they owe their existence to you, your thought	The quest must start with the mind turned inward to oppose the rushing thoughts	The supreme Jnana obtained with the touch of the Saint can never be won through the study of any number of Scriptures, or by any store of good deeds
  Represent this sentence for searching relevant passages: How do good actions purify the mind? (Chitta-shuddhi)	The seeker’s aim must be to drain away the vasanas from the heart and let no reflecting medium obstruct the light	If the mind, which is the cause of all thoughts and activities, vanishes, the external objects will vanish	Give up all efforts and surrender. Let the ‘I’, that wants the Divine World die, and the Divine in you will be realised	Get to business on the agreed point, namely that the ego must be got rid of	Drawing in the thoughts, restraining them and preventing them from going outwards is called vairagya	The mind should be Made to merge into the Self; the practice must be long because it is slow	Unless you have obtained the grace / Of the good guru who has subsumed / All triads in the One, you can / Have no abiding place / In the infinite bliss of moksha, / The ultimate goal and good	Win the state of Deliverance about the time they cease to be boys; and they do so with little or no effort	If the objects have an independent existence... it may be possible for you to go away from them. But they don’t exist apart from you; they owe their existence to you, your thought	The quest must start with the mind turned inward to oppose the rushing thoughts	The supreme Jnana obtained with the touch of the Saint can never be won through the study of any number of Scriptures, or by any store of good deeds
  Represent this sentence for searching relevant passages: How do good actions purify the mind? (Chitta-shuddhi)	Bestow grace on me so that my mind, now impregnated with impressions (vasanas) of the world may, rid of them, gain the vasana (fragrance) of Brahman that is worldless and grant me the knowledge of union of Atman and Brahman.	If the mind, which is the cause of all thoughts and activities, vanishes, the external objects will vanish	Give up all efforts and surrender. Let the ‘I’, that wants the Divine World die, and the Divine in you will be realised	Get to business on the agreed point, namely that the ego must be got rid of	Drawing in the thoughts, restraining them and preventing them from going outwards is called vairagya	The mind should be Made to merge into the Self; the practice must be long because it is slow	Unless you have obtained the grace / Of the good guru who has subsumed / All triads in the One, you can / Have no abiding place / In the infinite bliss of moksha, / The ultimate goal and good	Win the state of Deliverance about the time they cease to be boys; and they do so with little or no effort	If the objects have an independent existence... it may be possible for you to go away from them. But they don’t exist apart from you; they owe their existence to you, your thought	The quest must start with the mind turned inward to oppose the rushing thoughts	The supreme Jnana obtained with the touch of the Saint can never be won through the study of any number of Scriptures, or by any store of good deeds

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• num_train_epochs: 6.0
• learning_rate: 1e-05
• warmup_steps: 0.1
• weight_decay: 0.01
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• per_device_train_batch_size: 16
• num_train_epochs: 6.0
• max_steps: -1
• learning_rate: 1e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	atmic-val_cosine_ndcg@10
0.5618	50	4.3782	-
1.0	89	-	0.1125
1.1236	100	4.0802	-
1.6854	150	3.7841	-
2.0	178	-	0.1325
2.2472	200	3.6681	-
2.8090	250	3.4944	-
3.0	267	-	0.1331
3.3708	300	3.3190	-
3.9326	350	3.2448	-
4.0	356	-	0.1358
4.4944	400	3.1086	-
5.0	445	-	0.138
5.0562	450	3.1606	-
5.6180	500	3.0317	-
6.0	534	-	0.1378
-1	-1	-	0.1380

• The bold row denotes the saved checkpoint.

Training Time

• Training: 48.1 minutes
• Evaluation: 44.3 seconds
• Total: 48.8 minutes

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.12.1
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}