Hugging Face's logo

bhujith10
/
deberta-v3-base-setfit_finetuned

Text Classification
setfit
Safetensors
sentence-transformers
deberta-v2
generated_from_setfit_trainer
Model card Files Community
deberta-v3-base-setfit_finetuned / README.md
bhujith10's picture
bhujith10
Push model using huggingface_hub.
aaedd2d verified
preview code
|
raw
history blame contribute delete
17.9 kB
metadata
base_model: microsoft/deberta-v3-base
datasets:
  - bhujith10/multi_class_classification_dataset
library_name: setfit
metrics:
  - accuracy
pipeline_tag: text-classification
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      Title: Detecting Adversarial Samples Using Density Ratio Estimates,

      Abstract: Machine learning models, especially based on deep architectures
      are used in

      everyday applications ranging from self driving cars to medical
      diagnostics. It

      has been shown that such models are dangerously susceptible to adversarial

      samples, indistinguishable from real samples to human eye, adversarial
      samples

      lead to incorrect classifications with high confidence. Impact of
      adversarial

      samples is far-reaching and their efficient detection remains an open
      problem.

      We propose to use direct density ratio estimation as an efficient model

      agnostic measure to detect adversarial samples. Our proposed method works

      equally well with single and multi-channel samples, and with different

      adversarial sample generation methods. We also propose a method to use
      density

      ratio estimates for generating adversarial samples with an added
      constraint of

      preserving density ratio.
  - text: >-
      Title: Dynamics of exciton magnetic polarons in CdMnSe/CdMgSe quantum
      wells: the effect of self-localization,

      Abstract: We study the exciton magnetic polaron (EMP) formation in
      (Cd,Mn)Se/(Cd,Mg)Se

      diluted-magnetic-semiconductor quantum wells using time-resolved

      photoluminescence (PL). The magnetic field and temperature dependencies of
      this

      dynamics allow us to separate the non-magnetic and magnetic contributions
      to

      the exciton localization. We deduce the EMP energy of 14 meV, which is in

      agreement with time-integrated measurements based on selective excitation
      and

      the magnetic field dependence of the PL circular polarization degree. The

      polaron formation time of 500 ps is significantly longer than the
      corresponding

      values reported earlier. We propose that this behavior is related to
      strong

      self-localization of the EMP, accompanied with a squeezing of the
      heavy-hole

      envelope wavefunction. This conclusion is also supported by the decrease
      of the

      exciton lifetime from 600 ps to 200 - 400 ps with increasing magnetic
      field and

      temperature.
  - text: >-
      Title: Exponential Sums and Riesz energies,

      Abstract: We bound an exponential sum that appears in the study of
      irregularities of

      distribution (the low-frequency Fourier energy of the sum of several Dirac

      measures) by geometric quantities: a special case is that for all $\left\{
      x_1,

      \dots, x_N\right\} \subset \mathbb{T}^2$, $X \geq 1$ and a universal $c>0$
      $$

      \sum_{i,j=1}^{N}{ \frac{X^2}{1 + X^4 \|x_i -x_j\|^4}} \lesssim \sum_{k \in

      \mathbb{Z}^2 \atop \|k\| \leq X}{ \left| \sum_{n=1}^{N}{ e^{2 \pi i

      \left\langle k, x_n \right\rangle}}\right|^2} \lesssim \sum_{i,j=1}^{N}{
      X^2

      e^{-c X^2\|x_i -x_j\|^2}}.$$ Since this exponential sum is intimately tied
      to

      rather subtle distribution properties of the points, we obtain nonlocal

      structural statements for near-minimizers of the Riesz-type energy. In the

      regime $X \gtrsim N^{1/2}$ both upper and lower bound match for

      maximally-separated point sets satisfying $\|x_i -x_j\| \gtrsim N^{-1/2}$.
  - text: >-
      Title: Influence of Spin Orbit Coupling in the Iron-Based Superconductors,

      Abstract: We report on the influence of spin-orbit coupling (SOC) in the
      Fe-based

      superconductors (FeSCs) via application of circularly-polarized spin and

      angle-resolved photoemission spectroscopy. We combine this technique in

      representative members of both the Fe-pnictides and Fe-chalcogenides with
      ab

      initio density functional theory and tight-binding calculations to
      establish an

      ubiquitous modification of the electronic structure in these materials
      imbued

      by SOC. The influence of SOC is found to be concentrated on the hole
      pockets

      where the superconducting gap is generally found to be largest. This
      result

      contests descriptions of superconductivity in these materials in terms of
      pure

      spin-singlet eigenstates, raising questions regarding the possible pairing

      mechanisms and role of SOC therein.
  - text: >-
      Title: Zero-point spin-fluctuations of single adatoms,

      Abstract: Stabilizing the magnetic signal of single adatoms is a crucial
      step towards

      their successful usage in widespread technological applications such as

      high-density magnetic data storage devices. The quantum mechanical nature
      of

      these tiny objects, however, introduces intrinsic zero-point
      spin-fluctuations

      that tend to destabilize the local magnetic moment of interest by
      dwindling the

      magnetic anisotropy potential barrier even at absolute zero temperature.
      Here,

      we elucidate the origins and quantify the effect of the fundamental
      ingredients

      determining the magnitude of the fluctuations, namely the ($i$) local
      magnetic

      moment, ($ii$) spin-orbit coupling and ($iii$) electron-hole Stoner

      excitations. Based on a systematic first-principles study of 3d and 4d
      adatoms,

      we demonstrate that the transverse contribution of the fluctuations is

      comparable in size to the magnetic moment itself, leading to a remarkable

      $\gtrsim$50$\%$ reduction of the magnetic anisotropy energy. Our analysis
      gives

      rise to a comprehensible diagram relating the fluctuation magnitude to

      characteristic features of adatoms, providing practical guidelines for

      designing magnetically stable nanomagnets with minimal quantum
      fluctuations.
inference: false

SetFit with microsoft/deberta-v3-base

This is a SetFit model trained on the bhujith10/multi_class_classification_dataset dataset that can be used for Text Classification. This SetFit model uses microsoft/deberta-v3-base as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("bhujith10/deberta-v3-base-setfit_finetuned")
# Run inference
preds = model("Title: Influence of Spin Orbit Coupling in the Iron-Based Superconductors,
Abstract: We report on the influence of spin-orbit coupling (SOC) in the Fe-based
superconductors (FeSCs) via application of circularly-polarized spin and
angle-resolved photoemission spectroscopy. We combine this technique in
representative members of both the Fe-pnictides and Fe-chalcogenides with ab
initio density functional theory and tight-binding calculations to establish an
ubiquitous modification of the electronic structure in these materials imbued
by SOC. The influence of SOC is found to be concentrated on the hole pockets
where the superconducting gap is generally found to be largest. This result
contests descriptions of superconductivity in these materials in terms of pure
spin-singlet eigenstates, raising questions regarding the possible pairing
mechanisms and role of SOC therein.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 23 148.1 303

Training Hyperparameters

  • batch_size: (4, 4)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 0.4731 -
0.0078 50 0.4561 -
0.0155 100 0.4156 -
0.0233 150 0.2469 -
0.0311 200 0.2396 -
0.0388 250 0.2376 -
0.0466 300 0.2519 -
0.0543 350 0.1987 -
0.0621 400 0.1908 -
0.0699 450 0.161 -
0.0776 500 0.1532 -
0.0854 550 0.17 -
0.0932 600 0.139 -
0.1009 650 0.1406 -
0.1087 700 0.1239 -
0.1165 750 0.1332 -
0.1242 800 0.1566 -
0.1320 850 0.0932 -
0.1398 900 0.1101 -
0.1475 950 0.1153 -
0.1553 1000 0.0979 -
0.1630 1050 0.0741 -
0.1708 1100 0.0603 -
0.1786 1150 0.1027 -
0.1863 1200 0.0948 -
0.1941 1250 0.0968 -
0.2019 1300 0.085 -
0.2096 1350 0.0883 -
0.2174 1400 0.0792 -
0.2252 1450 0.1054 -
0.2329 1500 0.0556 -
0.2407 1550 0.0777 -
0.2484 1600 0.0922 -
0.2562 1650 0.076 -
0.2640 1700 0.0693 -
0.2717 1750 0.0857 -
0.2795 1800 0.0907 -
0.2873 1850 0.0621 -
0.2950 1900 0.0792 -
0.3028 1950 0.0608 -
0.3106 2000 0.052 -
0.3183 2050 0.056 -
0.3261 2100 0.0501 -
0.3339 2150 0.0559 -
0.3416 2200 0.0526 -
0.3494 2250 0.0546 -
0.3571 2300 0.0398 -
0.3649 2350 0.0527 -
0.3727 2400 0.0522 -
0.3804 2450 0.0468 -
0.3882 2500 0.0465 -
0.3960 2550 0.0393 -
0.4037 2600 0.0583 -
0.4115 2650 0.0278 -
0.4193 2700 0.0502 -
0.4270 2750 0.0413 -
0.4348 2800 0.0538 -
0.4425 2850 0.0361 -
0.4503 2900 0.0648 -
0.4581 2950 0.0459 -
0.4658 3000 0.0521 -
0.4736 3050 0.0288 -
0.4814 3100 0.0323 -
0.4891 3150 0.0335 -
0.4969 3200 0.0472 -
0.5047 3250 0.0553 -
0.5124 3300 0.0426 -
0.5202 3350 0.0276 -
0.5280 3400 0.0395 -
0.5357 3450 0.042 -
0.5435 3500 0.0343 -
0.5512 3550 0.0314 -
0.5590 3600 0.0266 -
0.5668 3650 0.0314 -
0.5745 3700 0.0379 -
0.5823 3750 0.0485 -
0.5901 3800 0.0311 -
0.5978 3850 0.0415 -
0.6056 3900 0.0266 -
0.6134 3950 0.0384 -
0.6211 4000 0.0348 -
0.6289 4050 0.0298 -
0.6366 4100 0.032 -
0.6444 4150 0.031 -
0.6522 4200 0.0367 -
0.6599 4250 0.0289 -
0.6677 4300 0.0333 -
0.6755 4350 0.0281 -
0.6832 4400 0.0307 -
0.6910 4450 0.0312 -
0.6988 4500 0.0488 -
0.7065 4550 0.03 -
0.7143 4600 0.0309 -
0.7220 4650 0.031 -
0.7298 4700 0.0268 -
0.7376 4750 0.0324 -
0.7453 4800 0.041 -
0.7531 4850 0.0349 -
0.7609 4900 0.0349 -
0.7686 4950 0.0291 -
0.7764 5000 0.025 -
0.7842 5050 0.0249 -
0.7919 5100 0.0272 -
0.7997 5150 0.0302 -
0.8075 5200 0.0414 -
0.8152 5250 0.0295 -
0.8230 5300 0.033 -
0.8307 5350 0.0203 -
0.8385 5400 0.0275 -
0.8463 5450 0.0354 -
0.8540 5500 0.0254 -
0.8618 5550 0.0313 -
0.8696 5600 0.0296 -
0.8773 5650 0.0248 -
0.8851 5700 0.036 -
0.8929 5750 0.025 -
0.9006 5800 0.0234 -
0.9084 5850 0.0221 -
0.9161 5900 0.0314 -
0.9239 5950 0.0273 -
0.9317 6000 0.0299 -
0.9394 6050 0.0262 -
0.9472 6100 0.0285 -
0.9550 6150 0.021 -
0.9627 6200 0.0215 -
0.9705 6250 0.0312 -
0.9783 6300 0.0259 -
0.9860 6350 0.0234 -
0.9938 6400 0.0222 -
1.0 6440 - 0.1609

Framework Versions

  • Python: 3.10.14
  • SetFit: 1.1.0
  • Sentence Transformers: 3.3.1
  • Transformers: 4.45.2
  • PyTorch: 2.4.0
  • Datasets: 3.0.1
  • Tokenizers: 0.20.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}