Add SetFit model

d1fa4c7 verified 9 months ago

21.1 kB

	---
	library_name: setfit
	tags:
	- setfit
	- sentence-transformers
	- text-classification
	- generated_from_setfit_trainer
	base_model: jhgan/ko-sbert-multitask
	metrics:
	- accuracy
	widget:
	- text: 제습기 전기세가 어느 정도 나오는지 궁금합니다. 요즘 비가 계속 오기도 했고 집 안의 습도가 너무 높아서 거의 하루 종일 제습기를 틀어놓고는
	하는데 인터넷에서 제습기 전기세가 꽤나 많이 나온다는 내용의 글응 보았습니다. 제습기 전기세가 다른 가전제품에 비해서 어느 정도 나오는지 궁금합니다.
	- text: '우주선에 사용하는 전선은 일상생활에 사용하는것과 같은 재질인가요? 궁금한데요. 영화같은데보면 일반전선이랑 같은거같은데 실제로 그러진않겠죠?
	지구대기랑 다른 악조건에 견디게 만들거같은데 어떨런지요? '
	- text: 부산 바다가 보이는 카페는 어떤 곳이 있나요? 다음달에 부산 여행을 갈 생각인데요. 부산 바다가 좀 보이는 카페를 가보고 싶어요. 하지만
	너무 사람들이 많이 오는곳은 제외 하구요. 1층이 아니라 고층이면 더 좋습니다!
	- text: 경희대학교 토목공학과 전망 괜찮을까요? 경희대학교 토목공학과 전망 괜찮은지 궁금합니다.토목공학 자체가 지금 그렇게 좋은 느낌은 아닌
	것 같은데..ㅠㅠ우스갯 소리로 잘되려면 통일이 되어야 토목공학과가 살아날 수 있다고 하던데ㅋ....그래도 학교 네임 밸류가 좋아서.. 괜찮을
	것 같기도하고전망 어떤가요!?
	- text: 치아관리 치아가잇몸이올라가 치아뿌리분이조금보여요 제목그대로 치아뿌리가보이고 잇몸이 올라가있는거같아요 잇몸에좋은치료방법이있나요. 잇몸치약을쓰고있는데.
	다른방법이있으면알려주세요
	pipeline_tag: text-classification
	inference: true
	model-index:
	- name: SetFit with jhgan/ko-sbert-multitask
	results:
	- task:
	type: text-classification
	name: Text Classification
	dataset:
	name: Unknown
	type: unknown
	split: test
	metrics:
	- type: accuracy
	value: 0.5528846153846154
	name: Accuracy
	---

	# SetFit with jhgan/ko-sbert-multitask

	This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [jhgan/ko-sbert-multitask](https://huggingface.co/jhgan/ko-sbert-multitask) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.

	The model has been trained using an efficient few-shot learning technique that involves:

	1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
	2. Training a classification head with features from the fine-tuned Sentence Transformer.

	## Model Details

	### Model Description
	- Model Type: SetFit
	- Sentence Transformer body: [jhgan/ko-sbert-multitask](https://huggingface.co/jhgan/ko-sbert-multitask)
	- Classification head: a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
	- Maximum Sequence Length: 128 tokens
	- Number of Classes: 208 classes
	<!-- - Training Dataset: [Unknown](https://huggingface.co/datasets/unknown) -->
	<!-- - Language: Unknown -->
	<!-- - License: Unknown -->

	### Model Sources

	- Repository: [SetFit on GitHub](https://github.com/huggingface/setfit)
	- Paper: [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
	- Blogpost: [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

	## Evaluation

	### Metrics
	\| Label \| Accuracy \|
	\|:--------\|:---------\|
	\| all \| 0.5529 \|

	## Uses

	### Direct Use for Inference

	First install the SetFit library:

	```bash
	pip install setfit
	```

	Then you can load this model and run inference.

	```python
	from setfit import SetFitModel

	# Download from the 🤗 Hub
	model = SetFitModel.from_pretrained("djsull/setfit_classifier")
	# Run inference
	preds = model("치아관리 치아가잇몸이올라가 치아뿌리분이조금보여요 제목그대로 치아뿌리가보이고 잇몸이 올라가있는거같아요 잇몸에좋은치료방법이있나요. 잇몸치약을쓰고있는데. 다른방법이있으면알려주세요")
	```

	<!--
	### Downstream Use

	List how someone could finetune this model on their own dataset.
	-->

	<!--
	### Out-of-Scope Use

	List how the model may foreseeably be misused and address what users ought not to do with the model.
	-->

	<!--
	## Bias, Risks and Limitations

	What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.
	-->

	<!--
	### Recommendations

	What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.
	-->

	## Training Details

	### Training Set Metrics
	\| Training set \| Min \| Median \| Max \|
	\|:-------------\|:----\|:--------\|:----\|
	\| Word count \| 10 \| 34.4464 \| 468 \|

	\| Label \| Training Sample Count \|
	\|:----------\|:----------------------\|
	\| PC·노트북 \| 0 \|
	\| 양도소득세 \| 0 \|
	\| 토목공학 \| 7 \|
	\| 책·독서 \| 7 \|
	\| 대인관계 \| 7 \|
	\| 기타 고민상담 \| 7 \|
	\| 공인노무사 자격증 \| 7 \|
	\| 서울·수도권 \| 7 \|
	\| 연세대학교 \| 7 \|
	\| 자동차 수리 \| 7 \|
	\| 반려동물 건강 \| 7 \|
	\| 예능 \| 7 \|
	\| 가압류·가처분 \| 7 \|
	\| 기타 육아상담 \| 7 \|
	\| 역사 \| 7 \|
	\| 손해사정사 자격증 \| 7 \|
	\| 환자 식단 \| 7 \|
	\| 서울시립대학교 \| 7 \|
	\| 기타 심리상담 \| 0 \|
	\| 형사 \| 7 \|
	\| 치과대학 \| 7 \|
	\| 예금·적금 \| 7 \|
	\| 물리 \| 7 \|
	\| 일본 \| 7 \|
	\| 낚시 \| 7 \|
	\| 현대중공업 \| 7 \|
	\| 교통사고 \| 7 \|
	\| 공인회계사 자격증 \| 7 \|
	\| 농구 \| 7 \|
	\| 종합부동산세 \| 7 \|
	\| 비뇨의학과 \| 7 \|
	\| 명예훼손·모욕 \| 0 \|
	\| 지게차운전기능사 \| 7 \|
	\| 가족·이혼 \| 7 \|
	\| 현대자동차 \| 7 \|
	\| 기계공학 \| 7 \|
	\| 영양제 \| 7 \|
	\| 연말정산 \| 7 \|
	\| 성균관대학교 \| 7 \|
	\| 미술 \| 7 \|
	\| 양육·훈육 \| 7 \|
	\| 경제용어 \| 0 \|
	\| 경희대학교 \| 7 \|
	\| 금융 \| 7 \|
	\| 스케이트 \| 7 \|
	\| 반려동물 훈련 \| 7 \|
	\| 한약 \| 7 \|
	\| 생물·생명 \| 7 \|
	\| 사회복지사 자격증 \| 7 \|
	\| 한의학 \| 7 \|
	\| 부동산·임대차 \| 7 \|
	\| 회생·파산 \| 7 \|
	\| 임금체불 \| 7 \|
	\| 변리사 자격증 \| 7 \|
	\| 피부과 \| 7 \|
	\| 놀이 \| 7 \|
	\| 대출 \| 7 \|
	\| 재산범죄 \| 0 \|
	\| 영유아·아동 식단 \| 7 \|
	\| 계절가전 \| 7 \|
	\| 재료공학 \| 7 \|
	\| 전라 \| 7 \|
	\| 다이어트 식단 \| 7 \|
	\| 경제정책 \| 0 \|
	\| 캠핑 \| 7 \|
	\| 철학 \| 7 \|
	\| 치과 \| 7 \|
	\| 서강대학교 \| 7 \|
	\| 자아·자기이해 \| 0 \|
	\| 환경·에너지 \| 7 \|
	\| 포항공과대학교 \| 7 \|
	\| 제주 \| 7 \|
	\| 행정사 자격증 \| 7 \|
	\| 임금·급여 \| 7 \|
	\| 세탁수선 \| 0 \|
	\| 중학교 생활 \| 7 \|
	\| 정보처리기사 \| 7 \|
	\| 수영 \| 7 \|
	\| 직장내괴롭힘 \| 0 \|
	\| 소아청소년과 \| 7 \|
	\| 재활·물리치료 \| 7 \|
	\| 부산 \| 7 \|
	\| 안과 \| 7 \|
	\| 런닝 \| 0 \|
	\| 지식재산권·IT \| 7 \|
	\| 사진·영상 \| 7 \|
	\| 세무사 자격증 \| 7 \|
	\| 축구·풋살 \| 7 \|
	\| 미주·중남미 \| 7 \|
	\| 중앙대학교 \| 7 \|
	\| 음향기기 \| 7 \|
	\| 한화 \| 7 \|
	\| 세무조사·불복 \| 7 \|
	\| 교통사고 과실 \| 0 \|
	\| 서울대학교 \| 7 \|
	\| 의료 \| 7 \|
	\| 식습관·식이요법 \| 7 \|
	\| 구조조정 \| 7 \|
	\| 법인세 \| 7 \|
	\| 기타 영양상담 \| 0 \|
	\| 자동차 \| 0 \|
	\| KAIST \| 7 \|
	\| 이비인후과 \| 0 \|
	\| 삼성 \| 0 \|
	\| 골프 \| 7 \|
	\| 경상 \| 7 \|
	\| 등산·클라이밍 \| 7 \|
	\| 기타 스포츠 \| 7 \|
	\| 성범죄 \| 7 \|
	\| GIST \| 7 \|
	\| 가드닝 \| 7 \|
	\| 한국외국어대학교 \| 7 \|
	\| PC 주변기기 \| 7 \|
	\| 기타 장치 \| 7 \|
	\| 연애·이성 \| 7 \|
	\| 취득세·등록세 \| 7 \|
	\| 산부인과 \| 7 \|
	\| 기타 세금상담 \| 0 \|
	\| 헬스 \| 7 \|
	\| 간호조무사 자격증 \| 7 \|
	\| 전기기사·기능사 \| 7 \|
	\| 오세아니아 \| 7 \|
	\| 무역 \| 7 \|
	\| 청소 \| 0 \|
	\| 공인중개사 자격증 \| 7 \|
	\| 중국 \| 0 \|
	\| 인테리어 \| 7 \|
	\| 동남아 \| 0 \|
	\| 군대 생활 \| 0 \|
	\| 영상가전 \| 0 \|
	\| 상속세 \| 0 \|
	\| LG \| 7 \|
	\| 부동산 \| 0 \|
	\| 기타 노무상담 \| 0 \|
	\| 웹툰·웹소설 \| 0 \|
	\| 뮤지컬·연극 \| 0 \|
	\| 네이버 \| 0 \|
	\| 근로계약 \| 7 \|
	\| 의료 보험 \| 7 \|
	\| 내과 \| 7 \|
	\| 세탁기·건조기 \| 7 \|
	\| 유럽·아프리카 \| 7 \|
	\| 기타 약료상담 \| 7 \|
	\| 드라마 \| 7 \|
	\| 반려동물 미용 \| 0 \|
	\| 스마트폰·태블릿 \| 7 \|
	\| 기타 의료상담 \| 7 \|
	\| 화학 \| 7 \|
	\| 청소기 \| 7 \|
	\| 테니스 \| 7 \|
	\| 충청 \| 7 \|
	\| 대학교 생활 \| 7 \|
	\| 야구 \| 7 \|
	\| 고등학교 생활 \| 7 \|
	\| 정형외과 \| 7 \|
	\| 재산 보험 \| 7 \|
	\| 화학공학 \| 0 \|
	\| 기타가전 \| 7 \|
	\| 문학 \| 7 \|
	\| 기타 법률상담 \| 7 \|
	\| 정신건강 \| 7 \|
	\| 크로스핏 \| 0 \|
	\| 요가·필라테스 \| 7 \|
	\| 스키·보드 \| 7 \|
	\| 보험설계사 자격증 \| 7 \|
	\| 해고·징계 \| 0 \|
	\| 휴일·휴가 \| 0 \|
	\| 성형외과 \| 7 \|
	\| 민사 \| 7 \|
	\| 음악 \| 7 \|
	\| 산업안전산업기사 \| 7 \|
	\| 유아교육 \| 7 \|
	\| 약 복용 \| 7 \|
	\| 주방가전 \| 7 \|
	\| 전기·전자 \| 7 \|
	\| 생활·미용가전 \| 7 \|
	\| 변호사 자격증 \| 0 \|
	\| 영화·애니 \| 7 \|
	\| 저축성 보험 \| 7 \|
	\| 증여세 \| 7 \|
	\| 회사 생활 \| 7 \|
	\| 기업·회사 \| 7 \|
	\| 자산관리 \| 7 \|
	\| 하이킹 \| 7 \|
	\| 연애·결혼 \| 0 \|
	\| 한의대학 \| 7 \|
	\| 지구과학·천문우주 \| 7 \|
	\| 롯데 \| 7 \|
	\| 산업재해 \| 7 \|
	\| 고려대학교 \| 7 \|
	\| 관세사 자격증 \| 7 \|
	\| 경제동향 \| 7 \|
	\| 회계자문 \| 7 \|
	\| 웨어러블 기기 \| 7 \|
	\| UNIST \| 7 \|
	\| SK \| 7 \|
	\| 주식·가상화폐 \| 7 \|
	\| 강원 \| 7 \|
	\| 부가가치세 \| 0 \|
	\| 신경외과 \| 7 \|
	\| 로스쿨 \| 7 \|
	\| 상해 보험 \| 7 \|
	\| 한양대학교 \| 7 \|
	\| 종합소득세 \| 7 \|
	\| 치아 관리 \| 7 \|
	\| 폭행·협박 \| 7 \|
	\| 의과대학 \| 7 \|

	### Training Hyperparameters
	- batch_size: (128, 128)
	- num_epochs: (2, 2)
	- max_steps: -1
	- sampling_strategy: oversampling
	- num_iterations: 80
	- body_learning_rate: (2e-05, 1e-05)
	- head_learning_rate: 0.01
	- loss: CosineSimilarityLoss
	- distance_metric: cosine_distance
	- margin: 0.25
	- end_to_end: False
	- use_amp: False
	- warmup_proportion: 0.1
	- seed: 42
	- eval_max_steps: -1
	- load_best_model_at_end: True

	### Training Results
	\| Epoch \| Step \| Training Loss \| Validation Loss \|
	\|:-------:\|:--------:\|:-------------:\|:---------------:\|
	\| 0.0005 \| 1 \| 0.1182 \| - \|
	\| 0.0275 \| 50 \| 0.1246 \| - \|
	\| 0.0549 \| 100 \| 0.0879 \| - \|
	\| 0.0824 \| 150 \| 0.0493 \| - \|
	\| 0.1099 \| 200 \| 0.0559 \| - \|
	\| 0.1374 \| 250 \| 0.0323 \| - \|
	\| 0.1648 \| 300 \| 0.025 \| - \|
	\| 0.1923 \| 350 \| 0.0296 \| - \|
	\| 0.2198 \| 400 \| 0.0198 \| - \|
	\| 0.2473 \| 450 \| 0.0163 \| - \|
	\| 0.2747 \| 500 \| 0.013 \| - \|
	\| 0.3022 \| 550 \| 0.013 \| - \|
	\| 0.3297 \| 600 \| 0.0193 \| - \|
	\| 0.3571 \| 650 \| 0.0161 \| - \|
	\| 0.3846 \| 700 \| 0.0163 \| - \|
	\| 0.4121 \| 750 \| 0.0095 \| - \|
	\| 0.4396 \| 800 \| 0.0055 \| - \|
	\| 0.4670 \| 850 \| 0.0178 \| - \|
	\| 0.4945 \| 900 \| 0.0094 \| - \|
	\| 0.5220 \| 950 \| 0.025 \| - \|
	\| 0.5495 \| 1000 \| 0.017 \| - \|
	\| 0.5769 \| 1050 \| 0.006 \| - \|
	\| 0.6044 \| 1100 \| 0.003 \| - \|
	\| 0.6319 \| 1150 \| 0.0108 \| - \|
	\| 0.6593 \| 1200 \| 0.0069 \| - \|
	\| 0.6868 \| 1250 \| 0.006 \| - \|
	\| 0.7143 \| 1300 \| 0.0025 \| - \|
	\| 0.7418 \| 1350 \| 0.0178 \| - \|
	\| 0.7692 \| 1400 \| 0.0042 \| - \|
	\| 0.7967 \| 1450 \| 0.0018 \| - \|
	\| 0.8242 \| 1500 \| 0.0046 \| - \|
	\| 0.8516 \| 1550 \| 0.0077 \| - \|
	\| 0.8791 \| 1600 \| 0.0041 \| - \|
	\| 0.9066 \| 1650 \| 0.0057 \| - \|
	\| 0.9341 \| 1700 \| 0.0115 \| - \|
	\| 0.9615 \| 1750 \| 0.0106 \| - \|
	\| 0.9890 \| 1800 \| 0.0086 \| - \|
	\| 1.0 \| 1820 \| - \| 0.0718 \|
	\| 1.0165 \| 1850 \| 0.0029 \| - \|
	\| 1.0440 \| 1900 \| 0.0078 \| - \|
	\| 1.0714 \| 1950 \| 0.0018 \| - \|
	\| 1.0989 \| 2000 \| 0.0066 \| - \|
	\| 1.1264 \| 2050 \| 0.0026 \| - \|
	\| 1.1538 \| 2100 \| 0.0014 \| - \|
	\| 1.1813 \| 2150 \| 0.0022 \| - \|
	\| 1.2088 \| 2200 \| 0.0038 \| - \|
	\| 1.2363 \| 2250 \| 0.0018 \| - \|
	\| 1.2637 \| 2300 \| 0.0169 \| - \|
	\| 1.2912 \| 2350 \| 0.0084 \| - \|
	\| 1.3187 \| 2400 \| 0.0022 \| - \|
	\| 1.3462 \| 2450 \| 0.0013 \| - \|
	\| 1.3736 \| 2500 \| 0.0015 \| - \|
	\| 1.4011 \| 2550 \| 0.0036 \| - \|
	\| 1.4286 \| 2600 \| 0.0012 \| - \|
	\| 1.4560 \| 2650 \| 0.0023 \| - \|
	\| 1.4835 \| 2700 \| 0.0013 \| - \|
	\| 1.5110 \| 2750 \| 0.0082 \| - \|
	\| 1.5385 \| 2800 \| 0.0073 \| - \|
	\| 1.5659 \| 2850 \| 0.0049 \| - \|
	\| 1.5934 \| 2900 \| 0.0013 \| - \|
	\| 1.6209 \| 2950 \| 0.0012 \| - \|
	\| 1.6484 \| 3000 \| 0.002 \| - \|
	\| 1.6758 \| 3050 \| 0.0038 \| - \|
	\| 1.7033 \| 3100 \| 0.0056 \| - \|
	\| 1.7308 \| 3150 \| 0.001 \| - \|
	\| 1.7582 \| 3200 \| 0.004 \| - \|
	\| 1.7857 \| 3250 \| 0.0012 \| - \|
	\| 1.8132 \| 3300 \| 0.0063 \| - \|
	\| 1.8407 \| 3350 \| 0.0016 \| - \|
	\| 1.8681 \| 3400 \| 0.0016 \| - \|
	\| 1.8956 \| 3450 \| 0.0063 \| - \|
	\| 1.9231 \| 3500 \| 0.0013 \| - \|
	\| 1.9505 \| 3550 \| 0.0033 \| - \|
	\| 1.9780 \| 3600 \| 0.0025 \| - \|
	\| 2.0 \| 3640 \| - \| 0.0775 \|

	* The bold row denotes the saved checkpoint.
	### Framework Versions
	- Python: 3.10.12
	- SetFit: 1.0.3
	- Sentence Transformers: 2.7.0
	- Transformers: 4.40.2
	- PyTorch: 2.0.1
	- Datasets: 2.19.1
	- Tokenizers: 0.19.1

	## Citation

	### BibTeX
	```bibtex
	@article{https://doi.org/10.48550/arxiv.2209.11055,
	doi = {10.48550/ARXIV.2209.11055},
	url = {https://arxiv.org/abs/2209.11055},
	author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
	keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
	title = {Efficient Few-Shot Learning Without Prompts},
	publisher = {arXiv},
	year = {2022},
	copyright = {Creative Commons Attribution 4.0 International}
	}
	```

	<!--
	## Glossary

	Clearly define terms in order to be accessible across audiences.
	-->

	<!--
	## Model Card Authors

	Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.
	-->

	<!--
	## Model Card Contact

	Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.
	-->