samta-kamboj commited on
Commit
0ca8c4d
·
verified ·
1 Parent(s): 474407e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -30
README.md CHANGED
@@ -27,7 +27,7 @@ We hope this extensive release will accelerate research in Arabic NLP, and enabl
27
 
28
  ## Jais Family Details
29
 
30
- - **Developed by:** Core42 (Inception), Cerebras Systems.
31
  - **Language(s):** (NLP): Arabic (MSA) and English.
32
  - **Input:** Text only data.
33
  - **Output:** Model generates text.
@@ -37,19 +37,19 @@ We hope this extensive release will accelerate research in Arabic NLP, and enabl
37
 
38
  | **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
39
  |:---------------------|:--------|:-------|:-------|
40
- | [jais-family-30b-16k](https://huggingface.co/core42/jais-family-30b-16k) | [Jais-family-30b-16k-chat](https://huggingface.co/core42/jais-family-30b-16k-chat) | 30B | 16,384 |
41
- | [jais-family-30b-8k](https://huggingface.co/core42/jais-family-30b-8k) | [Jais-family-30b-8k-chat](https://huggingface.co/core42/jais-family-30b-8k-chat) | 30B | 8,192 |
42
- | [jais-family-13b ](https://huggingface.co/core42/jais-family-13b) | [Jais-family-13b-chat](https://huggingface.co/core42/jais-family-13b-chat) | 13B | 2,048 |
43
- | [jais-family-6p7b](https://huggingface.co/core42/jais-family-6p7b) | [Jais-family-6p7b-chat](https://huggingface.co/core42/jais-family-6p7b-chat) | 6.7B | 2,048 |
44
- | [jais-family-2p7b](https://huggingface.co/core42/jais-family-2p7b) | [Jais-family-2p7b-chat](https://huggingface.co/core42/jais-family-2p7b-chat) | 2.7B | 2,048 |
45
- | [jais-family-1p3b](https://huggingface.co/core42/jais-family-1p3b) | [Jais-family-1p3b-chat](https://huggingface.co/core42/jais-family-1p3b-chat) | 1.3B | 2,048 |
46
- | [jais-family-590m](https://huggingface.co/core42/jais-family-590m) | [Jais-family-590m-chat](https://huggingface.co/core42/jais-family-590m-chat) | 590M | 2,048 |
47
 
48
  | **Adapted pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
49
  |:---------------------|:--------|:-------|:-------|
50
- | [jais-adapted-70b](https://huggingface.co/core42/jais-adapted-70b) | [Jais-adapted-70b-chat](https://huggingface.co/core42/jais-adapted-70b-chat) | 70B | 4,096 |
51
- | [jais-adapted-13b](https://huggingface.co/core42/jais-adapted-13b) | [Jais-adapted-13b-chat](https://huggingface.co/core42/jais-adapted-13b-chat) | 13B | 4,096 |
52
- | [jais-adapted-7b](https://huggingface.co/core42/jais-adapted-7b) | [Jais-adapted-7b-chat](https://huggingface.co/core42/jais-adapted-7b-chat) | 7B | 4,096 |
53
 
54
  ### Model Architecture:
55
  <a name="model-architecture"></a>
@@ -72,10 +72,10 @@ Below is sample code to use the model. Note that the model requires a custom mod
72
  import torch
73
  from transformers import AutoTokenizer, AutoModelForCausalLM
74
 
75
- model_path = "core42/jais-family-13b-chat"
76
 
77
- prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Core42' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:\n### Input: [|Human|] {Question}\n[|AI|]\n### Response :"
78
- prompt_ar = "### Instruction:اسمك \"جيس\" وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Core42 في الإمارات. أنت مساعد مفيد ومحترم وصادق. أجب دائمًا بأكبر قدر ممكن من المساعدة، مع الحفاظ على البقاء أمناً. أكمل المحادثة بين [|Human|] و[|AI|] :\n### Input:[|Human|] {Question}\n[|AI|]\n### Response :"
79
 
80
  device = "cuda" if torch.cuda.is_available() else "cpu"
81
 
@@ -136,16 +136,16 @@ We extensively preprocess and deduplicate the training data. For Arabic, we used
136
 
137
  | **Pre-trained model** | **English data (tokens)** | **Arabic data (tokens)** | **Code data (tokens)** | **Total data (tokens)** |
138
  |-------------------------|---------------------------|--------------------------|------------------------|------------------------|
139
- | [jais-family-30b-16k](https://huggingface.co/core42/jais-family-30b-16k) | 980B | 490B | 196B | 1666B |
140
- | [jais-family-30b-8k](https://huggingface.co/core42/jais-family-30b-8k) | 882B | 441B | 177B | 1500B |
141
- | [jais-family-13b ](https://huggingface.co/core42/jais-family-13b) | 283B | 141B | 56B | 480B |
142
- | [jais-family-6p7b](https://huggingface.co/core42/jais-family-6p7b) | 283B | 141B | 56B | 480B |
143
- | [jais-family-2p7b](https://huggingface.co/core42/jais-family-2p7b) | 283B | 141B | 56B | 480B |
144
- | [jais-family-1p3b](https://huggingface.co/core42/jais-family-1p3b) | 283B | 141B | 56B | 480B |
145
- | [jais-family-590m](https://huggingface.co/core42/jais-family-590m) | 283B | 141B | 56B | 480B |
146
- | [jais-adapted-70b](https://huggingface.co/core42/jais-adapted-70b) | 33B | 334B | 4B | 371B |
147
- | [jais-adapted-13b](https://huggingface.co/core42/jais-adapted-13b) | 127B | 140B | 13B | 280B |
148
- | [jais-adapted-7b](https://huggingface.co/core42/jais-adapted-7b) | 18B | 19B | 2B | 39B |
149
 
150
  ### Finetuning data
151
 
@@ -279,14 +279,14 @@ English prompts were translated to Arabic by our in-house linguists.
279
  In the following, we compare the models in this release of the jais family against previously released versions:
280
 
281
  <p align="center">
282
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/jais.png" alt="Jais-adapted GPT-4">
283
  </p>
284
  <p align="center">
285
  <em>GPT-4-as-a-judge evaluation of Jais in Arabic and English. Jais family models are significantly better than previous Jais at generations in both languages. </em>
286
  </p>
287
 
288
  <p align="center">
289
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/jais-adapted.png" alt="Jais-adapted GPT-4">
290
  </p>
291
  <p align="center">
292
  <em>GPT-4-as-a-judge evaluation of adapted Jais in Arabic and English. The generation quality of Arabic is significantly enhanced, while achieving improvement in English when compared to Llama-2 instruct. </em>
@@ -295,7 +295,7 @@ In the following, we compare the models in this release of the jais family again
295
  Besides pairwise comparison, we also perform MT-bench style single-answer grading on a scale of 1 to 10.
296
 
297
  <p align="center">
298
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/mt_bench.png" alt="MT-bench">
299
  </p>
300
  <p align="center">
301
  <em>MT-bench style single-answer grading evaluation of Jais and adapted Jais in Arabic and English. Comparisons are made between select corresponding models from earlier releases. The quality ratings of responses are generally improved, with significant enhancements in Arabic.</em>
@@ -345,7 +345,7 @@ The following are some example scenarios where the model should not be used.
345
 
346
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
347
 
348
- The Jais family is trained on publicly available data which was in part curated by Core42. We have employed different techniques to reduce bias in the model. While efforts have been made to minimize biases, it is likely that the model, as with all LLM models, will exhibit some bias.
349
 
350
  The fine-tuned variants are trained as an AI assistant for Arabic and English speakers. Chat models are limited to produce responses for queries in these two languages and may not produce appropriate responses to other language queries.
351
 
@@ -372,8 +372,8 @@ Through this release, we aim to make LLMs more accessible to Arabic NLP research
372
 
373
  @article{jaisfamilymodelcard,
374
  title={Jais Family Model Card},
375
- author={Core42 (Inception)},
376
  year={2024},
377
- url = {https://huggingface.co/core42/jais-family-30b-16k-chat/blob/main/README.md}
378
  }
379
  ```
 
27
 
28
  ## Jais Family Details
29
 
30
+ - **Developed by:** Inception, Cerebras Systems.
31
  - **Language(s):** (NLP): Arabic (MSA) and English.
32
  - **Input:** Text only data.
33
  - **Output:** Model generates text.
 
37
 
38
  | **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
39
  |:---------------------|:--------|:-------|:-------|
40
+ | [jais-family-30b-16k](https://huggingface.co/inceptionai/jais-family-30b-16k) | [Jais-family-30b-16k-chat](https://huggingface.co/inceptionai/jais-family-30b-16k-chat) | 30B | 16,384 |
41
+ | [jais-family-30b-8k](https://huggingface.co/inceptionai/jais-family-30b-8k) | [Jais-family-30b-8k-chat](https://huggingface.co/inceptionai/jais-family-30b-8k-chat) | 30B | 8,192 |
42
+ | [jais-family-13b ](https://huggingface.co/inceptionai/jais-family-13b) | [Jais-family-13b-chat](https://huggingface.co/inceptionai/jais-family-13b-chat) | 13B | 2,048 |
43
+ | [jais-family-6p7b](https://huggingface.co/inceptionai/jais-family-6p7b) | [Jais-family-6p7b-chat](https://huggingface.co/inceptionai/jais-family-6p7b-chat) | 6.7B | 2,048 |
44
+ | [jais-family-2p7b](https://huggingface.co/inceptionai/jais-family-2p7b) | [Jais-family-2p7b-chat](https://huggingface.co/inceptionai/jais-family-2p7b-chat) | 2.7B | 2,048 |
45
+ | [jais-family-1p3b](https://huggingface.co/inceptionai/jais-family-1p3b) | [Jais-family-1p3b-chat](https://huggingface.co/inceptionai/jais-family-1p3b-chat) | 1.3B | 2,048 |
46
+ | [jais-family-590m](https://huggingface.co/inceptionai/jais-family-590m) | [Jais-family-590m-chat](https://huggingface.co/inceptionai/jais-family-590m-chat) | 590M | 2,048 |
47
 
48
  | **Adapted pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
49
  |:---------------------|:--------|:-------|:-------|
50
+ | [jais-adapted-70b](https://huggingface.co/inceptionai/jais-adapted-70b) | [Jais-adapted-70b-chat](https://huggingface.co/inceptionai/jais-adapted-70b-chat) | 70B | 4,096 |
51
+ | [jais-adapted-13b](https://huggingface.co/inceptionai/jais-adapted-13b) | [Jais-adapted-13b-chat](https://huggingface.co/inceptionai/jais-adapted-13b-chat) | 13B | 4,096 |
52
+ | [jais-adapted-7b](https://huggingface.co/inceptionai/jais-adapted-7b) | [Jais-adapted-7b-chat](https://huggingface.co/inceptionai/jais-adapted-7b-chat) | 7B | 4,096 |
53
 
54
  ### Model Architecture:
55
  <a name="model-architecture"></a>
 
72
  import torch
73
  from transformers import AutoTokenizer, AutoModelForCausalLM
74
 
75
+ model_path = "inceptionai/jais-family-13b-chat"
76
 
77
+ prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Inception' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:\n### Input: [|Human|] {Question}\n[|AI|]\n### Response :"
78
+ prompt_ar = "### Instruction:اسمك \"جيس\" وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Inception في الإمارات. أنت مساعد مفيد ومحترم وصادق. أجب دائمًا بأكبر قدر ممكن من المساعدة، مع الحفاظ على البقاء أمناً. أكمل المحادثة بين [|Human|] و[|AI|] :\n### Input:[|Human|] {Question}\n[|AI|]\n### Response :"
79
 
80
  device = "cuda" if torch.cuda.is_available() else "cpu"
81
 
 
136
 
137
  | **Pre-trained model** | **English data (tokens)** | **Arabic data (tokens)** | **Code data (tokens)** | **Total data (tokens)** |
138
  |-------------------------|---------------------------|--------------------------|------------------------|------------------------|
139
+ | [jais-family-30b-16k](https://huggingface.co/inceptionai/jais-family-30b-16k) | 980B | 490B | 196B | 1666B |
140
+ | [jais-family-30b-8k](https://huggingface.co/inceptionai/jais-family-30b-8k) | 882B | 441B | 177B | 1500B |
141
+ | [jais-family-13b ](https://huggingface.co/inceptionai/jais-family-13b) | 283B | 141B | 56B | 480B |
142
+ | [jais-family-6p7b](https://huggingface.co/inceptionai/jais-family-6p7b) | 283B | 141B | 56B | 480B |
143
+ | [jais-family-2p7b](https://huggingface.co/inceptionai/jais-family-2p7b) | 283B | 141B | 56B | 480B |
144
+ | [jais-family-1p3b](https://huggingface.co/inceptionai/jais-family-1p3b) | 283B | 141B | 56B | 480B |
145
+ | [jais-family-590m](https://huggingface.co/inceptionai/jais-family-590m) | 283B | 141B | 56B | 480B |
146
+ | [jais-adapted-70b](https://huggingface.co/inceptionai/jais-adapted-70b) | 33B | 334B | 4B | 371B |
147
+ | [jais-adapted-13b](https://huggingface.co/inceptionai/jais-adapted-13b) | 127B | 140B | 13B | 280B |
148
+ | [jais-adapted-7b](https://huggingface.co/inceptionai/jais-adapted-7b) | 18B | 19B | 2B | 39B |
149
 
150
  ### Finetuning data
151
 
 
279
  In the following, we compare the models in this release of the jais family against previously released versions:
280
 
281
  <p align="center">
282
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/jais.png" alt="Jais-adapted GPT-4">
283
  </p>
284
  <p align="center">
285
  <em>GPT-4-as-a-judge evaluation of Jais in Arabic and English. Jais family models are significantly better than previous Jais at generations in both languages. </em>
286
  </p>
287
 
288
  <p align="center">
289
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/jais-adapted.png" alt="Jais-adapted GPT-4">
290
  </p>
291
  <p align="center">
292
  <em>GPT-4-as-a-judge evaluation of adapted Jais in Arabic and English. The generation quality of Arabic is significantly enhanced, while achieving improvement in English when compared to Llama-2 instruct. </em>
 
295
  Besides pairwise comparison, we also perform MT-bench style single-answer grading on a scale of 1 to 10.
296
 
297
  <p align="center">
298
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/mt_bench.png" alt="MT-bench">
299
  </p>
300
  <p align="center">
301
  <em>MT-bench style single-answer grading evaluation of Jais and adapted Jais in Arabic and English. Comparisons are made between select corresponding models from earlier releases. The quality ratings of responses are generally improved, with significant enhancements in Arabic.</em>
 
345
 
346
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
347
 
348
+ The Jais family is trained on publicly available data which was in part curated by Inception. We have employed different techniques to reduce bias in the model. While efforts have been made to minimize biases, it is likely that the model, as with all LLM models, will exhibit some bias.
349
 
350
  The fine-tuned variants are trained as an AI assistant for Arabic and English speakers. Chat models are limited to produce responses for queries in these two languages and may not produce appropriate responses to other language queries.
351
 
 
372
 
373
  @article{jaisfamilymodelcard,
374
  title={Jais Family Model Card},
375
+ author={Inception},
376
  year={2024},
377
+ url = {https://huggingface.co/inceptionai/jais-family-30b-16k-chat/blob/main/README.md}
378
  }
379
  ```