Upload Qwen2ForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "airev-ai/Amal-70b-v2",
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 29568,
+  "max_position_embeddings": 131072,
+  "max_window_layers": 28,
+  "model_type": "qwen2",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.44.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": 151643,
+  "max_new_tokens": 2048,
+  "transformers_version": "4.44.0"
+}

model-00001-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0176936410ae77d886b3e1ec011b0d7de259fd6f78113232a18375958855a090
+size 2491416712

model-00002-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51de4abdc7b983a1fafed75c6011d617fc0065d36c8d5049978867e64181074a
+size 1923164168

model-00003-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8d12c8a9a7781399ef7c37ddbb130f07b3090b84ad743398cffeeabcfd58a34
+size 1889588696

model-00004-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3bf0a32d7a39abfcb332291a98a72b48b252e188e8fa6dc39ffdade1aba4b19
+size 1755370856

model-00005-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3d915ba0edacb72d2f1897bc92872552241f07276321373d98036babdbf7a44
+size 1755370856

model-00006-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0818ef13275a04ca6480a0ce1242e19e0a50db69988c3275cb13ca2635b69697
+size 1755370856

model-00007-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76a9155d9ef59f35acaaf027666ec140a0a5ee21fd0cb91b52d139d95b80b0ea
+size 1755370856

model-00008-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1e1339c8d3357b8a985a7712b0977797538a486a419a8562abd32b112f84c31
+size 1755370856

model-00009-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a5bf658c9cbbd9600af528fc1961f208a7d3e05563faaecc7f470a529e4e78f
+size 1755370856

model-00010-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99eec3848be21a3338f352d7a5109f6c00cb445df5d090e36b8136a7b4de577a
+size 1755370856

model-00011-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8df42ac040eb8ba1c378dec8369a14bcfc67047fceef14468e3a3d1bc380e3b
+size 1755370840

model-00012-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6c362bf5cc61d5ce7c728a584e7041384ffd0651fc5e570b33c5a8041a3c8f0
+size 1755370864

model-00013-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06ad165506265870e1aad8a92d80b076c346d05c0f611adb12d623bf33c02537
+size 1755370864

model-00014-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bdba74e2be8a4ed01b7c822be67d8e5b35bdcbd9bb326bbda93bb010bd46f36
+size 1755370864

model-00015-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1612f938995af2388431bed0281353218b969617f040a529f5804817865086c1
+size 1755370864

model-00016-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12b5479358c6cedc70085c786d68511cdd7d11cc3a084d452665fd3c5cc40706
+size 1755370864

model-00017-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f21eff1433b14ed323f0fc95a57285d4bee7b732c459afde37e41619c11ea4ab
+size 1755370864

model-00018-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58f6b6eccfbe6657894de38d3e5e50bbae9c894975393e501274a3cce04d2fcd
+size 1755370864

model-00019-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8507a9da28253562b30f4377491a4d422e4b8f2511c069d8e42fff08a711361
+size 1755370864

model-00020-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac47c339c0bdf0f2e7cdfd60d834f753090fde8b0051ebf190f31ce3e64e0df4
+size 1755370864

model-00021-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:839a2faafe65dcd928123edc56b2a5e7a0288f9703fb66a189f1c7c2060b5fbc
+size 1755370864

model-00022-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:574e8600f26402cc17274f323a7426693e9576e304acade7edb09f559dda12c7
+size 1755370864

model-00023-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3050b3c026221174bf783e1a5854d39ac7c80e2fca94904750e4fe4863ed912c
+size 1755370864

model-00024-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d18353b08c0f992e70ece81158677d9adcf3c80db72639092b0882e31582b033
+size 1755370864

model-00025-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:886533a78392b1ec2d29c7a2595c0c1c6ca56ed4b3ea86ebece92aee65cbc12a
+size 1755370864

model-00026-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a199474ef30dfd86dc5afaf46944c2d0b4d6b55e7b4a303628145e8766d56b4
+size 1755370864

model-00027-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e38ebf77d67f9765f0387f3005f3a70c9d045730b6e1de963e60256276f4967d
+size 1755370864

model-00028-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b7214dd738d6c4146ee625e49c83c2e00ba71e84ab5fdf2dc298a52e27b592f
+size 1755370864

model-00029-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bf61bb6948be50bc28aaec45769abcc50f4c1382bbb2330245a17d0922bfaf9
+size 1755370864

model-00030-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:109e8f49752beb94acb10dedfd6ebfa7cf36749f476e2d3bcad385877db5c368
+size 1755370864

model-00031-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97b4da05215a01acba4034bf5944405793ff3f938bf28892a38ce9f9fe508768
+size 1755370864

model-00032-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06029dc04b817c16e5adbf09cd4ef9f15c3b6e6a10f52b1d7cedc4f32ad722b2
+size 1755370864

model-00033-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6da3c50d553e2275ac7511105efbdc4d28d666694c54ccb30be60dec5aa7306d
+size 1755370864

model-00034-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6b21c8dfb90111127bebe493ab8e362a22268f8c6db767872c58043496c5ebc
+size 1755370864

model-00035-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e1d9ef678646f30b869c77b0885855a38863accba06b1aa9b268918e2437fcb
+size 1755370864

model-00036-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:066aff6806c7b2631be18a124267186bb5f92e6f4f9aab3bfd5bcc28c0c4da69
+size 1755370864

model-00037-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dc0866cd8a653c014e3caa226df20140eb4852919fe5c7638c8d219458f1293
+size 1755370864

model-00038-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc5cff032f5cdeafc0c958282c15c65d02cda68177e8d1e9a964eebe893cdcc9
+size 1755370864

model-00039-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:473b8e65b40218912339a195c5ac4821f2285deaaa1a16c33830f6617b49d158
+size 1755370864

model-00040-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dfb895758ecbb76886cb7fcd4e72d301d02e40edd90de8fdb55e5f7b84e023b
+size 1755370864

model-00041-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f317ffa20d4dfc3aae9e477042467dfdc557990ebbb8fde4ef73cd4d866b31ee
+size 1755370864

model-00042-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2db7a3781b049ca1a95860eb88bdd6b4a38da0b1c3b8f5db4f1389a7b7d45d23
+size 1755370864

model-00043-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fee4879d6aa6dc189ce01b0b635754649cbb9445cbd037c75890b34fe584201a
+size 1755370864

model-00044-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e06812cb95364ee694244eb5df9b5c319ec70dd8541fef2875a18328a1e5c597
+size 1755370864

model-00045-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae562e2c1af4842f2d7c025aae79fba31462d320ac864ab8cc48f48f104467bd
+size 1755370864

model-00046-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62e3738f97a337618b217fbfd841437ced9b24e6271eb31277597aeb19735b7e
+size 1755370864

model-00047-of-00082.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:215c16a49425a1426b773d198ad430be6ea0a4554255f99d40565e6941652151
+size 1755370864