update README
Browse files
README.md
CHANGED
@@ -1,7 +1,12 @@
|
|
1 |
-
|
2 |
-
|
|
|
|
|
|
|
3 |
|
4 |
-
|
|
|
|
|
5 |
|
6 |
### Resource Considerations
|
7 |
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
datasets:
|
4 |
+
- databricks/dolly-15k
|
5 |
+
---
|
6 |
|
7 |
+
## Flan-UL2-Dolly - Building a commercially viable LLM
|
8 |
+
|
9 |
+
This [Github repository](https://github.com/ConiferLabsWA/flan-ul2-dolly) contains code for leveraging the [Dolly 15K](https://github.com/databrickslabs/dolly/tree/master/data) dataset [released by Databricks](https://github.com/databrickslabs/dolly/tree/master/data) to fine tune the [Flan-UL2](https://huggingface.co/google/flan-ul2) model, leveraging recent advances in instruction tuning. Flan-UL2 has been shown to outperform Flan-T5 XXL on a number of metrics and has a 4x improvement in receptive field (2048 vs 512 tokens). Additionally, both the Flan-UL2 model and the Dolly 15K dataset have the significant advantage of a commercially viable license.
|
10 |
|
11 |
### Resource Considerations
|
12 |
|