commit from user

README.md

@@ -0,0 +1,170 @@
+---
+language: en
+datasets: COCA
+---
+# docusco-bert
+
+## Model description
+
+**docusco-bert** is a fine-tuned BERT model that is ready to use for **token classification**. The model was trained on data from the Corpus of Contemporary American English ([COCA](https://www.english-corpora.org/coca/)) and classifies tokens and token sequences according to a system developed for the [DocuScope](https://www.cmu.edu/dietrich/english/research-and-publications/docuscope.html) dictionary-based tagger. Descriptions of the categories are included in a table below. 
+
+## About DocuScope
+DocuScope is a dicitonary-based tagger that has been developed at Carnegie Mellon University by David Kaufer and Suguru Ishizaki since the early 2000s. Its categories are rhetorical in their orientation (as opposed to part-of-speech tags, for example, which are morphosyntactic).
+
+## Intended uses & limitations
+
+#### How to use
+
+The model was trained on data with tags formatted using [IOB](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)), like those used in common tasks like Named Entity Recogition (NER). Thus, you can use this model with a Transformers NER *pipeline*.
+
+```python
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+from transformers import pipeline
+
+tokenizer = AutoTokenizer.from_pretrained("browndw/docusco-bert")
+model = AutoModelForTokenClassification.from_pretrained("browndw/docusco-bert")
+
+nlp = pipeline("ner", model=model, tokenizer=tokenizer)
+example = "My name is Wolfgang and I live in Berlin"
+
+ner_results = nlp(example)
+print(ner_results)
+```
+
+#### Limitations and bias
+
+This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains. Furthermore, the model occassionally tags subword tokens as entities and post-processing of results may be necessary to handle those cases. 
+
+## Training data
+
+This model was fine-tuned on data from the Corpus of Contemporary American English ([COCA](https://www.english-corpora.org/coca/)). The training data contain 1500 randomly sampled texts from each of 5 text-types: Academic, Fiction, Magazine, News, and Spoken.
+
+#### # of texts/chunks/tokens per dataset
+Dataset |Texts |Chunks |Tokens
+-|-|-|-
+Train |7500 |1,167,584 |32,203,828
+Test |500 |58,117 |1,567,997
+
+## Training procedure
+
+This model was trained on a single 2.3 GHz Dual-Core Intel Core i5 with recommended hyperparameters from the [original BERT paper](https://arxiv.org/pdf/1810.04805). 
+
+## Eval results
+### Overall
+metric|test
+-|-
+f1 |66.3
+accuracy |74.7
+
+### By category
+precision|recall|f1-score|support
+-|-|-|-
+AcademicTerms|0.69|0.70|0.69|54204
+AcademicWritingMoves|0.31|0.40|0.35|2860
+Character|0.68|0.70|0.69|86213
+Citation|0.61|0.47|0.53|4798
+CitationAuthority|0.48|0.39|0.43|1871
+CitationHedged|0.58|0.81|0.67|135
+ConfidenceHedged|0.65|0.74|0.69|8209
+ConfidenceHigh|0.51|0.61|0.56|8755
+ConfidenceLow|0.88|0.03|0.07|202
+Contingent|0.58|0.60|0.59|6802
+Description|0.56|0.62|0.59|98697
+Facilitate|0.57|0.54|0.55|3597
+FirstPerson|0.61|0.70|0.65|21285
+ForceStressed|0.55|0.59|0.57|29420
+Future|0.53|0.60|0.56|7253
+InformationChange|0.56|0.61|0.59|7427
+InformationChangeNegative|0.52|0.50|0.51|1338
+InformationChangePositive|0.53|0.48|0.51|2161
+InformationExposition|0.75|0.77|0.76|43144
+InformationPlace|0.78|0.84|0.81|14058
+InformationReportVerbs|0.64|0.69|0.66|8418
+InformationStates|0.69|0.74|0.71|11195
+InformationTopics|0.61|0.65|0.63|30214
+Inquiry|0.37|0.45|0.41|7109
+Interactive|0.59|0.64|0.61|32523
+MetadiscourseCohesive|0.89|0.91|0.90|17301
+MetadiscourseInteractive|0.40|0.48|0.44|4245
+Narrative|0.63|0.69|0.66|91799
+Negative|0.55|0.60|0.58|49234
+Positive|0.49|0.57|0.53|32228
+PublicTerms|0.63|0.64|0.63|18302
+Reasoning|0.71|0.75|0.73|16178
+Responsibility|0.51|0.43|0.47|1837
+Strategic|0.47|0.52|0.50|16889
+SyntacticComplexity|0.78|0.81|0.80|156361
+Uncertainty|0.36|0.32|0.34|2680
+Updates|0.44|0.39|0.41|6036
+-|-|-|-
+micro|avg|0.64|0.68|0.66|904978
+macro|avg|0.59|0.59|0.58|904978
+weighted|avg|0.64|0.68|0.66|904978
+
+
+## DocuScope Category Descriptions
+
+Category (Cluster)|Description|Examples
+-|-|-
+Academic Terms|Abstract, rare, specialized, or disciplinary-specific terms that are indicative of informationally dense writing|*market price*, *storage capacity*, *regulatory*, *distribution*
+Academic Writing Moves|Phrases and terms that indicate academic writing moves, which are common in research genres and are derived from the work of Swales (1981) and Cotos et al. (2015, 2017)|*in the first section*, *the problem is that*, *payment methodology*, *point of contention*
+Character|References multiple dimensions of a character or human being as a social agent, both individual and collective|*Pauline*, *her*, *personnel*, *representatives*
+Citation|Language that indicates the attribution of information to, or citation of, another source.|*according to*, *is proposing that*, *quotes from*
+Citation Authorized|Referencing the citation of another source that is represented as true and not arguable|*confirm that*, *provide evidence*, *common sense*
+Citation Hedged|Referencing the citation of another source that is presented as arguable|*suggest that*, *just one opinion*
+Confidence Hedged|Referencing language that presents a claim as uncertain|*tends to get*, *maybe*, *it seems that*
+Confidence High|Referencing language that presents a claim with certainty|*most likely*, *ensure that*, *know that*, *obviously*
+Confidence Low|Referencing language that presents a claim as extremely unlikely|*unlikely*, *out of the question*, *impossible*
+Contingent|Referencing contingency, typically contingency in the world, rather than contingency in one's knowledge|*subject to*, *if possible*, *just in case*, *hypothetically*
+Description|Language that evokes sights, sounds, smells, touches and tastes, as well as scenes and objects|*stay quiet*, *gas-fired*, *solar panels*, *soft*, *on my desk*
+Facilitate|Language that enables or directs one through specific tasks and actions|*let me*, *worth a try*, *I would suggest*
+First Person|This cluster captures first person.|*I*, *as soon as I*, *we have been*
+Force Stressed|Language that is forceful and stressed, often using emphatics, comparative forms, or superlative forms|*really good*, *the sooner the better*, *necessary*
+Future|Referencing future actions, states, or desires|*will be*, *hope to*, *expected changes*
+Information Change|Referencing changes of information, particularly changes that are more neutral|*changes*, *revised*, *growth*, *modification to*
+Information Change Negative|Referencing negative change|*going downhill*, *slow erosion*, *get worse*
+Information Change Positive|Referencing positive change|*improving*, *accrued interest*, *boost morale*
+Information Exposition|Information in the form of expository devices, or language that describes or explains, frequently in regards to quantities and comparisons|*final amount*, *several*, *three*, *compare*, *80%*
+Information Place|Language designating places|*the city*, *surrounding areas*, *Houston*, *home*
+Information Report Verbs|Informational verbs and verb phrases of reporting|*report*, *posted*, *release*, *point out*
+Information States|Referencing information states, or states of being|*is*, *are*, *existing*, *been*
+Information Topics|Referencing topics, usually nominal subjects or objects, that indicate the “aboutness” of a text|*time*, *money*, *stock price*, *phone interview*
+Inquiry|Referencing inquiry, or language that points to some kind of inquiry or investigation|*find out*, *let me know if you have any questions*, *wondering if*
+Interactive|Addresses from the author to the reader or from persons in the text to other persons. The address comes in the language of everyday conversation, colloquy, exchange, questions, attention-getters, feedback, interactive genre markers, and the use of the second person.|*can you*, *thank you for*, *please see*, *sounds good to me*
+Metadiscourse Cohesive|The use of words to build cohesive markers that help the reader navigate the text and signal linkages in the text, which are often additive or contrastive|*or*, *but*, *also*, *on the other hand*, *notwithstanding*, *that being said*
+Metadiscourse Interactive|The use of words to build cohesive markers that interact with the reader|*I agree*, *let’s talk*, *by the way*
+Narrative|Language that involves people, description, and events extending in time|*today*, *tomorrow*, *during the*, *this weekend*
+Negative|Referencing dimensions of negativity, including negative acts, emotions, relations, and values|*does not*, *sorry for*, *problems*, *confusion*
+Positive|Referencing dimensions of positivity, including actions, emotions, relations, and values|*thanks*, *approval*, *agreement*, *looks good*
+Public Terms|Referencing public terms, concepts from public language, media, the language of authority, institutions, and responsibility|*discussion*, *amendment*, *corporation*, *authority*, *settlement*
+Reasoning|Language that has a reasoning focus, supporting inferences about cause, consequence, generalization, concession, and linear inference either from premise to conclusion or conclusion to premise|*because*, *therefore*, *analysis*, *even if*, *as a result*, *indicating that*
+Responsibility|Referencing the language of responsibility|*supposed to*, *requirements*, *obligations*
+Strategic|This dimension is active when the text structures strategies activism, advantage-seeking, game-playing cognition, plans, and goal-seeking.|*plan*, *trying to*, *strategy*, *decision*, *coordinate*, *look at the*
+Syntactic Complexity|The features in this category are often what are called “function words,” like determiners and prepositions.|*the*, *to*, *for*, *in*, *a lot of*
+Uncertainty|References uncertainty, when confidence levels are unknown|*kind of*, *I have no idea*, *for some reason*
+Updates|References updates that anticipate someone searching for information and receiving it|*already*, *a new*, *now that*, *here are some*
+
+
+
+### BibTeX entry and citation info
+
+```
+@article{DBLP:journals/corr/abs-1810-04805,
+  author    = {Jacob Devlin and
+               Ming{-}Wei Chang and
+               Kenton Lee and
+               Kristina Toutanova},
+  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
+               Understanding},
+  journal   = {CoRR},
+  volume    = {abs/1810.04805},
+  year      = {2018},
+  url       = {http://arxiv.org/abs/1810.04805},
+  archivePrefix = {arXiv},
+  eprint    = {1810.04805},
+  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
+  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
+  bibsource = {dblp computer science bibliography, https://dblp.org}
+}
+```
+

config.json

@@ -0,0 +1,180 @@
+{
+  "_name_or_path": "bert-base-cased",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "B-Description",
+    "1": "B-Responsibility",
+    "2": "I-AcademicWritingMoves",
+    "3": "B-Strategic",
+    "4": "I-Contingent",
+    "5": "I-InformationStates",
+    "6": "I-Interactive",
+    "7": "B-SyntacticComplexity",
+    "8": "B-InformationChange",
+    "9": "I-Description",
+    "10": "I-Narrative",
+    "11": "B-InformationTopics",
+    "12": "B-MetadiscourseInteractive",
+    "13": "I-InformationPlace",
+    "14": "I-Responsibility",
+    "15": "I-Reasoning",
+    "16": "B-InformationExposition",
+    "17": "B-ForceStressed",
+    "18": "B-ConfidenceHedged",
+    "19": "B-Character",
+    "20": "B-Updates",
+    "21": "I-InformationReportVerbs",
+    "22": "B-InformationChangePositive",
+    "23": "B-PublicTerms",
+    "24": "I-MetadiscourseCohesive",
+    "25": "O",
+    "26": "B-AcademicTerms",
+    "27": "I-MetadiscourseInteractive",
+    "28": "I-Updates",
+    "29": "I-Negative",
+    "30": "B-InformationPlace",
+    "31": "B-Interactive",
+    "32": "I-AcademicTerms",
+    "33": "I-CitationAuthority",
+    "34": "I-Citation",
+    "35": "B-Narrative",
+    "36": "I-PublicTerms",
+    "37": "B-CitationAuthority",
+    "38": "B-Reasoning",
+    "39": "I-InformationExposition",
+    "40": "I-Facilitate",
+    "41": "B-FirstPerson",
+    "42": "I-ConfidenceHedged",
+    "43": "I-FirstPerson",
+    "44": "I-Character",
+    "45": "B-ConfidenceLow",
+    "46": "B-MetadiscourseCohesive",
+    "47": "B-InformationChangeNegative",
+    "48": "B-Uncertainty",
+    "49": "B-AcademicWritingMoves",
+    "50": "I-ConfidenceLow",
+    "51": "I-Strategic",
+    "52": "I-SyntacticComplexity",
+    "53": "B-Negative",
+    "54": "I-Inquiry",
+    "55": "I-InformationChangeNegative",
+    "56": "I-InformationTopics",
+    "57": "B-Future",
+    "58": "I-ConfidenceHigh",
+    "59": "B-Positive",
+    "60": "B-CitationHedged",
+    "61": "I-CitationHedged",
+    "62": "I-ForceStressed",
+    "63": "B-Inquiry",
+    "64": "I-InformationChangePositive",
+    "65": "B-ConfidenceHigh",
+    "66": "I-Uncertainty",
+    "67": "B-InformationReportVerbs",
+    "68": "I-InformationChange",
+    "69": "B-Citation",
+    "70": "B-InformationStates",
+    "71": "I-Future",
+    "72": "B-Facilitate",
+    "73": "I-Positive",
+    "74": "B-Contingent",
+    "75": "PAD"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-AcademicTerms": 26,
+    "B-AcademicWritingMoves": 49,
+    "B-Character": 19,
+    "B-Citation": 69,
+    "B-CitationAuthority": 37,
+    "B-CitationHedged": 60,
+    "B-ConfidenceHedged": 18,
+    "B-ConfidenceHigh": 65,
+    "B-ConfidenceLow": 45,
+    "B-Contingent": 74,
+    "B-Description": 0,
+    "B-Facilitate": 72,
+    "B-FirstPerson": 41,
+    "B-ForceStressed": 17,
+    "B-Future": 57,
+    "B-InformationChange": 8,
+    "B-InformationChangeNegative": 47,
+    "B-InformationChangePositive": 22,
+    "B-InformationExposition": 16,
+    "B-InformationPlace": 30,
+    "B-InformationReportVerbs": 67,
+    "B-InformationStates": 70,
+    "B-InformationTopics": 11,
+    "B-Inquiry": 63,
+    "B-Interactive": 31,
+    "B-MetadiscourseCohesive": 46,
+    "B-MetadiscourseInteractive": 12,
+    "B-Narrative": 35,
+    "B-Negative": 53,
+    "B-Positive": 59,
+    "B-PublicTerms": 23,
+    "B-Reasoning": 38,
+    "B-Responsibility": 1,
+    "B-Strategic": 3,
+    "B-SyntacticComplexity": 7,
+    "B-Uncertainty": 48,
+    "B-Updates": 20,
+    "I-AcademicTerms": 32,
+    "I-AcademicWritingMoves": 2,
+    "I-Character": 44,
+    "I-Citation": 34,
+    "I-CitationAuthority": 33,
+    "I-CitationHedged": 61,
+    "I-ConfidenceHedged": 42,
+    "I-ConfidenceHigh": 58,
+    "I-ConfidenceLow": 50,
+    "I-Contingent": 4,
+    "I-Description": 9,
+    "I-Facilitate": 40,
+    "I-FirstPerson": 43,
+    "I-ForceStressed": 62,
+    "I-Future": 71,
+    "I-InformationChange": 68,
+    "I-InformationChangeNegative": 55,
+    "I-InformationChangePositive": 64,
+    "I-InformationExposition": 39,
+    "I-InformationPlace": 13,
+    "I-InformationReportVerbs": 21,
+    "I-InformationStates": 5,
+    "I-InformationTopics": 56,
+    "I-Inquiry": 54,
+    "I-Interactive": 6,
+    "I-MetadiscourseCohesive": 24,
+    "I-MetadiscourseInteractive": 27,
+    "I-Narrative": 10,
+    "I-Negative": 29,
+    "I-Positive": 73,
+    "I-PublicTerms": 36,
+    "I-Reasoning": 15,
+    "I-Responsibility": 14,
+    "I-Strategic": 51,
+    "I-SyntacticComplexity": 52,
+    "I-Uncertainty": 66,
+    "I-Updates": 28,
+    "O": 25,
+    "PAD": 75
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.3.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 28996
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89ce3ddfce5a3b7f1b62c7f7f16728642c9c81284505155b8683fae6e5e501b3
+size 431198655

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d04ece69d04b890153ea3bd5c2ef5706f9181495a0778a2593c6118f7ce2dc3
+size 526681800

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": false, "max_len": 512, "init_inputs": []}