card

README.md

@@ -0,0 +1,88 @@
+---
+language: 
+- english
+thumbnail: 
+tags:
+- token classification
+license: 
+datasets:
+- EMBO/sd-panels
+metrics:
+-
+---
+
+# sd-ner
+
+## Model description
+
+This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang) and fine-tuned for token classification on the SourceData [sd-panels](https://huggingface.co/datasets/EMBO/sd-panels) dataset to perform Named Entity Recognition of bioentities. 
+
+
+## Intended uses & limitations
+
+#### How to use
+
+The intended use of this model is for Named Entity Recognition of biological entitie used in SourceData annotations (https://sourcedata.embo.org), including small molecules, gene products (genes and proteins), subcellular components, cell line and cell types, organ and tissues, species as well as experimental methods.
+
+To have a quick check of the model:
+
+```python
+from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification
+example = """<s> F. Western blot of input and eluates of Upf1 domains purification in a Nmd4-HA strain. The band with the # might corresponds to a dimer of Upf1-CH, bands marked with a star correspond to residual signal with the anti-HA antibodies (Nmd4). Fragments in the eluate have a smaller size because the protein A part of the tag was removed by digestion with the TEV protease. G6PDH served as a loading control in the input samples </s>"""
+tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)
+model = RobertaForTokenClassification.from_pretrained('EMBO/sd-ner')
+ner = pipeline('ner', model, tokenizer=tokenizer)
+res = ner(example)
+for r in res:
+    print(r['word'], r['entity'])
+```
+
+#### Limitations and bias
+
+The model must be used with the `roberta-base` tokenizer.
+
+## Training data
+
+The model was trained for token classification using the  [EMBO/sd-panels dataset](https://huggingface.co/datasets/EMBO/biolang) wich includes manually annotated examples.
+
+## Training procedure
+
+The training was run on a NVIDIA DGX Station with 4XTesla V100 GPUs.
+
+Training code is available at https://github.com/source-data/soda-roberta
+
+- Command: `python -m tokcl.train /data/json/sd_panels NER  --num_train_epochs=3.5`
+- Tokenizer vocab size: 50265
+- Training data: EMBO/biolang MLM
+- Training with 31410 examples.
+- Evaluating on 8861 examples.
+- Training on 15 features: O, I-SMALL_MOLECULE, B-SMALL_MOLECULE, I-GENEPROD, B-GENEPROD, I-SUBCELLULAR, B-SUBCELLULAR, I-CELL, B-CELL, I-TISSUE, B-TISSUE, I-ORGANISM, B-ORGANISM, I-EXP_ASSAY, B-EXP_ASSAY
+- Epochs: 3.5
+- `per_device_train_batch_size`: 32
+- `per_device_eval_batch_size`: 32
+- `learning_rate`: 0.0001
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+
+## Eval results
+
+On test set with `sklearn.metrics`:
+
+```
+                precision    recall  f1-score   support
+
+          CELL       0.77      0.81      0.79      3477
+     EXP_ASSAY       0.71      0.70      0.71      7049
+      GENEPROD       0.86      0.90      0.88     16140
+      ORGANISM       0.80      0.82      0.81      2759
+SMALL_MOLECULE       0.78      0.82      0.80      4446
+   SUBCELLULAR       0.71      0.75      0.73      2125
+        TISSUE       0.70      0.75      0.73      1971
+
+     micro avg       0.79      0.82      0.81     37967
+     macro avg       0.76      0.79      0.78     37967
+  weighted avg       0.79      0.82      0.81     37967
+```