TASS 2018-Task-4. Official Results
The results published in this web page are the official ones of the shared task.
S1: Monolingual classification
Test corpus: L1. SANSE-TEST-500
| Group | Run Id | Date | Macro-P | Macro-R | Macro-F1 | Accuracy |
|---|---|---|---|---|---|---|
| INGEOTEC | ingeotec-run1 | 2018-07-08 18:43:09 | 0.794 | 0.795 | 0.795 | 0.802 |
| ELiRF-UPV | elirf_upv_l1_run_1 | 2018-07-08 18:43:09 | 0.795 | 0.784 | 0.790 | 0.800 |
| ELiRF-UPV | elirf_upv_l1_run_2 | 2018-07-08 18:43:09 | 0.787 | 0.794 | 0.790 | 0.794 |
| rbnUGR | model1_l1 | 2018-07-08 18:43:09 | 0.784 | 0.764 | 0.774 | 0.786 |
| MEANINGCLOUD | meaningcloud_s1_l1_run_3 | 2018-07-08 18:43:09 | 0.767 | 0.767 | 0.767 | 0.776 |
| rbnUGR | mode3_l1 | 2018-07-08 18:43:09 | 0.763 | 0.765 | 0.764 | 0.772 |
| rbnUGR | model2_l1 | 2018-07-08 18:43:09 | 0.774 | 0.752 | 0.763 | 0.776 |
| SINAI | fmplaza_sinai_ujaen_task4_level1 | 2018-07-08 18:43:09 | 0.733 | 0.722 | 0.728 | 0.742 |
| MEANINGCLOUD | meaningcloud_s1_l1_run_2 | 2018-07-08 18:43:09 | 0.723 | 0.727 | 0.725 | 0.732 |
| MEANINGCLOUD | meaningcloud_s1_l1_run_1 | 2018-07-08 18:43:09 | 0.713 | 0.722 | 0.717 | 0.714 |
| lone_wolf | 1 | 2018-07-08 18:43:09 | 0.729 | 0.674 | 0.700 | 0.718 |
| TNT-UA-WFU | tnt-ua-wfu_run_2 | 2018-07-08 18:43:09 | 0.492 | 0.492 | 0.492 | 0.518 |
| TNT-UA-WFU | 01 | 2018-07-08 18:43:09 | 0.000 | 0.000 | 0.000 | 0.000 |
Test corpus: L2. SANSE-TEST-13152
| Group | Run Id | Date | Macro-P | Macro-R | Macro-F1 | Accuracy |
|---|---|---|---|---|---|---|
| ELiRF | elirf_upv_l2_run_1 | 2018-07-08 18:43:18 | 0.878 | 0.889 | 0.883 | 0.893 |
| rbnUGR | model1_l2 | 2018-07-08 18:43:18 | 0.880 | 0.867 | 0.873 | 0.888 |
| ELiRF | elirf_upv_l2_run_2 | 2018-07-08 18:43:18 | 0.850 | 0.884 | 0.867 | 0.865 |
| INGEOTEC | ingeotec-run1 | 2018-07-08 18:43:18 | 0.853 | 0.880 | 0.866 | 0.871 |
| rbnUGR | modell2-l2 | 2018-07-08 18:43:18 | 0.868 | 0.857 | 0.863 | 0.878 |
| rbnUGR | model3_l2 | 2018-07-08 18:43:18 | 0.838 | 0.870 | 0.853 | 0.853 |
| MEANINGCLOUD | meaningcloud_s1_l2_run_3 | 2018-07-08 18:43:18 | 0.781 | 0.804 | 0.793 | 0.801 |
| SINAI | fmplaza_sinai_ujaen_task4_level2 | 2018-07-08 18:43:18 | 0.769 | 0.777 | 0.773 | 0.793 |
| TNT-UA-WFU | tnt-ua-wfu_l2_run_2 | 2018-07-08 18:43:18 | 0.542 | 0.547 | 0.544 | 0.552 |
| lone_wolf | 1 | 2018-07-08 18:43:18 | 0.000 | 0.000 | 0.000 | 0.000 |
Test corpus: SANSE-408
| Group | Run Id | Date | Macro-P | Macro-R | Macro-F1 | Accuracy |
|---|---|---|---|---|---|---|
| INGEOTEC | ingeotec-run1 | 2018-07-08 18:50:17 | 0.722 | 0.715 | 0.719 | 0.737 |
| ELiRF-UPV | elirf_upv_2_run_2 | 2018-07-08 18:50:17 | 0.747 | 0.657 | 0.699 | 0.722 |
| ELiRF-UPV | elirf_upv_2_run_1 | 2018-07-08 18:50:17 | 0.736 | 0.649 | 0.690 | 0.715 |
| rbnUGR | model3 | 2018-07-08 18:50:17 | 0.687 | 0.678 | 0.683 | 0.631 |
| rbnUGR | model2 | 2018-07-08 18:50:17 | 0.679 | 0.672 | 0.676 | 0.698 |
| rbnUGR | model1 | 2018-07-08 18:50:17 | 0.683 | 0.661 | 0.672 | 0.700 |
| MEANINGCLOUD | meaningcloud_s2_run_3 | 2018-07-08 18:50:17 | 0.647 | 0.654 | 0.651 | 0.658 |
| ITAINNOVA | run1 | 2018-07-08 18:50:17 | 0.619 | 0.615 | 0.617 | 0.575 |
Gold Standard
The gold standard will be published in this web page after the evaluation deadline.
- test-s1-l1-tagged: Test with labels for SANSE corpus s1 l1.
- test-s1-l2-tagged: Test with labels for SANSE corpus s1 l2.
- test-s2-tagged: Test with labels for SANSE corpus s2.
Organized by:
