TASS 2015. Official Results
As some groups have requested extra time for submission of non-official experiments, the annotated test sets will be released in a few days.
Task 1: Sentiment Analysis at global level (5 levels, Full test corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| UCSP | UCSP-RUN-1 | 2015-06-25 22:56:00 | 0.273 | 0.149 | 0.259 | 0.190 |
| GSI | GSI-RUN-1 | 2015-06-26 21:47:27 | 0.618 | 0.472 | 0.532 | 0.500 |
| GSI | GSI-RUN-2 | 2015-06-26 21:52:57 | 0.610 | 0.466 | 0.526 | 0.495 |
| GSI | GSI-RUN-3 | 2015-06-26 21:57:12 | 0.608 | 0.466 | 0.525 | 0.493 |
| GAS-UCR | 01 | 2015-06-27 23:46:24 | 0.342 | 0.217 | 0.237 | 0.227 |
| CU | CU-Run-1 | 2015-06-29 05:35:02 | 0.495 | 0.393 | 0.441 | 0.416 |
| LyS | LyS-run-1 | 2015-06-29 10:00:47 | 0.552 | 0.438 | 0.509 | 0.471 |
| INGEOTEC | INGEOTEC-M1 | 2015-06-29 17:48:32 | 0.488 | 0.398 | 0.461 | 0.427 |
| LyS | Lys-run-2 | 2015-06-29 18:41:20 | 0.568 | 0.439 | 0.492 | 0.464 |
| ELiRF | run1 | 2015-06-30 12:15:24 | 0.648 | 0.493 | 0.541 | 0.516 |
| TID-spark | TID-spark-1 | 2015-06-30 12:40:34 | 0.462 | 0.385 | 0.450 | 0.415 |
| LyS | Lys-run-3 | 2015-06-30 13:59:17 | 0.505 | 0.465 | 0.313 | 0.374 |
| LIF | LIF-Run-1 | 2015-06-30 17:37:00 | 0.628 | 0.493 | 0.545 | 0.518 |
| LIF | LIF-Run-2 | 2015-06-30 17:38:34 | 0.654 | 0.503 | 0.539 | 0.520 |
| GTI-GRAD | GTI-GRAD-Run1 | 2015-06-30 19:41:05 | 0.592 | 0.433 | 0.491 | 0.460 |
| CU | CU-Run-2-CompMod | 2015-06-30 19:44:27 | 0.362 | 0.313 | 0.334 | 0.323 |
| SINAI_wd2v | SINAI_wd2v_500 | 2015-07-01 01:58:04 | 0.474 | 0.380 | 0.425 | 0.401 |
| SINAI_wd2v | SINAI_wd2v_300 | 2015-07-01 02:00:49 | 0.474 | 0.380 | 0.425 | 0.401 |
| DT | DT-RUN-1 | 2015-07-01 09:43:41 | 0.560 | 0.299 | 0.323 | 0.311 |
| DT | DT-RUN-3 | 2015-07-01 17:02:47 | 0.557 | 0.296 | 0.320 | 0.308 |
| DT | DT-RUN-2 | 2015-07-01 17:03:45 | 0.545 | 0.294 | 0.318 | 0.306 |
| ITAINNOVA | Ensemble exp1.1 | 2015-07-01 19:41:41 | 0.535 | 0.421 | 0.445 | 0.433 |
| ITAINNOVA | Ensemble exp2_emotions | 2015-07-01 19:46:34 | 0.524 | 0.419 | 0.397 | 0.408 |
| ITAINNOVA | Ensemble exp3_emotions | 2015-07-01 20:07:12 | 0.549 | 0.405 | 0.429 | 0.416 |
| BittenPotato | BP-rbf-v1 | 2015-07-01 20:13:34 | 0.494 | 0.361 | 0.379 | 0.370 |
| BittenPotato | BP-voted-v1 | 2015-07-01 20:15:47 | 0.522 | 0.389 | 0.397 | 0.393 |
| BittenPotato | BP-wvoted-v1 | 2015-07-01 20:16:57 | 0.522 | 0.388 | 0.402 | 0.395 |
| BittenPotato | BP-wvoted-v2 | 2015-07-01 20:34:42 | 0.009 | 0.396 | 0.007 | 0.013 |
| BittenPotato | BP-voted-v2 | 2015-07-01 20:39:49 | 0.535 | 0.396 | 0.421 | 0.408 |
| BittenPotato | BP-rbf-v2 | 2015-07-01 20:41:19 | 0.514 | 0.377 | 0.393 | 0.385 |
| BittenPotato | BP-wvoted-v2_1 | 2015-07-01 20:53:46 | 0.534 | 0.402 | 0.430 | 0.415 |
| ELiRF | run2 | 2015-07-01 22:22:08 | 0.658 | 0.490 | 0.519 | 0.504 |
| ELiRF | run3 | 2015-07-01 22:39:09 | 0.659 | 0.503 | 0.538 | 0.520 |
| DLSI | Run1 | 2015-07-02 00:13:07 | 0.595 | 0.517 | 0.432 | 0.471 |
| LIF | LIF-Run-3 | 2015-07-02 07:26:33 | 0.672 | 0.552 | 0.507 | 0.529 |
| ELiRF | run_out_of_date | 2015-07-02 09:53:47 | 0.673 | 0.531 | 0.465 | 0.496 |
| SINAI-ESMA | 1 | 2015-07-02 19:48:19 | 0.502 | 0.404 | 0.458 | 0.430 |
(System ranking by accuracy)
Task 1: Sentiment Analysis at global level (5 levels, 1k corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| GSI | GSI-RUN-1 | 2015-06-26 21:47:58 | 0.487 | 0.441 | 0.451 | 0.446 |
| GSI | GSI-RUN-2 | 2015-06-26 21:53:38 | 0.480 | 0.435 | 0.444 | 0.440 |
| GSI | GSI-RUN-3 | 2015-06-26 22:05:24 | 0.479 | 0.432 | 0.443 | 0.437 |
| GAS-UCR | 01 | 2015-06-27 23:47:13 | 0.338 | 0.229 | 0.233 | 0.231 |
| CU | CU-Run-1-CompMod | 2015-06-29 05:37:24 | 0.419 | 0.365 | 0.372 | 0.369 |
| LyS | LyS-run-1 | 2015-06-29 10:02:48 | 0.428 | 0.380 | 0.400 | 0.390 |
| INGEOTEC | INGEOTEC-M1 | 2015-06-29 17:42:54 | 0.431 | 0.398 | 0.411 | 0.404 |
| LyS | Lys-run-2 | 2015-06-29 18:41:56 | 0.434 | 0.355 | 0.395 | 0.374 |
| LyS | Lys-run-3 | 2015-06-30 11:56:51 | 0.430 | 0.391 | 0.361 | 0.376 |
| DT | DT-RUN-2 | 2015-06-30 12:30:21 | 0.408 | 0.258 | 0.335 | 0.291 |
| ELiRF | run1 | 2015-06-30 12:48:49 | 0.476 | 0.400 | 0.429 | 0.414 |
| TID-spark | TID | 2015-06-30 12:55:30 | 0.400 | 0.355 | 0.377 | 0.366 |
| DT | DT-RUN-1 | 2015-06-30 15:34:03 | 0.407 | 0.285 | 0.334 | 0.308 |
| DT | DT-RUN-3 | 2015-06-30 17:18:30 | 0.396 | 0.221 | 0.329 | 0.265 |
| LIF | LIF-Run-1 | 2015-06-30 17:42:56 | 0.481 | 0.428 | 0.439 | 0.433 |
| LIF | LIF-Run-2 | 2015-06-30 17:49:23 | 0.516 | 0.441 | 0.459 | 0.450 |
| GTI-GRAD | GTI-GRAD-Run1 | 2015-06-30 19:42:30 | 0.509 | 0.469 | 0.453 | 0.461 |
| CU | CU-Run-2 | 2015-06-30 19:49:27 | 0.397 | 0.345 | 0.372 | 0.358 |
| SEDEMO | E1 | 2015-06-30 19:50:25 | 0.397 | 0.309 | 0.345 | 0.326 |
| INGEOTEC | INGEOTEC-E1 | 2015-06-30 22:55:08 | 0.174 | 0.148 | 0.186 | 0.165 |
| INGEOTEC | INGEOTEC-E2 | 2015-06-30 23:33:07 | 0.168 | 0.156 | 0.159 | 0.158 |
| SINAI_wd2v | SINAI_wd2v | 2015-07-01 01:30:42 | 0.389 | 0.443 | 0.389 | 0.414 |
| ITAINNOVA | Ensemble exp1 !K | 2015-07-01 20:17:25 | 0.405 | 0.401 | 0.392 | 0.396 |
| BittenPotato | BP-rbf-v1 | 2015-07-01 20:18:30 | 0.418 | 0.334 | 0.354 | 0.344 |
| BittenPotato | BP-voted-v1 | 2015-07-01 20:18:52 | 0.408 | 0.363 | 0.367 | 0.365 |
| BittenPotato | BP-wvoted-v1 | 2015-07-01 20:19:08 | 0.416 | 0.346 | 0.363 | 0.354 |
| ITAINNOVA | Ensemble exp2 1K | 2015-07-01 21:40:29 | 0.384 | 0.400 | 0.373 | 0.386 |
| ITAINNOVA | Ensemble exp3 1K | 2015-07-01 21:44:59 | 0.396 | 0.371 | 0.365 | 0.368 |
| ELiRF | run2 | 2015-07-01 22:22:37 | 0.488 | 0.578 | 0.443 | 0.502 |
| ELiRF | run3 | 2015-07-01 22:40:07 | 0.477 | 0.396 | 0.433 | 0.414 |
| DLSI | Run1 | 2015-07-02 00:15:11 | 0.385 | 0.378 | 0.346 | 0.362 |
| SINAI-ESMA | 1 | 2015-07-02 20:00:42 | 0.411 | 0.366 | 0.380 | 0.373 |
(System ranking by accuracy)
Task 1: Sentiment Analysis at global level (3 levels, Full test corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| UCSP | UCSP-RUN-1 | 2015-06-22 19:13:08 | 0.602 | 0.492 | 0.501 | 0.497 |
| CU | CU-RUN-1 | 2015-06-26 03:06:13 | 0.597 | 0.492 | 0.503 | 0.497 |
| GSI | GSI-RUN-1 | 2015-06-26 21:49:23 | 0.690 | 0.558 | 0.541 | 0.550 |
| GSI | GSI-RUN-2 | 2015-06-26 21:55:54 | 0.679 | 0.559 | 0.533 | 0.546 |
| GSI | GSI-RUN-3 | 2015-06-26 22:00:02 | 0.678 | 0.559 | 0.533 | 0.545 |
| GAS-UCR | 01 | 2015-06-27 23:48:29 | 0.446 | 0.327 | 0.346 | 0.336 |
| LyS | LyS-run-1 | 2015-06-29 12:11:09 | 0.664 | 0.539 | 0.528 | 0.533 |
| CU | CU-Run2-CompMod | 2015-06-29 19:53:21 | 0.481 | 0.404 | 0.403 | 0.404 |
| LyS | Lys-run-2 | 2015-06-30 12:07:20 | 0.599 | 0.476 | 0.465 | 0.471 |
| ELiRF | run1 | 2015-06-30 12:48:11 | 0.712 | 0.589 | 0.559 | 0.573 |
| TID-spark | TID-spark-1 | 2015-06-30 13:11:17 | 0.594 | 0.518 | 0.491 | 0.504 |
| LIF | LIF-Run-1 | 2015-06-30 17:40:32 | 0.710 | 0.594 | 0.560 | 0.576 |
| LIF | LIF-Run-2 | 2015-06-30 17:41:55 | 0.725 | 0.598 | 0.567 | 0.582 |
| INGEOTEC | INGEOTEC-M1 | 2015-06-30 19:36:34 | 0.613 | 0.503 | 0.504 | 0.504 |
| GTI-GRAD | GTI-GRAD-Run1 | 2015-06-30 19:43:44 | 0.695 | 0.519 | 0.536 | 0.527 |
| SINAI_wd2v | SINAI_wd2v | 2015-06-30 23:58:49 | 0.619 | 0.471 | 0.483 | 0.477 |
| SINAI_wd2v | SINAI_d2v | 2015-07-01 00:00:06 | 0.429 | 0.371 | 0.350 | 0.360 |
| SINAI_wd2v | SINAI_w2v | 2015-07-01 00:01:24 | 0.604 | 0.470 | 0.463 | 0.466 |
| SINAI_wd2v | SINAI_wd2v_2 | 2015-07-01 00:03:38 | 0.613 | 0.470 | 0.483 | 0.477 |
| DT | DT-RUN-1 | 2015-07-01 09:47:32 | 0.490 | 0.419 | 0.372 | 0.394 |
| DT | DT-RUN-2 | 2015-07-01 13:15:16 | 0.625 | 0.474 | 0.486 | 0.480 |
| DT | DT-RUN-3 | 2015-07-01 13:16:59 | 0.608 | 0.462 | 0.473 | 0.467 |
| UCSP | UCSP-RUN-1-NB | 2015-07-01 17:08:51 | 0.560 | 0.501 | 0.438 | 0.467 |
| UCSP | UCSP-RUN-1-ME | 2015-07-01 17:11:24 | 0.600 | 0.519 | 0.472 | 0.494 |
| UCSP | UCSP-RUN-3 | 2015-07-01 17:27:50 | 0.613 | 0.515 | 0.487 | 0.501 |
| UCSP | UCSP-RUN-2 | 2015-07-01 17:38:54 | 0.594 | 0.496 | 0.512 | 0.504 |
| UCSP | UCSP-RUN-1-DT | 2015-07-01 17:45:15 | 0.536 | 0.435 | 0.441 | 0.438 |
| UCSP | UCSP-RUN-2-ME | 2015-07-01 17:51:08 | 0.479 | 0.381 | 0.380 | 0.380 |
| ITAINNOVA | Ensemble exp1_3_SPARK | 2015-07-01 19:52:37 | 0.610 | 0.498 | 0.497 | 0.497 |
| ITAINNOVA | Ensemble exp2_3_SPARK | 2015-07-01 19:56:35 | 0.591 | 0.504 | 0.493 | 0.498 |
| ITAINNOVA | Ensemble exp3_3 | 2015-07-01 20:01:36 | 0.594 | 0.484 | 0.482 | 0.483 |
| BittenPotato | BP-rbf-v1 | 2015-07-01 21:03:34 | 0.602 | 0.474 | 0.471 | 0.472 |
| BittenPotato | BP-voted-v1 | 2015-07-01 21:05:24 | 0.593 | 0.500 | 0.469 | 0.484 |
| BittenPotato | BP-wvoted-v1 | 2015-07-01 21:06:23 | 0.593 | 0.508 | 0.465 | 0.486 |
| ELiRF | run2 | 2015-07-01 22:23:32 | 0.722 | 0.574 | 0.559 | 0.567 |
| ELiRF | run3 | 2015-07-01 22:41:01 | 0.721 | 0.585 | 0.573 | 0.579 |
| DLSI | Run1 | 2015-07-02 00:33:20 | 0.655 | 0.574 | 0.513 | 0.542 |
| LIF | LIF-Run-3 | 2015-07-02 07:35:03 | 0.726 | 0.601 | 0.567 | 0.584 |
| ELiRF | run_out_of_date | 2015-07-02 10:02:29 | 0.725 | 0.541 | 0.556 | 0.548 |
(System ranking by accuracy)
Task 1: Sentiment Analysis at global level (3 levels, 1k corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| UCSP | UCSP-RUN-1 | 2015-06-22 19:11:17 | 0.582 | 0.457 | 0.476 | 0.466 |
| CU | CU-RUN-1 | 2015-06-26 16:56:42 | 0.578 | 0.450 | 0.493 | 0.470 |
| GSI | GSI-RUN-1 | 2015-06-26 21:50:05 | 0.658 | 0.526 | 0.537 | 0.531 |
| GSI | GSI-RUN-2 | 2015-06-26 21:54:48 | 0.646 | 0.455 | 0.524 | 0.487 |
| GSI | GSI-RUN-3 | 2015-06-26 22:06:26 | 0.647 | 0.455 | 0.524 | 0.487 |
| GAS-UCR | 01 | 2015-06-27 23:49:06 | 0.556 | 0.334 | 0.329 | 0.331 |
| CU | CU-Run-2-CompMod | 2015-06-29 05:12:01 | 0.600 | 0.461 | 0.481 | 0.471 |
| LyS | LyS-run-1 | 2015-06-29 12:10:20 | 0.634 | 0.478 | 0.515 | 0.496 |
| DT | DT-RUN-3 | 2015-06-29 16:17:03 | 0.571 | 0.405 | 0.472 | 0.436 |
| DR | DT-RUN-1 | 2015-06-29 17:13:44 | 0.514 | 0.363 | 0.309 | 0.334 |
| LyS | Lys-run-2 | 2015-06-30 11:51:07 | 0.605 | 0.428 | 0.465 | 0.446 |
| ELiRF | run1 | 2015-06-30 12:50:03 | 0.645 | 0.456 | 0.531 | 0.490 |
| TID-spark | TID-spark-1 | 2015-06-30 13:12:39 | 0.649 | 0.503 | 0.486 | 0.494 |
| DT | DT-RUN-1 | 2015-06-30 15:34:29 | 0.601 | 0.475 | 0.502 | 0.488 |
| DT | DT-RUN-2 | 2015-06-30 16:22:26 | 0.583 | 0.411 | 0.473 | 0.440 |
| LIF | LIF-Run-1 | 2015-06-30 17:45:43 | 0.632 | 0.719 | 0.547 | 0.621 |
| LIF | LIF-Run-2 | 2015-06-30 17:50:25 | 0.692 | 0.615 | 0.572 | 0.593 |
| GTI-GRAD | GTI-GRAD-Run1 | 2015-06-30 19:47:24 | 0.674 | 0.577 | 0.491 | 0.530 |
| INGEOTEC | INGEOTEC-M1 | 2015-06-30 19:53:45 | 0.595 | 0.463 | 0.485 | 0.474 |
| SINAI_wd2v | SINAI_d2v | 2015-07-01 01:26:07 | 0.510 | 0.317 | 0.297 | 0.306 |
| SINAI_wd2v | SINAI_w2v | 2015-07-01 01:26:46 | 0.627 | 0.442 | 0.492 | 0.466 |
| SINAI_wd2v | SINAI_wd2v | 2015-07-01 01:27:53 | 0.633 | 0.440 | 0.481 | 0.460 |
| SINAI_wd2v | SINAI_wd2v_2_500 | 2015-07-01 01:58:46 | 0.641 | 0.449 | 0.488 | 0.468 |
| UCSP | UCSP-RUN-1-ME | 2015-07-01 17:21:05 | 0.626 | 0.439 | 0.457 | 0.448 |
| UCSP | UCSP-RUN-1-NB | 2015-07-01 17:21:49 | 0.636 | 0.511 | 0.415 | 0.458 |
| UCSP | UCSP-RUN-3 | 2015-07-01 17:58:40 | 0.626 | 0.466 | 0.464 | 0.465 |
| UCSP | UCSP-RUN-2 | 2015-07-01 17:59:40 | 0.586 | 0.470 | 0.496 | 0.483 |
| UCSP | UCSP-RUN-1-DR | 2015-07-01 18:01:51 | 0.495 | 0.400 | 0.433 | 0.416 |
| UCSP | UCSP-RUN-2-NB | 2015-07-01 19:41:46 | 0.559 | 0.401 | 0.342 | 0.369 |
| UCSP | UCSP-RUN-2-ME | 2015-07-01 19:43:21 | 0.509 | 0.338 | 0.335 | 0.337 |
| BittenPotato | BP-rbf-v1 | 2015-07-01 21:09:18 | 0.611 | 0.432 | 0.492 | 0.460 |
| BittenPotato | BP-voted-v1 | 2015-07-01 21:09:31 | 0.611 | 0.563 | 0.489 | 0.523 |
| BittenPotato | BP-wvoted-v1 | 2015-07-01 21:09:41 | 0.632 | 0.703 | 0.492 | 0.579 |
| ITAINNOVA | Ensemble exp1 1K 3 | 2015-07-01 21:54:37 | 0.503 | 0.460 | 0.491 | 0.475 |
| ELiRF | run2 | 2015-07-01 22:25:06 | 0.610 | 0.703 | 0.534 | 0.607 |
| ELiRF | run3 | 2015-07-01 22:41:25 | 0.595 | 0.448 | 0.526 | 0.484 |
| ITAINNOVA | Ensemble exp2 1K 3 | 2015-07-01 23:18:54 | 0.611 | 0.467 | 0.501 | 0.483 |
| DLSI | Run1 | 2015-07-02 00:20:55 | 0.637 | 0.503 | 0.485 | 0.494 |
| ITAINNOVA | Ensemble exp3 1K 3 | 2015-07-02 08:28:53 | 0.614 | 0.466 | 0.503 | 0.484 |
(System ranking by accuracy)
Task 2: Aspect based Sentiment Analysis (Social-TV corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| GSi | GSI-RUN-1 | 2015-06-25 20:20:03 | 0.533 | 0.493 | 0.427 | 0.458 |
| LyS | LyS-run-1 | 2015-06-29 10:05:45 | 0.610 | 0.463 | 0.494 | 0.478 |
| TID-spark | TID-spark-1 | 2015-06-29 16:08:33 | 0.631 | 0.460 | 0.484 | 0.471 |
| GSI | GSI-RUN-3 | 2015-06-29 21:55:56 | 0.557 | 0.593 | 0.527 | 0.558 |
| GSI | GSI-RUN-2 | 2015-06-29 23:09:54 | 0.621 | 0.622 | 0.551 | 0.584 |
| GSI | GSI-RUN-1 | 2015-06-29 23:15:53 | 0.635 | 0.637 | 0.578 | 0.606 |
| ELiRF | run1 | 2015-06-30 14:02:59 | 0.655 | 0.481 | 0.512 | 0.496 |
| LyS | Lys-run-2 | 2015-06-30 18:42:08 | 0.522 | 0.443 | 0.442 | 0.443 |
(System ranking by accuracy)
Task 2: Aspect based Sentiment Analysis (STOMPOL corpus)
| Group | Run Id | Date | Accuracy | Macro-P | Macro-R | Macro-F1 |
|---|---|---|---|---|---|---|
| LyS | LyS-run-1 | 2015-06-29 10:06:21 | 0.599 | 0.472 | 0.471 | 0.472 |
| LyS | Lys-run-2 | 2015-06-30 12:16:40 | 0.540 | 0.372 | 0.444 | 0.405 |
| TID-spark | TID-spark-1 | 2015-06-30 13:15:12 | 0.557 | 0.252 | 0.297 | 0.272 |
| ELiRF | run1 | 2015-06-30 16:46:37 | 0.633 | 0.522 | 0.531 | 0.526 |
(System ranking by accuracy)
