TASS 2017. Official Results

The results published in this web page are the official ones of the shared task.

Task 1: Sentiment Analysis at tweet level

Test corpus: InterTASS

Group	Run Id	Date	Macro-P	Macro-R	Macro-F1	Accuracy
ELiRF-UPV	run1	2017-07-03 13:42:28	0.497	0.490	0.493	0.607
RETUYT	svm_cnn	2017-07-04 23:02:40	0.490	0.453	0.471	0.596
ELiRF-UPV	run3	2017-07-03 13:44:25	0.470	0.461	0.466	0.597
ITAINNOVA	model4	2017-07-04 23:15:46	0.469	0.454	0.461	0.576
jacerong	run-2	2017-07-02 20:06:02	0.490	0.434	0.460	0.602
jacerong	run-1	2017-07-02 20:03:48	0.479	0.440	0.459	0.608
INGEOTEC	evodag_001	2017-07-03 20:49:50	0.459	0.455	0.457	0.507
RETUYT	svm	2017-07-04 02:32:25	0.470	0.444	0.457	0.583
tecnolengua	sent_only	2017-07-04 16:13:26	0.453	0.459	0.456	0.582
ELiRF-UPV	run2	2017-07-03 13:43:44	0.481	0.424	0.450	0.436
ITAINNOVA	model3	2017-07-04 23:15:46	0.443	0.446	0.445	0.561
RETUYT	cnn3	2017-07-04 21:50:00	0.458	0.428	0.443	0.558
SINAI	w2v-nouser	2017-07-03 12:11:47	0.464	0.423	0.442	0.575
tecnolengua	run3	2017-07-03 01:07:03	0.437	0.445	0.441	0.576
tecnolengua	sent_only_fixed	2017-07-04 20:30:42	0.455	0.427	0.441	0.595
ITAINNOVA	model2	2017-07-05 09:01:49	0.450	0.423	0.436	0.576
LexFAR	run3	2017-07-04 21:58:34	0.433	0.431	0.432	0.541
LexFAR	run1	2017-07-04 18:33:01	0.433	0.427	0.430	0.539
jacerong	run-3	2017-07-02 20:07:27	0.427	0.434	0.430	0.576
SINAI	w2v-user	2017-07-03 12:15:02	0.443	0.415	0.428	0.569
INGEOTEC	evodag_002	2017-07-03 21:12:51	0.410	0.397	0.403	0.515
OEG	victor2	2017-07-04 21:50:14	0.400	0.389	0.395	0.451
OEG	victor0	2017-07-04 18:26:47	0.388	0.378	0.383	0.433
OEG	laOEG	2017-07-04 15:19:11	0.383	0.370	0.377	0.505
LexFAR	run2	2017-07-04 18:46:44	0.372	0.371	0.372	0.490
GSI	sent64-189	2017-07-02 22:55:51	0.397	0.349	0.371	0.524
SINAI	embed-rnn2	2017-07-03 12:32:38	0.339	0.328	0.333	0.391
GSI	sent64-149-ant-2	2017-07-02 23:41:11	0.291	0.322	0.306	0.479
GSI	sent64-149-ant	2017-07-02 23:40:34	0.000	0.000	0.000	0.000

Test corpus: General Corpus of TASS (3 levels, full test corpus)

Group	Run Id	Date	Macro-P	Macro-R	Macro-F1	Accuracy
INGEOTEC	evodag_003	2017-07-04 19:29:35	0.559	0.595	0.577	0.645
jacerong	run-1	2017-07-01 19:06:22	0.587	0.552	0.569	0.706
jacerong	tass_2016-run_3	2017-07-01 19:01:21	0.568	0.567	0.568	0.705
ELiRF-UPV	run2	2017-07-03 13:51:45	0.539	0.560	0.549	0.659
ELiRF-UPV	run3	2017-07-03 13:54:58	0.541	0.556	0.548	0.725
RETUYT	svm_cnn	2017-07-04 23:22:05	0.555	0.537	0.546	0.674
jacerong	run-2	2017-07-01 19:11:11	0.542	0.547	0.545	0.701
ELiRF-UPV	run1	2017-07-03 13:47:16	0.539	0.546	0.542	0.666
RETUYT	cnn	2017-07-04 03:42:15	0.534	0.548	0.541	0.638
RETUYT	cnn3	2017-07-04 22:23:18	0.542	0.536	0.539	0.654
tecnolengua	run3	2017-07-03 01:33:23	0.527	0.529	0.528	0.657
tecnolengua	final	2017-07-03 20:36:39	0.516	0.517	0.517	0.632
tecnolengua	531F1_no_ngrams	2017-07-01 17:27:16	0.508	0.509	0.508	0.652
INGEOTEC	evodag_001	2017-07-03 21:01:10	0.463	0.432	0.447	0.514
OEG	victor2	2017-07-05 00:15:33	0.395	0.384	0.389	0.496
INGEOTEC	evodag_002	2017-07-04 16:52:47	0.362	0.366	0.364	0.449
OEG	laOEG	2017-07-04 15:26:02	0.350	0.342	0.346	0.407
GSI	64sent99ally	2017-07-04 23:20:13	0.302	0.348	0.324	0.434

Test corpus: General Corpus of TASS (3 levels, 1k corpus)

Group	Run Id	Date	Macro-P	Macro-R	Macro-F1	Accuracy
RETUYT	svm	2017-07-04 02:40:58	0.545	0.580	0.562	0.700
RETUYT	cnn4	2017-07-04 22:32:17	0.542	0.572	0.557	0.694
RETUYT	cnn2	2017-07-04 15:48:18	0.563	0.547	0.555	0.694
INGEOTEC	evodag_003	2017-07-04 18:59:15	0.501	0.553	0.526	0.595
tecnolengua	run3	2017-07-03 01:12:12	0.501	0.543	0.521	0.638
ELiRF-UPV	run1	2017-07-03 14:00:16	0.496	0.544	0.519	0.630
jacerong	tass_2016-run_3	2017-07-02 20:36:56	0.500	0.538	0.518	0.625
jacerong	run-1	2017-07-02 19:15:06	0.476	0.545	0.508	0.678
jacerong	run-2	2017-07-02 20:34:09	0.473	0.544	0.506	0.673
ELiRF-UPV	run2	2017-07-03 14:00:49	0.480	0.530	0.504	0.596
tecnolengua	final	2017-07-03 18:19:43	0.470	0.507	0.488	0.618
tecnolengua	run4	2017-07-03 13:52:52	0.466	0.502	0.483	0.612
ELiRF-UPV	run3	2017-07-03 14:01:23	0.442	0.518	0.477	0.588
INGEOTEC	evodag_002	2017-07-03 20:58:20	0.435	0.443	0.439	0.431
INGEOTEC	evodag_001	2017-07-03 21:19:16	0.387	0.390	0.388	0.486
OEG	victor3b	2017-07-05 07:20:51	0.402	0.337	0.367	0.386
OEG	victor2	2017-07-04 21:58:49	0.361	0.370	0.366	0.412
OEG	laOEG	2017-07-04 15:23:02	0.348	0.345	0.346	0.448
GSI	run-1	2017-07-03 16:14:50	0.303	0.355	0.327	0.558
GSI	64sent99ally	2017-07-04 22:43:45	0.337	0.306	0.321	0.499

Task 2: Aspect based Sentiment Analysis

Test corpus: Social-TV

Group	Run Id	Date	Macro-P	Macro-R	Macro-F1	Accuracy
ELiRF-UPV	run3	2017-07-03 16:19:43	0.529	0.546	0.537	0.615
ELiRF-UPV	run2	2017-07-03 16:19:16	0.523	0.505	0.513	0.600
ELiRF-UPV	run1	2017-07-03 16:18:58	0.473	0.479	0.476	0.625
RETUYT	svm2	2017-07-04 21:16:05	0.432	0.421	0.426	0.595
RETUYT	svm	2017-07-04 18:13:41	0.419	0.406	0.413	0.493

Test corpus: STOMPOL

Group	Run Id	Date	Macro-P	Macro-R	Macro-F1	Accuracy
ELiRF-UPV	run1	2017-07-03 16:12:07	0.534	0.541	0.537	0.615
RETUYT	svm2	2017-07-04 21:48:29	0.516	0.501	0.508	0.590
ELiRF-UPV	run2	2017-07-03 16:13:55	0.492	0.481	0.486	0.541
ELiRF-UPV	run3	2017-07-03 16:14:16	0.482	0.490	0.486	0.578
C100T-PUCP	run3	2017-07-05 07:28:05	0.452	0.438	0.445	0.528
C100T-PUCP	run1	2017-07-04 20:37:38	0.418	0.413	0.415	0.563
C100T-PUCP	run2	2017-07-04 22:00:08	0.412	0.416	0.414	0.517
RETUYT	svm	2017-07-04 21:38:44	0.366	0.388	0.377	0.514

Gold Standard

The gold standard will be published in this web page after the evaluation deadline.

• TASS2017_T1_test_res.qrel: QREL file for InterTASS corpus
• general-sentiment-3l.qrel: QREL file for General corpus test set, sentiment analysis, 3 levels
• general-sentiment-3l-1k.qrel: QREL file for General corpus test 1k set, sentiment analysis, 3 levels
• socialtv-sentiment.qrel: QREL file for Social-TV corpus test set
• stompol-sentiment.qrel: QREL file for STOMPOL corpus test set

---

Organized by: