Sobre el taller

About the workshop

Negation is a complex linguistic phenomenon that has been extensively studied from a theoretical perspective but not practical. Detection and treatment of negation is relevant in a wide range of applications such as information extraction, machine translation or sentiment analysis where it is crucial to know when a part of the text should have a different meaning due to the presence of negation. This part of the text is what is known as the scope. Negation detection is becoming an important task in Natural Language Processing. In fact, in recent years, several challenges and shared tasks have included the extraction of this element. However, most of the research on negation has been done for English. Therefore, this workshop aims to advance the study of this phenomenon in Spanish, the second most widely spoken language in the world and the third most widely used on the Internet. The main objective is to bring together the scientific community that is working on this issue to discuss how this phenomenon is being addressed, what are the main problems encountered, as well as sharing resources and tools aimed at negation in Spanish.

Tasks

Task 1: Annotation guidelines

The objective of this task is to reach an agreement on the guidelines to follow for the annotation of negation. The corpora annotated so far in Spanish belong to 3 domains (news, clinical reports and product reviews) and are based on different guidelines. In this task, the guides used for the annotation of the corpora are available below in order to the participants can analyze them. Once the analysis period has elapsed, participants must send a document indicating which aspects of the guidelines they agree with and which they do not, all duly justified. This information will be sent to the participants, prior to the workshop, so that during the workshop we can discuss the aspects of interest and reach a consensus.

Corpus: UAM Spanish TreeBank. Domain: News.
Annotation guidelines (pages 51-55): http://www.lllf.uam.es/ESP/pdf/Moreno&Garrote_RIL8_2013.pdf
Reference: Sandoval, A. M., & Salazar, M. G. (2013). La anotación de la negación en un corpus escrito etiquetado sintácticamente. Revista Iberoamericana de Lingüística: RIL, (8), 45-60.

Corpus: IxaMed-GS. Domain: Clinical reports.
Annotation guidelines (page 322): https://doi.org/10.1016/j.jbi.2015.06.016
Reference: Oronoz, M., Gojenola, K., Pérez, A., de Ilarraza, A. D., & Casillas, A. (2015). On the creation of a clinical gold standard corpus in spanish: Mining adverse drug reactions. Journal of biomedical informatics, 56, 318-332.

Corpus: SFU ReviewSP-NEG. Domain: Product reviews.
Annotation guidelines (pages 538-559): https://doi.org/10.1007/s10579-017-9391-x
Reference: Jiménez-Zafra, S. M., Taulé, M., Martín-Valdivia, M. T., Ureña-López, L. A., & Martí, M. A. (2018). SFU Review SP-NEG: a Spanish corpus annotated with negation for sentiment analysis. A typology of negation patterns. Language Resources and Evaluation, 52(2), 533-569. First published online May 22, 2017.

Corpus: UHU-HUVR. Domain: Clinical reports.
Annotation guidelines (pages 54-57): http://www.aclweb.org/anthology/W17-1808
Reference: Cruz, N., Morante, R., López, M. J. M., Vázquez, J. M., & Calderón, C. L. P. (2017). Annotating Negation in Spanish Clinical Texts. In Proceedings of the Workshop Computational Semantics Beyond Events and Roles (pp. 53-58).

Corpus: IULA Spanish Clinical Record. Domain: Clinical reports.
Annotation guidelines (pages 45-49): http://www.aclweb.org/anthology/W17-1807
Reference: Marimon, M., Vivaldi, J., & Bel, N. (2017). Annotation of negation in the IULA Spanish Clinical Record Corpus. In Proceedings of the Workshop Computational Semantics Beyond Events and Roles (pp. 43-52).

Submission

Participants must send a document in Word format in which they indicate which aspects of the guides they agree with and which they do not, all duly justified. The document must include the names and affiliations of all the participants and the file must be named as follows, team_task1.docx. To submit the file, participants login here.

Task 2: Negation cues detection

The aim of this task is to advance in the identification of negation cues automatically. To do this, the participants must develop a system able to identify all the negation cues present in a document. The SFU ReviewSP-NEG corpus (Jiménez-Zafra et al., 2017) will be used to train and test the systems.

Jiménez-Zafra, S. M., Taulé, M., Martín-Valdivia, M. T., Ureña-López, L. A., & Martí, M. A. (2018). SFU Review SP-NEG: a Spanish corpus annotated with negation for sentiment analysis. A typology of negation patterns. Language Resources and Evaluation, 52(2), 533-569. First published online May 22, 2017. https://doi.org/10.1007/s10579-017-9391-x

Dataset

You must have an account in order to download the dataset. If you don't have an account, go to register form. If you have an account, login here.

Submission

Submitted files must have the format specified in the “OUTPUT” section of the README file provided with the dataset and they must be named as follows, team_domain_task2.txt. To submit the files, participants must login here.


Task 3: Sentiment analysis

It is a specific task whose objective is to evaluate the role of negation in sentiment analysis. In this case, participants should develop a system that uses the negation information contained in the SFU ReviewSP-NEG corpus (Jiménez-Zafra et al., 2017) to improve the task of polarity classification.

Jiménez-Zafra, S. M., Taulé, M., Martín-Valdivia, M. T., Ureña-López, L. A., & Martí, M. A. (2018). SFU Review SP-NEG: a Spanish corpus annotated with negation for sentiment analysis. A typology of negation patterns. Language Resources and Evaluation, 52(2), 533-569. First published online May 22, 2017. https://doi.org/10.1007/s10579-017-9391-x

Dataset

You must have an account in order to download the dataset. If you don't have an account, go to register form. If you have an account, login here.

Submission

Submitted files must have the format specified in the “OUTPUT FORMAT” section of the README file provided with the dataset and they must be named as follows, team_task3.txt. To submit the files, participants must login here.

Registration

NOTE: Submitting this form only implies registration in the workshop tasks. To attend it, the registration must be made through the space provided for that purpose on the page of the XXXIV International Congress of the Spanish Society for Natural Language Processing (http://www.sepln2018.com/).

Task 1: Annotation guidelines
Task 2: Negation cues detection
Task 3: Sentiment analysis

Schedule

  • April 20, 2018

    Registration open

  • May 14, 2018

    Training and development sets released

  • June 13, 2018

    Test set released

  • July 6, 2018 - 23:59 UTC+2

    End of system submission

  • July 9, 2018

    Results posted

  • July 16, 2018 - 23:59 UTC+2

    Deadline for paper submission

  • July 20, 2018

    Notification of acceptance

  • July 27, 2018 - 23:59 UTC+2

    Deadline for camera ready paper submission

  • September 18, 2018

    Workshop NEGES (SEPLN 2018, Sevilla)

  • Registration

  • Welcome and presentation of the workshop

  • Presentation of works: Task 2

    Task 2: Negation cues detection
  • Presentation of works: Task 3

    Task 3: Sentiment analysis
  • Coffee Break

  • Panel discussion Task 1

    Task 1: Annotation guidelines
  • Conclusions and closure

Papers format

Participants can send a document for each task with the description of their system. Submitted papers must follow SEPLN format (LaTeX or Word) and must have at least 4 pages (plus references) and a maximum of 8 pages (plus references). The papers presented will be reviewed by a scientific committee and those accepted will be published in CEUR.
To submit the papers, participants must login here.

Organizing committee

speaker img

Salud María Jiménez Zafra

Universidad de Jaén

speaker img

Maite Martín Valdivia

Universidad de Jaén

speaker img

Noa Cruz Díaz

Savana Médica, Madrid

speaker img

Roser Morante

VU Amsterdam

Program committee

  • Altuna, Begoña (Unversidad del País Vasco)
  • Bel, Nuria (Universitat Pompeu Fabra, España)
  • Blanco, Eduardo (University of North Texas, USA)
  • Carrillo de Albornoz, Jorge (UNED, España)
  • Casillas, Arantza (Universidad del País Vasco, España)
  • Cruz Díaz, Noa (Savana Médica, Madrid, España)
  • Fernández-Gavilanes, Milagros (Universidad de Vigo, España)
  • García Méndez, Silvia (Universidad de Vigo, España)
  • Jiménez Zafra, Salud María (Universidad de Jaén, España)
  • Maña, Manuel (Universidad de Huelva, España)
  • Marimón, Montserrat (Universitat Pompeu Fabra, España)
  • Martí, M. Antonia (Universitat de Barcelona, España)
  • Martín Valdivia, Maite (Universidad de Jaén, España)
  • Martínez, Paloma (Universidad Carlos III, Madrid, España)
  • Martínez Barco, Patricio (Universidad de Alicante, España)
  • Martínez-Cámara, Eugenio (Universidad de Granada, España)
  • Morante, Roser (VU Amsterdam, Países Bajos)
  • Segura, Isabel (Universidad Carlos III, Madrid, España)
  • Taboada, Maite (Simon Fraser University, Canadá)
  • Taulé, Mariona (Universitat de Barcelona, España)
  • Vilares, David (Universidad de la Coruña, España)
  • Vivaldi, Jorge (Universitat Pompeu Fabra, España)

Hospital Universitario Virgen del Rocío

Av. Manuel Siurot, s/n, 41013 Sevilla