Call for Papers

Overview

The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) invites the submission of long and short papers on substantial, original, and unpublished research in empirical methods for Natural Language Processing. As in recent years, some of the presentations at the conference will be for papers accepted by the Transactions of the ACL (TACL) and Computational Linguistics (CL) journals.

Important Dates

  • Anonymity period begins: May 1, 2020
  • Submission deadline (long & short papers): June 3, 2020
  • Author response period: August 7-13, 2020
  • Notification of acceptance (long & short papers): September 14, 2020
  • Camera-ready papers due (long & short papers): October 5, 2020
  • Main conference: November 16-18, 2020
  • Workshops and tutorials: November 19-20, 2020

All deadlines are 11.59 pm UTC -12h ("anywhere on Earth").

Submissions

EMNLP 2020 has the goal of a broad technical program. Relevant topics for the conference include, but are not limited to, the following areas (in alphabetical order):

  • Computational Social Science and Social Media
  • Dialogue and Interactive Systems
  • Discourse and Pragmatics
  • Generation
  • Information Extraction
  • Information Retrieval and Text Mining
  • Interpretability and Analysis of Models for NLP
  • Language Grounding to Vision, Robotics and Beyond
  • Linguistic Theories, Cognitive Modeling and Psycholinguistics
  • Machine Learning for NLP
  • Machine Translation and Multilinguality
  • NLP Applications
  • Phonology, Morphology and Word Segmentation
  • Question Answering
  • Resources and Evaluation
  • Semantics: Lexical, Sentence level, Textual Inference and Other areas
  • Sentiment Analysis, Stylistic Analysis, and Argument Mining
  • Speech and Multimodality
  • Summarization
  • Syntax: Tagging, Chunking and Parsing

Paper Submission Information

Long Papers

Long paper submissions must describe substantial, original, completed and unpublished work. Wherever appropriate, concrete evaluation and analysis should be included. Review forms will be made available prior to the deadlines. Long papers may consist of up to 8 pages of content, plus unlimited pages for references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account.

Long papers will be presented orally or as posters as determined by the program committee. The decisions as to which papers will be presented orally and which as poster presentations will be based on the nature rather than the quality of the work. There will be no distinction in the proceedings between long papers presented orally and as posters.

Short Papers

Short paper submissions must describe original and unpublished work. Please note that a short paper is not a shortened long paper. Instead short papers should have a point that can be made in a few pages. Some kinds of short papers are:

  • A small, focused contribution
  • A negative result
  • An opinion piece
  • An interesting application nugget

Short papers may consist of up to 4 pages of content, plus unlimited references. Upon acceptance, short papers will be given 5 content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions.

Short papers will be presented orally or as posters as determined by the program committee. While short papers will be distinguished from long papers in the proceedings, there will be no distinction in the proceedings between short papers presented orally and as posters.

Authorship

The author list for submissions should include all (and only) individuals who made substantial contributions to the work presented. Each author listed on a submission to EMNLP 2020 will be notified of submissions, revisions and the final decision. No changes to the order or composition of authorship may be made to submissions to EMNLP 2020 after the paper submission deadline.

Citation and Comparison

You are expected to cite all refereed publications relevant to your submission, but you may be excused for not knowing about all unpublished work (especially work that has been recently posted and/or is not widely cited).

In cases where a preprint has been superseded by a refereed publication, the refereed publication should be cited instead of the preprint version.

Papers (whether refereed or not) appearing less than 3 months before the submission deadline are considered contemporaneous to your submission, and you are therefore not obliged to make detailed comparisons that require additional experimentation and/or in-depth analysis.

For more information, see the ACL Policies for Submission, Review, and Citation

Multiple Submission Policy

EMNLP 2020 will not consider any paper that is under review in a journal or another conference at the time of submission, and submitted papers must not be submitted elsewhere during the EMNLP 2020 review period. This policy covers all refereed and archival conferences and workshops (e.g., COLING, NeurIPS, ACL workshops). For example, a paper under review at an ACL workshop cannot be dual-submitted to EMNLP 2020. We make an exception to the above:

  • papers can be dual-submitted to both EMNLP 2020 and an EMNLP workshop which has its submission deadline falling after our original notification date of August 8, 2020;

In addition, we will not consider any paper that overlaps significantly in content or results with papers that will be (or have been) published elsewhere. Authors submitting more than one paper to EMNLP 2020 must ensure that their submissions do not overlap significantly (>25%) with each other in content or results.

NEW: Ethics Policy

Authors are required to honour the ethical code set out in the ACM Code of Ethics. The consideration of the ethical impact of our research, use of data, and potential applications of our work has always been an important consideration, and as artificial intelligence is becoming more mainstream, these issues are increasingly pertinent. We ask that all authors read the code, and ensure that their work is conformant to this code. Where a paper may raise ethical issues, we ask that you include in the paper an explicit discussion of these issues, which will be taken into account in the review process. We reserve the right to reject papers on ethical grounds, where the authors are judged to have operated counter to the code of ethics, or have inadequately addressed legitimate ethical concerns with their work

Paper Submission and Templates

Submission is electronic, using the Softconf START conference management system. Both long and short papers must follow the EMNLP 2020 two-column format, using the supplied official style sheets. Please do not modify these style files, nor should you use templates designed for other conferences. Submissions that do not conform to the required styles, including paper size, margin width, and font size restrictions, will be rejected without review.

Optional Supplementary Materials: Appendices, Software and Data

Each EMNLP 2020 submission can be accompanied by one PDF appendix for the paper, one PDF for prior reviews and author response, one .tgz or .zip archive containing software, and one.tgz or .zip archive containing data. EMNLP 2020 encourages the submission of these supplementary materials to improve the reproducibility of results, and to enable authors to provide additional information that does not fit in the paper. For example, anonymised related work (see above), preprocessing decisions, model parameters, feature templates, lengthy proofs or derivations, pseudocode, sample system inputs/outputs, and other details that are necessary for the exact replication of the work described in the paper can be put into the appendix. However, the paper submissions need to remain fully self-contained, as these supplementary materials are completely optional, and reviewers are not even asked to review or download them. If the pseudo-code or derivations or model specifications are an important part of the contribution, or if they are important for the reviewers to assess the technical correctness of the work, they should be a part of the main paper, and not appear in the appendix. Supplementary materials need to be fully anonymized to preserve the double-blind reviewing policy.

Anonymity Period

The following rules and guidelines are meant to protect the integrity of double-blind review and ensure that submissions are reviewed fairly. The rules make reference to the anonymity period, which runs from 1 month before the submission deadline (starting May 1st, 2020) up to the date when your paper is accepted or rejected (September 14th, 2020). Papers that are withdrawn during this period will no longer be subject to these rules.

  • You may not make a non-anonymized version of your paper available online to the general community (for example, via a preprint server) during the anonymity period. Versions of the paper include papers having essentially the same scientific content but possibly differing in minor details (including title and structure) and/or in length.
  • If you have posted a non-anonymized version of your paper online before the start of the anonymity period, you may submit an anonymized version to the conference. The submitted version must not refer to the non-anonymized version, and you must inform the programme chairs that a non-anonymized version exists.
  • You may not update the non-anonymized version during the anonymity period, and we ask you not to advertise it on social media or take other actions that would further compromise double-blind reviewing during the anonymity period.
  • You may make an anonymized version of your paper available (for example, on OpenReview), even during the anonymity period.
  • Note that, while you are not prohibited from making a non-anonymous version available online before the start of the anonymity period, this does make double-blind reviewing more difficult to maintain, and we therefore encourage you to wait until the end of the anonymity period. Alternatively, you may consider submitting your work to the Computational Linguistics journal, which does not require anonymization and has a track for “short” (i.e., conference-length) papers.

Update, April 29, 2020: We are making an exception to EMNLP’s anonymity period policy for papers related to Covid-19. If authors deem their research is highly timely and need to share their results with the Covid-19 global research community during the EMNLP anonymity period (that is, starting May 1st until paper notification Sep 14, 2020), we allow authors to put papers on Arxiv or share in other forms during this period. However, authors need to email EMNLP Program Chairs (emnlp2020programmechairs@gmail.com) to get approval in advance that their papers are exempt from the conference’s anonymity policy. Reviewers for these papers will know that the papers are not violating the policy.

Instructions For Double-Blind Review

As reviewing will be double blind, papers must not include authors’ names and affiliations. Furthermore, self-references or links (such as github) that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …” must be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …” Papers that do not conform to these requirements will be rejected without review.

Papers should not refer, for further detail, to documents that are not available to the reviewers. For example, do not omit or redact important citation information to preserve anonymity. Instead, use third person or named reference to this work, as described above (“Smith showed” rather than “we showed”). If important citations are not available to reviewers (e.g., awaiting publication), these paper/s should be anonymised and included in the appendix. They can then be referenced from the submission without compromising anonymity.

Papers may be accompanied by a resource (software and/or data) described in the paper, but these resources should also be anonymized.

NEW: Sticky Reviews (optional)

Authors resubmitting a paper that has been rejected from another venue are invited to submit alongside their paper the previous version of the paper, the reviews and an author response. This is strictly optional. It is designed to mimic the revise-and-resubmit procedure underlying journals like TACL, and this trial for EMNLP will help to inform potential changes to the review process under consideration for future EMNLP and ACL conferences. We expect that the fact that a paper was rejected from another venue will not necessarily affect the paper’s decision in a negative way, but is likely to be beneficial to authors who believe they have addressed the problems identified, and can argue strongly for how the paper has been improved. The prior reviews will not be seen by reviewers, but be used as part of the EMNLP decision process, primarily by area chairs and program chairs in review quality control, resolving disagreements between reviewers, and in deciding borderline papers.

NEW: Reproducibility Criteria

To foster reproducibility, authors will be asked to answer all questions from the following Reproducibility Checklist during the submission process. Authors are not required to meet all criteria on the checklist, but rather check off the criteria relevant to their submission. The answers will be made available to the reviewers to help them evaluate the submission. Reviewers will be expressly asked to assess the reproducibility of the work as part of their reviews.

The following list is a preliminary checklist we will use.

For all reported experimental results:

  • [ ] A clear description of the mathematical setting, algorithm, and/or model.
  • [ ] A link to a downloadable source code, with specification of all dependencies, including external libraries
  • [ ] Description of computing infrastructure used
  • [ ] Average runtime for each approach
  • [ ] Number of parameters in each model
  • [ ] Corresponding validation performance for each reported test result
  • [ ] Explanation of evaluation metrics used, with links to code

For all experiments with hyperparameter search:

  • [ ] Bounds for each hyperparameter
  • [ ] Hyperparameter configurations for best-performing models
  • [ ] Number of hyperparameter search trials
  • [ ] The method of choosing hyperparameter values (e.g., uniform sampling, manual tuning, etc.) and the criterion used to select among them (e.g., accuracy)
  • [ ] Expected validation performance, as introduced in Section 3.1 in * Dodge et al, 2019, or another measure of the mean and variance as a function of the number of hyperparameter trials.

For all datasets used:

  • [ ] Relevant statistics such as number of examples
  • [ ] Details of train/validation/test splits
  • [ ] Explanation of any data that were excluded, and all pre-processing steps
  • [ ] A link to a downloadable version of the data
  • [ ] For new data collected, a complete description of the data collection process, such as instructions to annotators and methods for quality control.

Thanks to Jesse Dodge for helping with the above checklist. It is based on Dodge et al, 2019 and Joelle Pineau's reproducibility checklist

Presentation Requirement

All accepted papers must be presented at the conference to appear in the proceedings. Authors of papers accepted for presentation at EMNLP 2020 must notify the program chairs by the camera-ready deadline if they wish to withdraw the paper.

Previous presentations of the work (e.g. preprints on arXiv.org) should be indicated in a footnote in the final version of papers appearing in the EMNLP 2020 proceedings. Please note that this footnote should not be in the submission version of the paper.

At least one author of each accepted paper must register for EMNLP 2020 by the early registration deadline.

Reviewing Policy

Please refer to our reviewing policy. In the paper submission form, we ask you to provide the usernames of authors that have signed up already as reviewers or that you are nominating as reviewers at submission time. If none of the authors can review, please let program chairs know your reasons. Papers without such information may be rejected without review.