Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and professor for Software Engineering at Saarland University. His research on automated debugging, mining software archives, specification mining, and security testing has proven highly influential. Zeller is one of the few researchers to have received two ERC Advanced Grants, most recently for his S3 project. Zeller is an ACM Fellow and holds an ACM SIGSOFT Outstanding Research Award.
Phone: +49 681 87083-1001
Hosted on GitHub Pages — Theme by orderedlist
by Andreas Zeller
I compiled the following patterns for rebuttals (also known as author clarifications) for major software engineering conferences (ICSE, ESEC, FSE, ASE, ISSTA), having seen a number of rebuttals as PC chair of ESEC/FSE 2011 and having written a number of rebuttals for top conferences. These patterns may or may not be applicable in your context; use at your own risk.
Most SE conferences apply a process called Identify the Champion. Carefully read these process patterns, understanding the roles of the reviewers and the PC chair, and where your rebuttal can make a difference.
These are reviewers who may be swayed by your arguments. You can identify them (a) by generally liking your paper, but not the details, (b) because they provide their scores, or (c) because they explicitly state that they may be swayed. Focus your arguments on these reviewers. (And if you ever become a PC chair: include scores in the reviews, such that your authors know whom to focus upon.)
The champion likes your paper anyway, so don’t bother too much with her review. If there’s no potential champion, chances of getting your paper accepted are slim.
In the final discussion, your champion will need arguments against the detractors, especially strong detractors. Refute every single issue raised by the detractors to give your champion extra arguments for acceptance. Lower the confidence in the detractors’ reviews by pointing out mistakes.
A strong detractor can only be countered by a strong champion. Rather than trying to dissuade a strong detractor, your aim should be on arming the champion (see above).
Some conferences want you to only answer the specific questions the reviewers have asked, and/or instruct their PC members to only focus on these answers. Try to relate every argument of yours to a specific question.
If your reviewers have done a bad job, say so in the beginning, and the PC chair may assign you another reviewer. Stick to facts (review is just one paragraph, reviewer claims “no Foo” when Section 3 is named “All about Foo”, etc.) that the PC chair can check in less than a minute. If a reviewer completely misunderstood the paper, feel free to report this, but keep in mind it is your duty to ensure even a casual reader gets the message. If all reviewers misunderstood your paper, the problem is not the reviewers.
When your paper is discussed during the PC meeting, reviewers will glance through the reviews, and also read your rebuttal. If there’s one thing you want the committee to know, say so in the very beginning (because that’s all most reviewers will read). This assumes that your paper will be discussed, so focus on getting it discussed first.
Convince the reviewers that you’ll be able to improve the paper. If the reviewer asks: “Do you have a formal definition of Foo?”, don’t say “Yes”. But don’t say “Thanks! We’ll add this.” either. Say “A Foo is a Bar within a Qux range; we’ll add a detailed definition.” You don’t need to provide the full change; just sketch the change you will make.
Focus on major concerns - the ones where the reviewer assumes you may not be able to improve the paper. You do not need to convince the reviewer that you’re able to fix typos, straight-forward presentation issues, language issues, or anything else that can be fixed by simple proofreading. This is taken for granted.
Make it easy for reviewers to find their questions (and your arguments). Refer to reviewers by R1, R2, …; questions by R1Q1, R1Q2, … Use bullet points and short sentences to allow for quick scanning. Start with the most important and most common concerns, going down towards lesser issues.
If the limit is 500 words,
do_not_trick_the_process by_replacing_spaces_with_underscores or likewise; all that will happen is that the PC chair will delete your rebuttal. If it’s 2500 characters, do not assume that reviewers KAAYAYA and AYP because they FSITPOG (know all about you and your acronyms and accept your paper because they feel stupid in the presence of genius).
A rebuttal is not a place to vent off your fumes, even though your work is so great and the reviewers are all so stupid. Reviewing is hard work and even the least significant review contains grains of truth which will be helpful for your future work.
Your rebuttal will not save your paper. Your rebuttal will play a role only if your paper is on the edge, or if reviewers made very obvious mistakes. If the reviewers update their reviews to reply to your arguments, that’s already a lot. But regardless of the outcome, your rebuttal will help to improve your paper and eventually get it accepted - if not now, there’s always another deadline on the horizon.