Guide to Reviewing Long Papers

(This text is adapted from the CHI 2023 and 2024 reviewing guides to suit the needs of OzCHI.)

Key points

Your primary criterion for judging a paper is: Does this submission provide a strong contribution to the field of HCI? Remember that there are many ways a paper can make a contribution to HCI, and you should review the paper appropriately.

Reviewers make an overall recommendation (a 5-1 rating from 'Definite Accept' to 'Definite Reject') on each submission, and provide a written appraisal that matches their recommendation.

A high-quality review is typically about a page of written text; very short reviews are frustrating for authors, tend to be less constructive, and don't adequately support the review process. Always put yourself in the author's position: what level of detailed feedback would you like to see for your own work?

We encourage supervisors to mentor students who are new to reviewing.

All reviewers are encouraged to read Ken Hinckley's essay on reviewing in HCI.

As a reviewer, you are responsible for the content and accuracy of your reviews, including the references cited. This is especially relevant if you choose to use AI tools based on Large-scale Language Models (LLM) such as ChatGPT, which are prone to producing plausible sounding garbage. Such reviews can negatively reflect on the reviewer and their reputation.

Maintaining confidentiality is crucial in the review process; thus quoting or disclosing papers under review must be avoided. Copying and pasting content from a manuscript into an online LLM risks breaching confidentiality.

Please address these issues in your review:

• Originality: what new ideas or approaches are introduced? A paper must make a clear contribution to Human-Computer Interaction.
• Significance: will others gain from the paper? Why does the contribution matter?
• Rigor: were the methods sufficiently rigorous to support the paper's argument?
• Validity: how confidently can researchers and practitioners use the results?
• Previous work: is prior work adequately acknowledged and reviewed?
• Presentation: how well is the research framed, and is the argument clear throughout the paper? Is the length of the paper commensurate with its contribution?

Please consider making any other recommendations that you think might be of use to the author(s).