What I Learned Reviewing for ACM CHI

Being selected as a reviewer for ACM CHI 2024 - the premier conference in Human-Computer Interaction - was both humbling and eye-opening. As an early-career researcher without a PhD, I wasn’t sure what to expect. Here’s what I learned.

The Review Process

CHI uses a rigorous multi-stage review process. For each paper, I had to:

  1. Read deeply - not just skim, but engage with every claim, method, and finding
  2. Assess contribution - Does this advance our understanding? Is it novel?
  3. Evaluate methodology - Are the user studies well-designed? Is the analysis sound?
  4. Provide constructive feedback - Even for rejected papers, the goal is to help authors improve

I reviewed papers, case studies, and alt.chi submissions - each with different expectations and evaluation criteria.

What Surprised Me

The diversity of methods. HCI papers use everything from controlled lab experiments to ethnographic fieldwork, from machine learning to speculative design. No single methodology dominates, and the best papers often combine approaches creatively.

How much I learned. Reviewing forced me to read work outside my comfort zone. I encountered research on accessible gaming, algorithmic auditing, and VR therapy - topics I wouldn’t have sought out on my own.

The responsibility. Behind every paper is months (or years) of someone’s work. Writing a thoughtful review is a responsibility I took seriously.

Advice for Early-Career Researchers

If you get the opportunity to review:

  • Say yes. It’s the best way to understand what makes a strong paper
  • Be specific. “This is weak” helps no one. Explain why and suggest alternatives
  • Be kind. You can be rigorous without being harsh
  • Read the review guidelines carefully. Each venue has its own expectations

Beyond CHI

After CHI, I also reviewed for ACM CUI ‘24, ACM DIS ‘24, and icSoftComp (2022–2024). Each conference taught me something different about the standards and expectations of different research communities.

Reviewing hasn’t just made me a better evaluator - it’s made me a better researcher. I now approach my own work with the same critical eye I apply to others’.