Last Updated: January 24th 2025.
One thing that was done in my alma mater and which I am trying to emulate in my current academic environment is a paper swap session. Different people, all working towards the same deadline, exchange papers and provide feedback on how to improve the paper, whether it’s making a certain argument clearer, bringing in additional literature, or how to make the paper more concise. Great for getting feedback on your papers and bonding with everyone else in your department.
Inevitably when organizing these sessions, someone (particularly young and junior academics) will ask:
But Abraham! I am but a novice in the world of academia. I have never given feedback on a paper; how do I go about this?
This is a fair question. My first response is to point them to the various resources that already exist, such as Ken Hinckley’s guide on how to give excellent reviews, or the CHI guide to reviews. These are extremely useful and insightful guides that I myself have used in the past. However, after rereading these guides, I realized there are two small issues. The first is that they are focused on how to review a paper rather than providing feedback on a paper: these two overlap but are distinct (feedback means providing helpful advice to a colleague on how to improve a paper; review means judging a paper for its academic merit and whether it is worthy of being published or not). But more than that, these guides had a lot of high-level feedback rather than low-level, on-the-ground advice. Advice such as have a positive attitude, give constructive feedback, and focus on the positives as well as the negatives is all good, but how does one know what feedback to give? What are the steps people should take to give feedback?
This blogpost provides a more granular, detailed guide for how to give feedback on a paper. This blog is primarily meant for those who are early in their academic journeys and those insecure on how to give feedback, but I hope everyone will benefit from this! Furthermore, this is written from the point of view of someone in the Human-Computer Interaction field; other fields and disciplines may have other ways of providing feedback.
The first thing to get out of the way is that a lot of first-time feedbackers can feel insecure or unsure of the feedback they provide. They may feel they do not know enough or are not experienced enough to provide useful insights into a paper. Let’s get this out of the way. If you have done, are doing, or have thought of doing research, your opinion on a paper is valid. You are the paper’s intended audience, and so any opinion of yours, positive or negative, is valid. Finally, even if your worst nightmares are true and your feedback is somehow worthless, you are giving feedback, not orders. It is up to the authors to decide whether to accept your feedback or not.
So get your confidence going, and get your feedbacker hat on, son, because it’s about to get serious!
The first step is to read the paper from start to finish without looking out for flaws or critiques. Just read it. Then, summarize the paper in 2-3 sentences.
This is useful because it helps you get the full context of the paper, as well as serving as a check on yourself. Did you understand the paper? Or have you not understood what the paper is about after reading it (which can be a point of feedback for the authors: ideally, a paper should be clear enough so that a simple skim gives the reader a rough idea of what the paper is trying to do).
Moreover, if you then give this summary to the authors, it can be helfpul for them too. By summarizing what you think the paper is about, the authors can contextualize the feedback accordingly, and know what areas to tackle. For example, if they conducted an ethnography, but my summary says, “The authors ran an experiment,” then clearly there is some mistranslation going on, and the authors need to make their method clearer.
The next step is to go through the paper and give feedback on it. Ok this is a bit too broad. Basically, you want to write down aspects of the paper you liked and aspects that you didn’t like. Feedback is in the eye of the beholder: anything you want to mention, you can (and should!), ranging from high level “what is the point of the paper” to low level feedback like the number of typos and such.
For those seeking a bit more guidance, the way I think about this is there are four main categories of feedback you can give: (1) writing; (2) prior work; (3) logic / argumentation; and (4) contribution. For each area of feedback, I give several questions you should ask yourself (and answer!) when reviewing.
This part concerns itself with the clarity of the writing. The questions you should ask yourself are:
Write a few sentences on these prompts. For this (and other prompts), it is important to mention not only the bad things but also the good things. This lets the authors know what they are doing well and should continue doing.
For guidance:
Prior work refers to how well the authors talk about work that was done before. Research is meant to build on top of each other, so a good paper explains what this other work is and how the current paper builds on top of it.
Write a few sentences on these prompts:
For guidance:
Next up is logic, or argumentation. All academic papers are basically full of logical arguments. The world has this problem: therefore, it is important to address it. We wanted to answer this research question, so we used this method. We got this data, and thus, we made these recommendations.
For the arguments the authors make, ask yourself:
For guidance:
The final area of feedback is contribution. This might be the most complicated one to gauge (CHI even wrote a whole blogpost about the different types of HCI contributions, but the way I see it, contribution is showing that the paper is important and that it helps improve the world by existing, whether it is because it adds knowledge, insights, or somehow improves humanity.
After reading the paper, write a few sentences on these prompts:
For guidance:
If those areas aren’t enough, here are a few additional areas of feedback you can comment on.
An advanced tips is to think about the context of a paper. By ‘context’, I mean things beyond what is written in the paper: I am talking about who is writing the paper, where they are writing to, and so on and so forth. Here are three examples of how to take context into account when giving feedback:
Whenever someone asks me to provide feedback on one of their papers, I always counter-ask: what type of feedback do you want me to give? The reason I say this is because not all papers ask or require the same type of feedback. In some cases, the authors already have finished product and they just want to cut down on space. In a different paper, the authors may be concerned about motivating their problem because they are writing to a hostile audience who they are not sure is convinced by the importance of their research questions. Once you know what type of feedback the authors want, it can help you drill down on the specific areas they are asking about, meaning you can provide more focused feedback (as well as saving you some bandwidth by focusing on a few areas!). Similarly, I also ask: what is the preferred method for receiving feedback? Some people like comments in a pdf file. Others like comments in an Overleaf document or Google Docs. Some people prefer a written report. And finally, some people may just want a quick 3-minute conversation. Your goal as a feedbacker is to help these authors improve their paper, and therefore, knowing what type of feedback they would like to have and in what format helps you help them improve their paper. Of course, you may have your own preferences as a feedbacker: if you despise the Overleaf interface and hate making comments there, you shouldn’t have to inconvenience yourself just for their sake (you are the one doing them a favor after all). But if you are indifferent to the format you can give, then something as simple as changing the medium through which the feedback is delivered can make a huge difference. Finally, you can ask about the tone of feedback the authors want. Some people prefer brutal, honest, and aggressive feedback (“Give it to me straight, doc”). Others prefer gentler, more encouraging feedback. Neither method is inherently better than the other one, but changing the tone of your feedback can vastly impact how the feedback is received and whether the authors leave with a sense of motivation and ambition or whether they are demotivated, even if the piece of advice being transmitted is the same. So, contact the authors and figure out what type of feedback they prefer.
Another advanced tip when giving feedback is to ask yourself whether the authors are capable of implementing the feedback you give. Take the following two scenarios:
Obviously, given the difference in both ability and the amount of time left for the deadline, there are changes that can be implemented in Situation (A) that cannot be implemented in Situation (B) (e.g., collecting additional data or redoing the entire analysis). Therefore, if the goal is to help improve a paper in time for the submission deadline, you might consider tailoring your feedback accordingly. It does not make sense to give people feedback that they cannot reasonably implement in time; if anything, it can be counterproductive because (a) that is more feedback the authors have to sift through, and (b) the authors are stressing out about flaws of their paper that they don’t have time to address. If your goal is to prevent the person from submitting the paper at all, then sure, include that feedback. But if you do want them to submit the paper, and the feedback isn’t mission-critical, maybe focus on small-level changes and leave the big-level comments for a point where the authors are more relaxed and can address the feedback. Feedback is not neutral. While true that the ultimate responsibility lies with the author as to whether they accept your feedback or not, unaddressable feedback can demotivate someone, especially if it comes from someone in a position of power or authority over the authors.
Another way to incorporate context is by providing the author’s information as to how crucial or important you think your feedback is. Some feedback is very important: it makes a paper unreadable if not addressed. Other feedback is more of an improvement, where having it would be better, but not having it is fine too. By letting the authors know what feedback you are giving is a big deal versus which one is not, it lets them know which ones they need to address versus which ones they can comfortably ignore: plus, in case the authors do not have enough time to address all of your points before a submission, it helps them prioritize feedback accordingly.
I myself use a 1-2-3-4 system, as follows:
Feedback with a 1 is mission-critical feedback. If I was a reviewer, and I saw this flaw, I would reject the paper. For example, you did not properly anonymize participant quotes, and I can easily identify who said what.
Feedback with a 2 is important feedback. Doing this will fix the paper, but it isn’t mission-critical. For example, the findings section is confusing to read, and it would benefit from a reorganizing for clarity.
Feedback with a 3 is my own opinion. It is my own preferred style of doing things, but it wouldn’t sway my opinion on the quality of the paper, and the authors can easily ignore this. For example, when reporting the results of an interview study, I like highlighting quotes in italics so they stand out in the paper.
Feedback with a 4 is additional notes that isn’t necessarily feedback or something to do with the paper, but rather, miscellaneous thoughts that appeared after reading the paper. For example, “This makes me think of a cool follow-up study involving squirrels and quantum computing.”
I created a paper guide, or template, for providing feedback. It can be found here.
Disclaimer: This post was partially edited via Grammarly for typos and small mistakes