Spring 2009

Clay Shields

front | classes | research | personal | contact

How to Read, Review, and Present Research Papers in Class

back to class page

In this class, we will be reading a large number of network security research papers. These are papers that each attempt to address some problem, presenting a possible solution, backed by analysis, simulation, or experimentation.

Academic researchers write these papers to gain fame, fortune, grants, promotion, and sometimes tenure. The papers are submitted to workshops and conferences, where they are reviewed. Papers are returned with reviewer comments, and the ones that are accepted have small changes made before publication. Articles in journals go through a longer process, in which referees make comments on the articles and the authors respond, either by clarifying the paper or by doing additional work to address the referees concerns. Sometimes, work from multiple conference papers are combined into a larger journal article.

You will be reading these papers to understand both what constitute topics of interest to researchers, how to conduct research, and how to present research results. You will probably pick up some knowledge of network security as well.

How to approach reading papers:

At first, reading papers can be intimidating. While authors usually try and provide some background for readers, space limitations often keep these sections small. Because of this, it may seem that the authors are speaking their own private language. Unfortunately, that is often the case, as the authors are generally experts in their area and assume that most of the readers are too. The best way to become an expert is to read more papers. You can also ask for explanations of concepts from people who have been exposed to them already. I am happy to support you in that, so feel free to ask me.

You initial reading of papers should be to understand the technical material. Most academics don't sit and read the paper all the way through, at first. Instead, they read the abstract, the introduction, and the conclusions. They look at any graphs or pictures and the captions to see what information is being presented. This gives the overview of the paper. In particular, it gives insight into these questions:

  • What problem are the authors trying to solve?
  • What is their approach to solving it?
  • How are they demonstrating that their approach works?
  • What do the authors claim is new in their work?

These are the large questions that you will be addressing. (See the paper by Phillip Fong in handed out in class for more a more detailed description of how to read CS research papers.)

Most academics then go back and read the entire paper. During this more thorough reading, they will be asking themselves these deeper questions:

  • What assumptions did the authors make in writing the paper (these are often not explicit, and can be difficult to find)? Are they reasonable assumptions? Is there a way to address the same problem with different assumptions?
  • Is the supporting evidence for the work strong or weak? How could it be improved?
  • What problems are introduced or left unsolved?
  • Can the techniques used be applied to other problems?

Presenting papers in class:

In presenting papers in class, I will expect you to address the questions raised above. Format and neatness doesn't count as much compared to critical investigation, so spend your time in analysis and not on power point. A chalkboard talk is fine. The goal is for you to lead a discussion of the paper in class, for 20-30 minutes. I would expect the general presentation to go along these lines (though of course it depends on the paper):

  • Overview of problem.
    • Why it is an interesting/worthy topic to write about
    • Overview of the authors approach
    • Description of results

  • Critical analysis of the work
    • What assumptions did they make?
    • What if you change the assumptions, does the work stand? If not, what could you do under a different set of assumptions?
    • Does the evidence support the work? If not, why not? What would you do to make the evidence stronger?
    • What problems are left, or what problems are raised?
    • Do they introduce techniques that are applicable to other problems?

For this class, we are also interested in a few other questions, as applicable. For papers about problems or attacks, these questions include:

  • What happened to make the attack possible?
  • What principles of security were violated to make the attack possible?
  • How do we prevent similar problems in the future?
  • What are the trade-offs in providing a solution?

In terms of grading, I will rate your paper presentation on a 4 point scale - (no show, fair, good, great). If you have questions about a particular paper, don't hesitate to ask me in advance. You might also look online for papers that cite your paper, to see what critique other researchers have made. Just be clear when you have done so.