Writing

• Question: What kind of tradeoffs did you make in allocating your time for reading and writing and actually doing research?

  Hamming: I believed, in my early days, that you should spend at least as much time in the polish and presentation as you did in the original research. Now at least 50% of the time must go for the presentation. It's a big, big number.
   

Practice

• When writing a paper, read books that are well-written, thinking in background mode about the syntactic mechanics. You'll find yourself absorbing the author's style.

  The major conferences choose prize papers on the basis of excellence both of content and presentation from among those accepted. Read them.
   

• Learning to write well requires doing a lot of it, over a period of years, and getting and taking seriously criticism of what you've written. There's no way to get dramatically better at it quickly.
  • Writing letters is good practice. Most technical papers would be improved if the style was more like a letter to a friend. Keeping a diary is also a way to practice writing (and lets you be more stylistically experimental than technical papers). Both practices have other substantial benefits.
  • If you put off writing until you've done all the work, you'll lose most of the benefit. Once you start working on a research project, it's a good idea to get into the habit of writing an informal paper explaining what you are up to and what you've learned every few months. Start with the contents of your research notebook. Take two days to write it-if it takes longer, you are being perfectionistic. This isn't something you are judged on; it's to share with your friends. Write DRAFT-NOT FOR CITATION on the cover. Make a dozen copies and give them to people who are likely to be interested (including your advisor!). This practice has most of the benefits of writing a formal paper (comments, clarity of thought, writing practice, and so forth), but on a smaller scale, and with much less work invested. Often, if your work goes well, these informal papers can be used later as the backbone of a more formal paper, from an AI Lab Working Paper to a journal article.
  • Continually update your:
    • Problem statement
    • Goals
    • Approach (or a list of possible approaches)
    • One-minute version of your research (aka the elevator ride summary)
    • Five-minute version of your research
       
• You will certainly suffer from writer's block at some point. Writer's block has many sources and no sure cure.
  • Perfectionism can lead to writer's block: whatever you start to write seems not good enough. Realize that writing is a debugging process. Write something sloppy first and go back and fix it up. Starting sloppy gets the ideas out and gets you into the flow. If you ``can't'' write text, write an outline. Make it more and more detailed until it's easy to write the subsubsubsections. If you find it really hard to be sloppy, try turning the contrast knob on your terminal all the way down so you can't see what you are writing. Type whatever comes into your head, even if it seems like garbage. After you've got a lot of text out, turn the knob back up and edit what you've written into something sensible.
  • Another mistake is to imagine that the whole thing can be written out in order. Usually you should start with the meat of the paper and write the introduction last, after you know what the paper really says. Another cause of writer's block is unrealistic expectations about how easy writing is. Writing is hard work and takes a long time; don't get frustrated and give up if you find you write only a page a day.
  • Perfectionism can also lead to endless repolishing of a perfectly adequate paper. This is a waste of time. (It can also be a way of semideliberately avoiding doing research.) Think of the paper you are writing as one statement in a conversation you are having with other people in the field. In a conversation not everything goes perfectly; few expect that what they say in a single utterance will be the whole story or last word in the interchange.
  • It's a common trap to spend more time hacking the formatter macro than the content. Avoid this. LaTeX is imperfect, but it has most of the macrology you want. If that's not enough, you can probably borrow code from someone else who has wanted to do the same thing. Most sites (including MIT) maintain a library of locally-written extensions.

• Know what you want to say. This is the hardest and most important factor in writing clearly.
  • Make it easy for the reader to find out what you've done. Put the sexy stuff up front, at all levels of organization from paragraph up to the whole paper. Carefully craft the abstract. Be sure it tells what your good idea is. Be sure you yourself know what it is! Then figure out how to say it in a few sentences. Too many abstracts tell what the paper is generally about and promise an idea without saying what it is.
  • After you have written a paper, delete the first paragraph or the first few sentences. You'll probably find that they were content-free generalities, and that a much better introductory sentence can be found at the end of the first paragraph of the beginning of the second.
  • Realize that your audience is almost guaranteed to be less familiar with your subject than you are. Explain your motivations, goals, and methodology clearly. Be repetitive without being boring, by presenting your ideas at several levels of abstraction, and by using examples to convey the ideas in a different way.
  • Write for people, not machines. It's not enough that your argument be correct, it has to be easy to follow. Don't rely on the reader to make any but the most obvious deductions. That you explained how the frobnitz worked in a footnote on page seven is not a justification when the reader gets confused by your introducing it without further explanation on page twenty-three. Formal papers are particularly hard to write clearly. Do not imitate math texts; their standard of elegance is to say as little as possible, and so to make the reader's job as hard as possible. This is not appropriate for AI.
  • Make sure your paper has an idea in it. If your program solves problem X in 10 ms, tell the reader why it's so fast. Don't just explain how your system is built and what it does, also explain why it works and why it's interesting.
  • If you write something clumsy and can't seem to fix it, probably you aren't sure what you really want to say. Once you know what to say, just say it.
  • Often you'll write a clause or sentence or paragraph that you know is bad, but you won't be able to find a way to fix it. This happens because you've worked yourself into a corner and no local choice can get you out. You have to back out and rewrite the whole passage. This happens less with practice.
  • Don't ``sell'' what you've done with big words or claims. Your readers are good people; honesty and self-respect suffice. Contrariwise, don't apologize for or cut down your own work.
• You should explain complex ideas as simply as you can. This principle applies to all expository writing, and your ability to carry it out is usually a function of how well you understand your subject matter. Keep asking your self, as you prepare your talk,
  • How can I make this concept simple?
  • How can I explain a new idea in terms of the old paradigms?
  • With what familiar terms can I describe this notion? Which parts of my presentation are sufficiently new that they require a new name?
  • Are the different components of my presentations---be they study groups, hardware modules, cows, or newt populations---sufficiently delineated? Are they named such that my audience will know which is which?
  • What analogies can I use to help people grasp this notion?
  • How can I tie this idea to everyday situations with which people are familiar?
  • What examples can I use to clarify meaning?

• Once you become part of the Secret Paper Passing Network, you'll find that people give you copies of draft papers that they want comments on. Getting comments on your papers is extremely valuable. You get people to take the time to write comments on yours by writing comments on theirs. So the more people's papers you write comments on, the more favors are owed you when you get around to writing one good politics. Moreover, learning to critique other people's papers will help your own writing.
   
• It will be helpful (to you and to the person whose paper you're reviewing) to organize comments on a paper in descending order of abstraction:
  • high-level content-oriented comments: High-level comments describing your overall impression of the paper, making suggestions for organization, presentation and alternative approaches to try, potential extensions, and relevant references are generally the most useful and the hardest to give.
  • mid-level stylistic and presentation comments
  • low-level nitpicky comments on syntax and grammar. Low-level comments are more appropriate for a paper that is being submitted for publication than for an unpublished paper such as a proposal or description of preliminary research.
     
• Writing useful comments on a paper is an art.
  • To write really useful comments, you need to read the paper twice, once to get the ideas, and the second time to mark up the presentation.
  • If someone is making the same mistake over and over, don't just mark it over and over. Try to figure out what the pattern is, why the person is doing it, and what they can do about it. Then explain this explicitly at length on the front page and/or in person.
  • The author, when incorporating your comments, will follow the line of least resistance, fixing only one word if possible, or if not then one phrase, or if not then one sentence. If some clumsiness in their text means that they have to back up to the paragraph level, or that they have to rethink the central theme of a whole section, or that the overall organization of the paper is wrong, say this in big letters so they can't ignore it.
  • Don't write destructive criticism like ``garbage'' on a paper. This contributes nothing to the author. Take the time to provide constructive suggestions. It's useful to think about how you would react to criticism of your own paper when providing it for others.
  • There are a variety of sorts of comments. There are comments on presentation and comments on content. Comments on presentation vary in scope. Copy-edits correct typos, punctuation, misspellings, missing words, and so forth. Learn the standard copy-editing symbols. You can also correct grammar, diction, verbosity, and muddied or unclear passages. Usually people who make grammatical mistakes do so consistently, using comma splices for example; take the time to explain the problem explicitly. Next there are organizational comments: ideas out of order at various scales from clauses through sentences and paragraphs to sections and chapters; redundancy; irrelevant content; missing arguments.

    Comments on content are harder to characterize. You may suggest extensions to the author's ideas, things to think about, errors, potential problems, expressions of admiration. ``You ought to read X because Y'' is always a useful comment.
     

• In requesting comments on a paper, you may wish to specify which sorts are most useful. For an early draft, you want mostly comments on content and organization; for a final draft, you want mostly comments on details of presentation. Be sure as a matter of courtesy to to run the paper through a spelling corrector before asking for comments.

  You don't have to take all the suggestions you get, but you should take them seriously. Cutting out parts of a paper is particularly painful, but usually improves it. Often if you find yourself resisting a suggestion it is because while it points out a genuine problem with your paper the solution suggested is unattractive. Look for a third alternative.

• Getting your papers published counts. This can be easier than it seems. Basically what reviewers for AI publications look for is a paper that (a) has something new to say and (b) is not broken in some way. If you look through an IJCAI proceedings, for example, you'll see that standards are surprisingly low. This is exacerbated by the inherent randomness of the reviewing process. So one heuristic for getting published is to keep trying. Here are some more:
  • Make sure it is readable. Papers are rejected because they are incomprehensible or ill-organized as often as because they don't have anything to say.
  • Circulate drafts for a while before sending it in to the journal. Get and incorporate comments. Resist the temptation to hurry a result into publication; there isn't much competition in AI, and publication delays will outweigh draft-commenting delays anyway.
  • Read some back issues of the journal or conference you are submitting to to make sure that the style and content of your paper are appropriate to it.
  • Most publications have an ``information for authors,'' a one page summary of what they want. Read it.
  • Papers get rejected-don't get dejected.
     
• If your paper is rejected, keep trying! Take the reviews to heart and try to rewrite the paper, addressing the reviewer's comments. You'll get more substantial and useful reviews from journals than conferences or workshops. Often a journal paper will be returned for revisions; usually a conference paper will just be accepted or rejected outright. After reading the review the first time, put it aside. Come back to it later, reading the paper closely to decide whether the criticisms were valid and how you can address them. You will often find that reviewers make criticisms that are off-target because they misinterpreted some aspect of your paper. If so, don't let it get to you -- just rewrite that part of your paper more clearly so that the same misunderstanding won't happen again. It's frustrating to have a paper rejected because of a misunderstanding, but at least it's something you can fix. On the other hand, criticisms of the content of the paper may require more substantial revisions -- rethinking your ideas, running more tests, or redoing an analysis.
   
• Like all else in research, paper writing always takes a lot longer than you expect. If you publish several forms of the paper, like a short conference version and a long journal version, this may go through several rounds. The result is that you are still working on a paper years after you thought you were through with it and after the whole topic has become utterly boring. This suggests a heuristic: don't do some piece of research you don't care for passionately on the grounds that it won't be hard to get a publication out of it: the pain will be worse than you expect.