Champion Sponsors

What’s been provisionally accepted?

In this blog we consider the big picture from the CHI 2019 Papers track. We look at what’s been accepted, what hasn’t. We also consider how things went with reviewing and look at some author-level data. We will analyze the effect of rebuttals in this cycle in another blog post.


This year, 2958 papers were left in the system for review. Of these, 705 have been provisionally accepted for the Papers track at CHI 2019. That is a 23.8% acceptance rate, down from 25.7% last year. Eighty-three of these accepts are being shepherded*. The overall mean score for all submissions, including desk and quick rejects, was 2.52 (SD=0.35).

As in previous years, a significant proportion of papers score between 2.0 and 3.0 (inclusive) – this year more than half (1,494 submissions, 50.5%).  Figure 1 shows the distribution of accepts by score. The figure collapses accepted and shepherded papers into ‘A’s and all other decisions (including withdrawal) into ‘R’s. Accepted papers had a mean score of 3.66 (Range 2.38-4.88). This was lower than last year’s mean of 3.73. Forty-two papers with mean scores lower than 3.0 have been accepted (which is precisely double last year’s 21).

The highest-scoring reject had an average of 3.62. There were 117 submissions rejected with a score of ≥3.0 (but only 65 >3.0). There were 224 submissions that scored >2.8 and <3.2. Of these, 70 were accepted. Submissions averaging >3.25 (n=587) were extremely likely to be accepted (97.1%).

Histogram showing the distribution of scores, grouped by decision to accept or reject.
Figure 1. Distribution of scores, grouped by decision.


A total of 2799 external reviewers completed 5433 external reviews (Mdn=1, M=1.94). As you will see in the next section, this means that authors outnumber reviewers just under 3:1. The General Chairs, TPCs and Papers Chairs thank reviewers for this absolutely essential contribution. No reviews means no accepts, no accepts means no conference.

The most reviews completed by an external reviewer was 12. Five or more reviews were completed by 189 reviewers. The distribution of completed reviews by a given reviewer is illustrated in Figure 2.

Histogram showing number of reviews completed by external reviewers.
Figure 2. Distribution of external reviewer effort. Most external reviewers complete one review.

External reviewers are only half the picture, though. The other half is the extremely dedicated programme committee, whose associate chairs write reviews and meta reviews and who made the decisions over the 7th and 8th of December. The 400 members of the PC produced 6074 reviews (or around 15 each). Just producing the reviews is a gargantuan amount of work. Unfortunately we don’t have a chart that shows the work that goes into finding external reviewers, managing those reviewers, sending reminders, facilitating discussions, attending the PC meeting, writing final post-PC comments, shepherding, checking camera-ready submissions,… it goes on and on. The General Chairs, TPCs and Papers Chairs are extremely grateful to the ACs for their efforts in putting together CHI’s biggest ever programme.

Histogram showing the number of submissions handled by ACs.
Figure 3. Distribution of AC reviewing effort. ACs generally completed 15 or 16 reviews, split between standard reviews and meta reviews.


A total of 8,319 authors were listed on submissions, give or take a few duplicate accounts. The vast majority of authors (6,425, 77%) were involved in one submission. Of these authors, 1,462 (22.8%) had their names on an accepted submission. Being on a large number of submissions is uncommon. Only 265 of those 8,319 authors (3%) made five or more submissions. Only 13 authors (0.15%) have their names attached to five or more accepted papers. We congratulate these authors on their success, but note that the vast majority of authors submit one paper and that submission is not successful. Do not be disheartened if things have not gone as well as you might have hoped. You might consider submitting a revised version of the paper to a different SIGCHI conference, many of which will be issuing calls of submissions soon.

Are people who make more submissions generally more successful with them? Only thirteen authors had three or more accepts without a reject. Two authors had four. Conversely, 45 authors made five or more submissions without any success. Figure 4 helps to make this a little bit clearer. It plots, for each author, how many accepts they had, versus the number of rejects they had. Note, authors with more than four accepts (n=13) are excluded from this plot as the published proceedings may allow for their identity to be determined. Authors with more than 7 rejects (n=25) are also excluded.

A plot showing the number of authors who had N papers accepted and rejected.
Figure 4. Individual accept/reject numbers for the CHI Papers track. Authors with more than seven rejects (n=25) or more than four accepts (n=13) are not shown. Note the density of points for each combination (e.g., 2 accepts, 1 reject) reflects its frequency across the whole set of authors. The more square the ‘patch’ appears, the more authors there are with a given combination of accepted and rejected submissions.

*ACs can require changes to papers before the camera-ready submissions are made. A small number of papers are assigned a ‘shepherd’. This is an AC who helps the authors make changes to the manuscript in a way that meets the requirements of the PC and reviewers. It often happens when a paper needs specific changes that are carefully made in order for it to be it accepted.