Saturday, August 21, 2010

Reviewer Anonymity Risk

I'm on the program committee for the upcoming ACM Conference on Web Search and Data Mining (WSDM). The reviewing process is done completely via the EasyChair system. One seemingly nice feature is that in any descriptions of submitted papers, clicking on the author's names will take you to their personal web pages (if the authors submitted such information). For example, here is the info page for a submission that I'm a co-author on:
Note the links to the authors' home pages at the bottom of the page. (FYI the primary reason the submission information is blurred out is because I have not yet obtained permission from my co-authors allowing me to broadcast this information.)

While this seems like a pretty convenient feature, using it can actually carry a non-trivial degree of risk, depending on your institution. The reason for this is because many (most?) researchers track visits to their web pages. For example, when I click on the my name in the page shown above, my webpage stat tracker logs the following event:
Location:Ithaca, New York, United States
IP Address:Cornell University (128.84.154.13)
Page Loaded:www.yisongyue.com/
Referrer:www.easychair.org/conferences/submission.cgi?submission=468064;track=24725;a=a09cfab06091
Time:21st August 2010 17:23:23
For a variety of reasons motivated by preserving fairness, the reviewing process is supposed to be blind, i.e. the authors shouldn't know who their assigned reviewers are. But, basically, what the situation described above means is that unsuspecting program committee members and paper reviewers can reveal significant information to the authors regarding the identity of the reviewers. For example, if I see an event in my logs like the one above from a relatively small school with perhaps one or two people in my field of research, then I can actually make a very strong inference about who one of my reviewers might be.

Now, this issue can be largely avoided if one uses a less conspicuous network than that of a university or a tech company (e.g., Starbucks), but it still poses a risk nonetheless. I certainly won't be clicking on these links anytime soon.

This also speaks to the larger issue of how to make queries or collect information anonymously, and what the social implications are for breaking anonymity. Privacy is a huge topic that I cannot hope to do full justice on. But, as is my wont, I'd like to touch on another completely different setting.

Most online social networking sites do not disclose pageview information, so you can't see who has been visiting your profile. What would happen if such information becomes available?

A few sites, such as OkCupid, allow you to opt in to remove this layer of anonymity. Any users who opt in can then see which other users (that have also opted in) have visited their profile, but users who have not opted in cannot see this information. This leads to the expected behavior of people creating two profiles: one official and opted in, and the other for stalking.

But we needn't even need to use examples that involve technology (although they are more fun). Suppose you could goto parties and freely check out attractive potential dating partners without anyone noticing. How would that change your behavior? (As a side note, I was once asked by someone to lend him my sunglasses at a social gathering, and I'm almost certain it was for this exact reason.)

3 comments:

Parisa said...

I think your point about www.easychair.org is pretty valid. You should send them an email about it and recommend they implement a simple redirector service that acts as a proxy to strip any Referer information that is going to be logged and analyzed by the destination/author's site.

Parisa said...

And to your last point, everyone is doing this... it's just a matter of how skilled they are at not getting caught :)

Yisong Yue said...

Yeah, that would help a lot. Since people submit their own home page information, they could still potentially create a special URL for that specific paper submission (e.g., www.yisongyue.com/?referer=wsdm2011).