Friday, December 19, 2014

Thoughts on NIPS 2014

NIPS 2014 happened last week, and what a great conference it was. Lots of great papers, workshops, and invited talks.
The NIPS Experiment
In contrast to previous years, the most talked about thing from NIPS this year was not any new machine learning approach, but rather a reviewing experiment called the NIPS Experiment.

In a nutshell, about 10% of submissions were reviewed independently by two sets of reviewers (including two different Area Chairs). The goal of the NIPS Experiment was to assess to what extent reviewers agreed on accept/reject decisions. The outcome of the experiment has been a challenge to interpret properly.

The provocative and thought provoking blog post by Eric Price has garnered the most attention from the broader scientific community. Basically, one reasonable way of interpreting the NIPS Experiment results is that of the papers accepted for publication at NIPS 2014, roughly half of them would be rejected if they were reviewed again by a different set of reviewers. This, of course, highlights the degree of subjectivity and randomness (likely exacerbated by sub-optimal reviewing) inherent in reviewing for a such a broad field as machine learning.

The most common way to analyze this is from a certain viewpoint about fairness. I.e., if we had a budget for K papers, did the top K submissions get published? From that standpoint, the answer seems to be a resounding no, no matter how you slice it. One can argue about the degree of unfairness, which is a much murkier subject.
Alternative Viewpoint via Regret Minimization
However, as echoed in a blog post by Bert Huang, NIPS was AWESOME this year. The poster sessions had lots of great papers, and the oral presentations were good.

So I'd like to offer a different viewpoint about NIPS, one based on regret minimization. Let's assume that the accepted papers that were more likely to be rejected in a second review are "borderline" papers (seems like a reasonable assumption, but perhaps there are arguments against it). Then, had we swapped out a bunch of borderline papers with other borderline papers that got rejected, would the quality of the conference have been that much better?

In other words, given a budget of K papers to accept, what is the collective quality of K papers actually accepted versus the quality of the "optimal" set of K papers we should've accepted? It's conceivable that the regret on quality difference could be quite low despite the paper overlap being substantially different.

One might even argue, as alluded to here, that long-term regret minimization (i.e., reviewing for NIPS over many years) requires some amount of randomness and/or disagreement between reviewers. Otherwise, there could be a more serious risk of group-think or intellectual inbreeding that can cause the field to stagnate.

Not sure to what extent this viewpoint is appropriate. For instance, NIPS is also a venue by which junior researchers become established in the field. Having a significant amount of randomness in the reviewing process can definitely be detrimental to the morale and career prospects of junior researchers.
On to the Actual Papers
There were many great papers at NIPS this year. Here are a few that caught my eye:

Sequence to Sequence Learning with Neural Networks
by Ilya Sutskever, Oriol Vinyals & Quoc Le.
Ilya gave, hands down, the best talk at NIPS this year. Ever since it started becoming popular, Deep Learning has carried with it the idea that only Geoff Hinton & company could make them work well. Ilya spent most of his talk describing how this is not the case anymore. He also showed how to incorporate a type of gradient momentum called Long Short-Term Memory in order to do sequence-to-sequence prediction with deep neural networks.

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics
by Sergey Levine & Pieter Abbeel.
This paper combined reinforcement learning and neural networks in order to do policy search. What's shocking about this approach is how few training examples they needed to train a neural network. Granted, the neural network wasn't very deep, but still, the low amount of training data is quite surprising.

Learning to Optimize via Information-Directed Sampling
by Dan Russo & Benjamin Van Roy.
Dan Russo has been doing some great work recently on analyzing bandit/MDP algorithms and proposing new algorithms. This paper proposes the first (mostly) fundamentally new bandit algorithm design philosophy that I've seen in a while. It's not clear yet how to make this algorithm practical in a wide range of complex domains, but it's definitely exciting to think about.

Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
by Adarsh Prasad, Stefanie Jegelka & Dhruv Batra.
This paper deals with how to do submodular maximization when the ground set is exponentially large. This paper exploits specific structure in the ground set, e.g., it can be solved via cooperative cuts, in order to arrive an efficient solution. It would be interesting to try to learn the diversity/submodular objective function rather than hand-craft a relatively simple one (from a modeling perspective).

From MAP to Marginals: Variational Inference in Bayesian Submodular Models
by Josip Djolonga & Andreas Krause.
Log submodular models are a new family of probabilistic models that generalizes things like associative Markov random fields. This paper shows how to perform variational marginal inference on log submodular functions, which might be wildly intractable when viewed through the lens of conventional graphical models (e.g., very large factors that obey a submodular structure). Very cool stuff.

Non-convex Robust PCA
by Praneeth Netrapalli, Niranjan U N, Sujay Sanghavi, Animashree Anandkumar & Prateek Jain.
This paper gives a very efficient and provably optimal approach for robust PCA, where a matrix is assumed to be low-rank but except for a few sparse components. This optimization problem is non-convex, and convex relaxations can often give sub-optimal results. They also have a cool demo.

How transferable are features in deep neural networks?
by Jason Yosinski, Jeff Clune, Yoshua Bengio & Hod Lipson.
Along with a scientific study on the transferability of neural network features, Jason Yosinski also developed a cool demo that can visualize the various hidden layers of a deep neural network.

Conditional Random Field Autoencoders for Unsupervised Structured Prediction
by Waleed Ammar, Chris Dyer & Noah A. Smith.
This paper gives a surprisingly efficient approach for learning unsupervised auto-encoders that avoids making overly restrictive independence assumptions. The approach is based off CRFs. I wonder if one can do this with a more expressive model class such as structured decision trees.

A* Sampling
by Chris J. Maddison, Daniel Tarlow & Tom Minka.
I admit that I don't really understand what's going on in this paper. But it seems like it's doing something quite new so there are perhaps many interesting connections to be made here. This paper also won one of the Outstanding Paper Awards at NIPS this year.


Chong said...

hi yisong,

thanks for the great summary.

mamam said...

Thanks for sharing.I found a lot of interesting information here. A really good post, very thankful and hopeful that you will write many more posts like this one.

Unknown said...

Thanks for sharing this post, it was great reading this article! would like to know more! keep in touch and stay connected! Also Check here

ShowBox Apk

gbwhatsapp 6.55 apk download


Mini Militia Hack

Best Apk Mods

Cheap Smm Panel

Just Smm Panel

Unknown said...

Cartoon HD
Lucky Patcher

Unknown said...

Sample Data said...

music paradise pro apk

codeinverse said...
funny instagram captions
instagram captions for friends
birthday wishes
instagram captions for selfies

Sample Data said...

tubemate for pc

Sample Data said...

Unknown said...

9Apps APK

Raj said...

Game killer Apk Download may be a powerful nonetheless straightforward to use, supports most variety of games and is compatible on latest humanoid versions Lollipop and candy.

Shamima said...

Ankita Singh said...

cyberflix tv
Hotstar Premium Mod Apk TR Vibes
Taraftar TV


Ankita Singh said...

major league baseball

Anonymous said...

Tutuapp Free Download Latest Version

netflix apk said...

Thank you for this great article please check my article too
click here
i really appritiate your afforts Framaroot for Android one again thankyou so much
Framaroot Descargar please visit my site too.

latest bollywood songs 2019 said...

Looking for the best site for Vidmate Download ? Vidmate Official site is the right place for you! Vidmate is the best app that lets you download videos and songs from YouTube, Facebook, Twitter, etc and various other sites.

Ahmad Suhaib said...
This comment has been removed by the author.
Ahmad Suhaib said...

APKdodo snaptube Mobdro

Rocky said...

This is a blog you can get useful information and better stuff right here cara mengecilkan lengan

Anonymous said...

Thanks for the information about this Really nice know more good and intresting topics. dental implant cost

Pankaj Singh said...

Nice blog, thank you so much for sharing such an amazing blog with us. Visit for the best Website Designing and SEO Services in Delhi, India.
SEO Service in Delhi

Nucle said...

There is no doubt in the app you can easily download the app the most exciting features are waiting for you can easily go with, so Download TutuApp APK Official App For Android, iOS devices, iPad, iPhone TutuApp. There are nothing extra features required in this app because the most important items are already available in his update versions.

The India said...

Content Writing Company in Delhi
Content Writing Services in Delhi
Mobile App Development Company Delhi
PPC Company in Delhi
PPC Company in India

HowTo said...

Thanks for this post and To Learn Cooking By Sitting At Home

Thanks for this post and How To Make A Tea

Thanks for this post and How To Make Chicken Role

Thanks for this post and How To Make Chicken Vegetable Sandwich

HowTo said...

Thanks for this post and To Learn Cooking By Sitting At Home

Thanks for this post and How To Make A Tea

Thanks for this post and How To Make Chicken Role

Thanks for this post and How To Make Chicken Vegetable Sandwich

lajwantidevi said...

This is a nice Site to watch out for and we provided information on
vidmate make sure you can check it out and keep on visiting our Site.

shilpi said...

Vidmate is one of the best known applications currently available for downloading videos and songs from online services like Vimeo,

Dailymotion, YouTube, Instagram, FunnyorDie, Tumblr, Soundcloud, Metacafe, and tons of other multimedia portals. With this highly

recommended app, you’ll get to download from practically any video site.A free application for Windows users that allows you to download online

An entertaining application
Vidmate latest update in pie
Amazing features of Vidmate

lajwantidevi said...

This is a nice Site to watch out for and we provided information on
vidmate make sure you can check it out and keep on visiting our Site.

kajal singh rajput said...

Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows.

pat said...

vidmate app
9 apps

Periocenter said...

Thanks for the sharing this. pls check this also.
Periodontist Houston TX

pat said...

vidmate app

Mutual Fundwala said...

Thanks for this amazing blog, visit Mutualfundwala for the best Mutual Fund Agent and Mutual Fund Distributor.
Mutual Fund Distributor

pat said...


pat said...


pat said...


pat said...


pat said...


pat said...


pat said...


pat said...


pat said...


Alice said...

Impressive article. For short video making and sharing, get Likee app. Share your wonderful moments with others and become an influencer.

Audrey Signs said...

Listen to the Old Audio Bible Stories here.
Audio Bible Stories

jasmin said...

call adultxxx
call girl

Assignment Help said...

There are many online tools out there to provide assignment help in just about any field of study.
programming assignment help

Anonymous said...

I found this website eventually. Really good inoperative, Thanks for the post and effort! Please keep sharing more such blogs. I would like to share about CMET Jobs.

TechnoBridage InfoTech said...

software development company in surat

New Year said...

New Year 2020 Quotes for friends

Unknown said...

digital marketing consultancy surat

priya sharma said...

Such A nice post... thanks For Sharing !!Great information for new guy like Happy New year 2020

vidmate apk said...

Really i found this article more informative, thanks for sharing this article! Also Check here

Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows

Vidmate App Download

Vidmate apk for Android devices

Vidmate App

download Vidmate for Windows PC

download Vidmate for Windows PC Free

Vidmate Download for Windows 10

Download Vidmate for iOS

Download Vidmate for Blackberry

Vidmate For IOS and Blackberry OS

Pradeep Appslure said...

Thank you so much for this amazing information sharing with us. Visit Appslure WebSolution for the best mobile app development company in Gurgaon
Mobile app development company in gurgaon

Shweta gaur makeup artist said...

Shweta gaur is one of the famous makeup artist in all over India. We are providing the best makeup artist courses and more other courses in over branches in Delhi.

Makeup Artist in Delhi
Makeup Artist
Best Makeup Artist in Delhi
Best Makeup Artist in East Delhi
Top Makeup Artist in Delhi
Top Makeup Artist in India
Bridal Makeup
Best Bridal Makeup Artist in Delhi
Bridal Makeup Artist in Delhi with Price
Bridal Makeup Artist in Delhi NCR
Top Bridal Makeup Artist in Delhi
Bridal Makeup in Delhi
Bridal Makeup Charges in Delhi
Bridal Makeup Packages Prices
Freelance Makeup Artist
Bridal Makeup Charges in Delhi
Freelance Bridal Makeup Artist in Delhi
Reception Makeup
Party makeup
Wedding makeup artist
Makeup Artist in Noida

lajwantidevi said...


Nisha Sharma said...

Mobile app development company in mumbai

customized gifts for him said...

My online shop provides Pretty and customized pillow ! include accent pillow case ,burlap pillow cover diy,baby burlap,burlap throw pillow covers and other pillow!

accent pillow case baby burlap
accent pillow case home decor
housewarming gift

Ethan said...

Amazing blog post i really like it thanks for sharing OGWhatsApp Apk 2020

modded app said...

picsay pro mod apk
turbo vpn mod apk
nordvpn mod apk
opera mini mod apk

modded app said...

filmorago pro mod apk
beauty plus mod apk
hulu mod apk
hola vpn mod apk

Jenifer said...

Best Biology Assignment Help is provided to students by professionals of Abc Assignment Help. We have best Biology Assignment Helper to avail quality writing service in minimum time. Our online service is available 24/7 to help students with their doubts on any topic of Biology. For complete details, get in touch with our online members.

Law Assignment said..., you can get all the requisite resources that you may need to complete a Corporate governance essayor assignment. It is seen that as one of the most reliable sources of academic assistance on the internet. You can see for yourself.
Corporate Law essay assignment
Corporate law assignment help
Corporate governance expert
Corporate governance essay
Corporate law thesis
Corporate governance dissertation

Assignment help said...

Myassignmenthelp online is a premium academic writing service offering professional assistance in every course and subject of your choice. Contact us and get help in any topic and our tutors will guide you through identifying expert answers with proper arguments and references. We offer expert writing service for assignments, essays, coursework, report writing, case study, research paper and thesis within your budget.

Unknown said... subscription cost to how to install and uninstall , here is the complete guide that takes you through the entire process. +1855 905 0307 Toll-Free
To know more click:
office com setup
office setup
office product key
How to Install Microsoft Office

Nouman Shakeel said...

how to make money online
how to win friends and influence people

Law Assignment said...

Call our Business Law Homeworkwriter on searching business law online UK,Australia and Other Countries for improve your marks

Business Law Assignment Help
Business Law assignment
Business Law help
Business Law Homework
Business Law term paper
Business Law thesis
Business Law coursework

Nick Hebrew said...

Such a wonderful information blog post on this topic All assignment services provides assignment service at affordable cost in a wide range of subject areas for all grade levels, we are already trusted by thousands of students who struggle to write their academic papers and also by those students who simply want MATLAB Programming Help to save their time and make life easy.

Unknown said...

Hey Check out my new Blog Post You all will love this smartnews for windows 10
& Cotomovies For Pc
& Free Fire Diamond Generator
& Facetime for pc
& Ola TV Apk

Unknown said...

If you still find issues in the update process contact the expert’s team to get help for Garmin Nuvi update USA via Garmin Nuvi update customer support number 18559050307 toll-free. To know more visit:

Garmin Nuvi update
Garmin Nuvi Update Customer Support Number
Garmin Nuvi Map update
Garmin Nuvi update Support Number
Garmin map update

If you still are not able to set up the email in your iPhone even after following the instructions then you should get in touch with the experts through verizon email support phone number +1855 905 0307 toll-free to get the best assistance from our team instantly as they are available 24X7 for the service of the users. To know more visit:

Verizon Email Help Phone Number
Verizon Email Support Phone Number
Verizon Email Support
Verizon Yahoo Email Customer Service Number

Bitdefender tech support phone number to fix Bitdefender error code 1020 at our +1855 905 0307 toll-free number which is available 24/7. To know more visit:

Bitdefender technical support number
Bitdefender Tech Support
Bitdefender Antivirus support
Bitdefender error code 1020

If you are still not able to connect to the Wi-Fi and still facing this issue even after following the steps then call us at HP customer support number tollfree. To know more visit:
HP support number
HP customer support number tollfree
HP help customer support number
HP customer support
HP technical support

Open bin files said...

Open bin files

NFL Fan said...

Chiefs Game Live Stream
Bears Game Live Stream
Raiders Game Live Stream
Steelers Game Live Stream
Rams Game Live Stream

gbappsdownload said...

backyard revolution solar review
15 minute manifestation

John Cooper said...

Business law expects help with law assignments to students who learn various resolutions and enactments for various business ventures. This requires a lot of exertion and ability, which most students need, and in this way, they require law assignment help, to finish their scholastic law assignments and get the best grades.

Gift Cards 2021 said...

the thought behing this is so amazing Gift Cards 2021

R ADK said...

Software IT Coaching Center in Chennai | Drilling consultants

Businessassignmenthelp said...

business assignment help

statanalytica said...

data research assignment Help

codeavail said...

computer science assignment help

Java Assignment Help said...

Java Assignment Help

buycollegeessay said...

Essay Writing Help

Assignment Help said...

I loved this assignment article and kept updating interesting articles. I will be a regular reader. I am offering help to students over the globe at a low price.
Nursing Dissertation

Unknown said...

Great Job here on your blog. I read a lot of blog posts and i never heard of a topic like this. I love this topic you made. you can also enjoy my blog shayari guide - check links
Hindi Shayari

UniversityResult said...

Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man, Keep it up. check my blogs
Uniraj Result

Calltutors Australia said...

I have shared this link with others to keep posting such information to provide the best in class Australian homework help online at very affordable prices.
Assignment Help Adelaide
Assignment Help Canberra
Assignment Help Melbourne
Assignment Help Perth
Assignment Help Sydney
Assignment Help Darwin
Assignment Help Gold Coast

Nouman Shakeel said...

I am happy to find your distinguished way of writing the post. Now you make it easy for me to understand and implement the concept. Thank you for the post.
zoya name meaning in urdu
eshal name meaning in urdu
maria name meaning in urdu
nimra name meaning in urdu

AtoZAssignments said...

Wow. I am giving 5 out 5 stars to this information. Thanks a ton from my side

My Assignment Help said...

Writing effective essays has become critical to academic success. It is a part of standardized test, educational assignments or a college application. Perhaps, as essay writing is given importance in curricula, the contemporary students are bombarded with such assignments. The burden of essay writing prevails from primary to postgraduate level.These reasons get the students under tremendous pressure to secure A-grade paper in each semester so students require essay writer australia Online.

Admin said...

Thanks for the interesting content. I like your post and your blog is amazing.
If you are interested in Video Downloader apps you can check my blog site. It is new and really informative.

VidMate 2016 download