Saturday, December 14, 2013

Thoughts on NIPS 2013

I recently attended NIPS 2013 at South Lake Tahoe, which was an absolutely fantastic conference (the limited dining choices and general casino debauchery notwithstanding). Highlights included listening to an awesome talk by Daphne Koller on online education (and Coursera in particular), and meeting Mark Zuckerberg and learning about the new Facebook AI research lab.

I also attended the Ben Taskar Memorial Session, which, while very saddening, was also inspiring to myself and hopefully many others who attended. Ben Taskar was a truly exceptional researcher and human being, and it was very touching listening to his friends and collaborators talk about and celebrate his life and work. Ben leaves behind a wife and daughter who requires around-the-clock care. I encourage everyone who can to donate to his memorial fund.

*EDIT* -- you can read more about my thoughts on this Quora answer.

The NIPS workshops have consistently been my favorite machine learning event over the past several years. This year, I was privileged to be an invited speaker at the Discrete and Combinatorial Problems in Machine Learning workshop. The guest of honor at the workshop was Kazuo Murota, who gave a great overview on his work on discrete convexity (slides here). As was usual, I jumped around between several workshops, including Bayesian Optimization, Machine Learning for Sustainability, and Machine Learning for Clinical Data and Healthcare.

Daniel Russo presented a very interesting result at the Bayesian Optimization workshop showing a very simple way of analyzing Thompson sampling based on existing analysis for UCB-style algorithms. He mainly showed results in a Bayesian regret setting, and I wonder if one could extend his result to show account for the full spectrum between Bayesian expected regret and worst-case regret.

There were many great papers at NIPS this year (more than I could get around to reading, it seems like). Here are a few that I found particularly interesting:

Sequential Transfer in Multi-armed Bandit with Finite Set of Models, by Mohammad Azar, Alessandro Lazaric, and Emma Brunskill. This paper shows how to use knowledge learned from previous online-learning problems to learn faster for new online learning problems (i.e., transfer learning). They leverage a recent result on learning tensor decompositions in order to arrive at a provably data-efficient approach. Such problems are particularly relevant in many online systems which need to continuously personalize to new users and domains.

High-Dimensional Gaussian Process Bandits, by Josip Djolonga, Andreas Krause, and Volkan Cevher. Like the above-mentioned paper, this paper addresses how to transfer learning across different actions. Rather than focusing on sequential transfer learning like above, this paper instead treats all the actions (e.g., items) and contexts (e.g., users) jointly as points in very high dimensional space. This paper leverages a different result on low-rank matrix recovery. I've thought about this type of approach a little while back, and am really excited that the authors got this method to work. One thing that kind of bugs me is that the first stage requires a very restrictive random sampling technique, which makes the result less useful in practice.

Eluder Dimension and the Sample Complexity of Optimistic Exploration, by Daniel Russo, and Benjamin Van Roy. This paper proposes a new measure of complexity for analyzing online learning problems. This seems to be a refreshingly unique perspective on the problem, although the relationship with more understood measures such as information gain is currently unclear. But it does have the potential to help us analyze more complex prediction settings.

Multi-Task Bayesian Optimization, by Kevin Swersky, Jasper Snoek, and Ryan Adams. This paper is very cool. Hyper-parameter tuning is a big problem in machine learning. This method proposes a way to use a smaller dataset to extrapolate likely outcomes when running a learning algorithm (with a certain hyper-parameter setting) on the larger actual dataset. The experiments show that it can offer SIGNIFICANT speedups compared to standard approaches.

BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables, by Cho-Jui Hsieh, Matyas Sustik, Inderjit Dhillon, Pradeep Ravikumar, and Russell Poldrack. Estimating high-dimensional inverse covariance matrices is an increasingly more important problem in machine learning. This method absolutely blows the competition out of the water.

Learning Adaptive Value of Information for Structured Prediction, by David Weiss, and Ben Taskar. This paper is also very cool. It is essentially a meta-learning algorithm that trains an adaptive policy on which features to compute for the base classifier to use. For certain applications, computing features can be very expensive, and this paper shows how to achieve almost identical performance while using only a small fraction of the features. Furthermore, certain features can be harmful for certain classification instances, and the authors show how this method can actually predict to avoid computing such features depending on the problem instance. There's a lot about this setup that I don't quite understand from a theoretical perspective, but it seems very powerful.

Latent Structured Active Learning, by Wenjie Luo, Alex Schwing, and Raquel Urtasun. Active learning for structured prediction models is an important problem. Most active learning results address only the unstructured setting, e.g., binary labels. This issue is even more important in structured models because acquiring structured labels is typically much more expensive than binary labels. The authors propose a method for actively eliciting labels on components of the full structured prediction problem (thus leaving much of the structure latent). One thing that's worth exploring here is a mixed-initiative approach rather than a purely system-driven approach.

234 comments:

«Oldest   ‹Older   201 – 234 of 234
abc assignment help said...

I am glad after visited this site. It contains useful information for the visitors. Thank you! We are an online Python assignment help provider that assist students in writing college assignments. Students with assignment problem can visit our site and can get a complete solution.

Unknown said...

Download nighthawk app for computer is a very reliable application to control the Nighthawk router login settings from the computer and smartphone. This is a very popular application, and it makes network controlling very easy.

Kevin Wick said...

At the point when they are obscure about the ideas expected to tackle the recognized math assignment help, they start to search Math Assignment Help services.

Emma Megan said...

Payment Processing, Merchant Service Provider and

eCheck Services

rickysam said...

Use Assignment Help services to finish your work effectively. If you are facing issues in writing your academic papers, you must use an online source of expert help.

Sarah Winget said...

Have you decided to get the executives assignment help online? Obviously, you have arrived at the ideal spot to find support and ask the administration specialists themselves to do my administration assignment. ABC Assignment Experts is a main and spearheading stage, a one of its sort, where you can get the correct proficient help from the specialists to give the executives assignment composing administration. write my assignment , programming assignment help

investmango said...

Thank you for your outstanding article. Our goal is to offer top-class specialized services to you that create a fruitful, long-lasting relationship between us. Our excellent real estate portfolio management is the reason why our clients become like family. We provide the Great Deals and Discounts on Ace Divino Greater Noida West 3 BHK Flats, 4 BHK Flats, Apartments, Penthouse, Corporate Properties, Commercial Properties in Noida, Greater Noida West, Delhi NCR. Learn more about Investmango: visit our website and call now: +918076042671

jasminedisouza said...

Thank you for sharing such a good and informative article posted here. I want to share with you useful source Netflix Contact Number

cash app problems said...

On the off chance that the $ symbol isn't showing as you anticipate that it should be, at that point check your ledger that is connected to the Cash application to check whether the exchange you made is forthcoming

Alice Addy said...

I am Alice Addy, a Corporate Sales Specialist operating in the Latin American region. We are experts in Market Research (Mercado Latinoamericano de la Biomimética). We provide market reports across various industry verticals and also help with procurement and supply chain intelligence. By giving you better insights, we help you make informed business decisions.

Alice Addy said...

I am Alice Addy, a Corporate Sales Specialist operating in the Latin American region. We are experts in Market Research (Mercado Latinoamericano de Gafas). We provide market reports across various industry verticals and also help with procurement and supply chain intelligence. By giving you better insights, we help you make informed business decisions.

James Ian said...

Thank you for sharing such a clear information on topic at hand. You’ve just conducted a chemistry experiment. You’ve also recorded the results in your lab notebook. In addition, you have the supplementary data you need for your experiment. Now, it’s time to compile your chemistry lab report. Do you need someone to Write My Chemistry Lab Report
.

Holy Basil Mediclinic said...

Nice Post and Are you Looking Visit Us Best Physiotherapist in Mohali

liyasmith200 said...

Hey! I loved how smartly you have placed the words in this post. It’s definitely insightful. Some sections are ye openers. I agree that it is difficult to get away from the burden of academic assignments but there definitely is an effective way out. The Assignment Help service by the LiveWebTutors platform is well-known for offering 100% original and premium quality outcomes that can fetch the best grades for you.

Admin said...

Very nice blog.
Greetings from : Bola369

Max Willor said...

Hands down, I agree with you on that. Well done for presenting such a beautiful post. The writers and editors of the Myassignmenthelpau platform are Ph.D. and Masters qualified professionals who strive to online homework help students achieve the highest possible grades in their academic program by helping them to submit flawless assignments every time in Australia. You can get in touch with them easily by making only a few clicks here and there.

Alice Addy said...

I am Alice Addy, a Corporate Sales Specialist operating in the Latin American region. We are experts in Market Research (Mercado Latinoamericano de Caseína). We provide market reports across various industry verticals and also help with procurement and supply chain intelligence. By giving you better insights, we help you make informed business decisions.

Alice Addy said...

I am Alice Addy, a Corporate Sales Specialist operating in the Latin American region. We are experts in Market Research (Mercado Latinoamericano de Titanato de Bario). We provide market reports across various industry verticals and also help with procurement and supply chain intelligence. By giving you better insights, we help you make informed business decisions.

Entire Help said...

If you face any problem Please Visit Below Links
Aol Customer Service
Aol Mail Support
Aol Contact Number
Disney Plus Error Code 83

jobkari said...

Hi! I read your article that is great.
Our platform Jobkari.com is working very hard to provide you the latest results, information regarding the latest Sarkari jobs, exams admit card, and their syllabus. Our company is working 24x7 hours to provide you valuable content within time, and we try our best to provide you the latest notice at first on the internet. So, keep in touch with our program to get the latest results and other latest information regarding government jobs and government exams.

jobkari said...

Hi! my name is Lucy. I read your article that is great.
Our platform Jobkari.com is working very hard to provide you the latest results, information regarding the latest Sarkari jobs, exams admit card, and their syllabus. Our company is working 24x7 hours to provide you valuable content within time, and we try our best to provide you the latest notice at first on the internet. So, keep in touch with our program to get the latest results and other latest information regarding government jobs and government exams.

Cash-App-Help said...

Place a call on the toll-free number and have a talk with the expert or high professional technician for asking your queries related to Cash App Failed for My Protection? They will help you instantly in their every possible way with the 100% satisfactory answer to Cash App Payment Failed for My Protection. We are here to reach you anytime, connect with call, mail, or chat.

Cash-App-Help said...

Cash App Login is done by many users, to send and receive payment for various sectors. It is the safest and cheapest way to convey the information, feeling any kind of difficulty connecting with the experts and technicians by dialing the toll-free number and asking about Cashapp Login.

Bipard said...

Your blog is filled with unique good articles! I was impressed how well you express your thoughts.

UHBVN | UHBVN Bill Payment | UHBVN View Bill | UHBVN Bill Login | UHBVN Payment/Bill History Check | uhbvn new connection@uhbvn.org.in

Anonymous said...

Keep up the good work coming drunk hard

James Alter said...

What makes Delta Airlines so great is the fact that the airline provides great customer service over the phone and online. You will find Delta Airlines Office in Santiago. Delta has deployed a team of experts in its office that has answers to all the questions and queries of customers.

Kavya said...

Above all, let us know whether or not you have booked any hotel where you would meet passionate Delhi Airport escorts girls. It is amazing if you have an accommodation facility otherwise you can check-in at our certified hotels around the airport campus to enjoy escort service for Parties in Delhi. There are several hotels and around the assets of Aerocity where you can meet with our Strippers in Delhi. Searching the best housewife escorts in Delhi IGI airport, where you can enjoy maximum with hot chicks babes. We have almost each type of escorts you think in your dreams. Choose top quality incall escort service in Delhi with escorts who are top models in the fashion industry. Also, you can hire Air Hostess, Punjabi, College Girls, High Profile Escorts Girls. As Aerocity is nearest to the Airport, most of the girl come here who are not less than any hot actress. Even you can find college girls, young teens, very vast range of foreigner escorts as well as top class married women. Even if you are a girl and want to join escort service in delhi then you can also be a part of our elite escort agency in Delhi

Simone Watson said...

Thanks for sharing such an informative post, being a renowned top web app development company. we are willing to support local small businesses with web app development services to engage audiences.

ronanjenifer said...

Thank you for sharing such a good and informative article posted here. I want to share with you a useful source YouTube Customer Support

Unknown said...

Your post is very helpful and information is reliable. I am satisfied with your post. Thank you so much for sharing this wonderful post.
Best GBA Games
how to delete sent mail from receiver's inbox in gmail
VIPBox
Roblox Error Code 277

Great Jobs said...

After graduation, applying for any Nigeria Immigration Service Recruitment will be the first option you have before committing to your plans, and here we will teach you how to apply for any Nigerian Prisons Service Recruitment of your choosing, so here we will teach you everything you need to know.

https://sabkisarkarinaukri.com/ said...

Thanks for nice blog.

Persian said...

Really Nice Post. Thank You Visit Us Persian Cats For Sale Online in Chandigarh Bangalore Mumbai Delhi Pune Kolkata

Emberly Joe said...

Want to take Victoria University Assignment Help .? Check it now from here.

«Oldest ‹Older   201 – 234 of 234   Newer› Newest»