Random Ponderings

In Beijing

2011-08-03T05:31:00.000-04:00

I'm currently staying with my grandparents in Beijing after spending a week at SIGIR 2011. Thoughts on the conference will be forthcoming, but for now, thoughts on China.

Every year, the prices increase another notch, especially in the more touristy areas. Many experts think that China's will surpass the United States in terms of GDP within the next 5 to 10 years. Such an estimate does not seem unreasonable given China's (over-)population. After all, 1.4 billion is a pretty big number.

It certainly seems that the Chinese have become well calibrated to the prices that Western tourists are happy to pay. For example, Peking Duck at the historic Quanjude runs about $45 USD per person, which is about the upper bound on what most people (including myself) would pay at a high class restaurant in a "developing" country.

Although I've been to Beijing many times before, I spend most of my time with my grandparents, who are more the sedentary type. As a result, I always come across something new to check out every time I come back. Last week, I visited an ancient mosque with Khalid, and actually got to observe a Friday service. It was bit of an out-of-world experience to listen to a Chinese preacher espouse the virtues of and describe the preparation of Ramadan with a heavy Beijing accent.

This visit has also been a nice opportunity for me to try out my new camera:

Random Amusement of the Day

2011-05-23T00:33:00.002-04:00

What do you get when you cross a Bayesian with an Asian?

Vari-Asianal Inference!!

SIGIR 2011 Tutorial: Practical Online Retrieval Evaluation

2011-05-03T00:19:00.006-04:00

Filip Radlinski and I will be teaching a tutorial titled "Practical Online Retrieval Evaluation" later this year at SIGIR 2011. The tutorial is currently scheduled for the afternoon session on July 24th, 2011. A list of all SIGIR tutorials can be found here.

Tutorial Description:

Online evaluation is amongst the few evaluation techniques available to the information retrieval community that is guaranteed to reflect how users actually respond to improvements developed by the community. Broadly speaking, online evaluation refers to any evaluation of retrieval quality conducted while observing user behavior in a natural context. However, it is rarely employed outside of large commercial search engines due primarily to a perception that it is impractical at small scales. The goal of this tutorial is to familiarize information retrieval researchers with state-of-the-art techniques in evaluating information retrieval systems based on natural user clicking behavior, as well as to show how such methods can be practically deployed. In particular, our focus will be on demonstrating how the Interleaving approach and other click based techniques contrast with traditional offline evaluation, and how these online methods can be effectively used in academic-scale research. In addition to lecture notes, we will also provide sample software and code walk-throughs to showcase the ease with which Interleaving and other click-based methods can be employed by students, academics and other researchers.

Topics include:

Overview of online evaluation

Collecting usage data: How to be their search engine (with code walk-through)

The Interleaving approach for click-based evaluation

Practical issues in deploying Interleaving experiments (with code walk-through)

Analyzing and interpreting Interleaving results

Quantitative comparison of Interleaving with other evaluation methods (both online and offline)

Tricky issues, extensions, and limitations

From evaluation to optimization: Deriving reliable training data from user feedback

Thoughts on Pittsburgh, CMU

2011-04-16T15:10:00.002-04:00

Having been at CMU for several months now, I've formed a few opinions regarding the school, especially in how it contrasts to Cornell, where I spent the previous five years.

First of all, I must say that it has been a pleasure working and interacting with Carlos and his group. The group's research profile covers an impressive breadth of interesting research topics, ranging from large-scale machine learning, efficient non-parametric inference methods, probabilistic permutation models, and novel ways of managing our growing information overload.

From a grad student's perspective, the thing that stands out most is the fact that students must choose advisors within the first month of starting. In contrast, at Cornell, students typically settle on advisors sometime during their second year, and are encouraged to spend their first year taking project courses and "shopping around". Both styles have their benefits and drawbacks. Students tend to start working on serious research projects much sooner at CMU. However, those who wish to spend some time exploring different research areas will likely feel much more comfortable at Cornell.

As a school, the research culture at CMU is significantly more focused on computer science than Cornell is. Oftentimes, one gets the impression that the entire school revolves around computer science and related fields. In contrast, research at Cornell is much more diverse, and this is reflected in the graduate student culture. On the other hand, the relative proximity of different research areas at CMU seems to foster more collaboration between research groups.

As a city, Pittsburgh has been more agreeable to me than I was expecting. Located on rolling hills and at the intersection of three rivers, the surrounding landscape can be quite beautiful. I recently learned that Pittsburgh has 446 bridges! (Although it's unclear how bridges are defined in that statistic.) The city is also entrenched in culture from its steel boom days, although I haven't had too much time to explore that aspect thus far. I particularly liked the Google Pittsburgh office, which had the feel of a converted steel mill.

And the CMU campus isn't half-bad either:

All in all, it's been an enjoyable experience thus far.

My Take on "Asian" Parenting

2011-03-18T23:10:00.002-04:00

A recent book by Yale law professor Amy Chua, titled Battle Hymn of the Tiger Mother, has been stirring up quite a bit of controversy. One can find an introduction of the book in the WSJ article Why Chinese Mothers Are Superior.

I have not read the book (nor do I plan to do so if that requires me to purchase it), but from reading the WSJ article, I have to believe that Mrs Chua is overstating and/or sensationalizing her point for the sake of making the point (and thus boosting book sales).

In essence, Mrs Chua characterizes herself as a prototypical "Chinese" mother. In contrast to prototypical "Western" mothers who often employ permissive parenting, Mrs Chua stresses that (quoted from the WSJ article)

What Chinese parents understand is that nothing is fun until you're good at it. To get good at anything you have to work, and children on their own never want to work, which is why it is crucial to override their preferences.

To put it simply, this is ridiculous. In fact, her elder daughter's response depicts Mrs Chua as a caring mother who, while strict, also encourages her daughters to expand their horizons. This clearly contradicts the WSJ article, which depicts Mrs Chua as totally controlling from both time-management and thought-management standpoints.

I furthermore find her use of the term "Chinese" distasteful. After all, she construes Chinese so broadly that it loses nearly all meaning (not to mention the fact that she barely qualifies as Chinese in the real sense of the word). All this suggests a blatant emotional ploy to conflate her parenting philosophy with the pervasive American fear of losing economic competitiveness to China (and thus boosting book sales).

Of course, stereotypical Asian parents are more strict than stereotypical Western parents. No one denies this. But Mrs Chua's sensationalist characterization is so extreme it borders on the absurd. To pigeonhole all model Asian parents into Mrs Chua's caricature is very misleading. And even should some Asian parents mirror Mrs Chua's description (and these people do exist), it is folly to think that the children are always better off having received such an upbringing.

Take my upbringing, for instance. It is true that, as a child, my parents often criticized my work and were generally dismissive of my accomplishments. If I succeeded, it was to be expected. If I failed, it was a disappointment. Sure, they had high expectations. But I still participated in my share of school plays. My mom still picked me up from after-school rehearsals. I still sang in a choir for thirteen years. And even though my parents thought such activities were mostly a waste of time, they also recognized that childhoods are fleeting. In fact, over the years, their opinions have slowly changed to the point where they now value the fact that I participated in so many extra-curricular activities. Some of my proudest moments include performing in front of my family.

I most certainly agree that strong parenting should include an element of "tough love" and setting high expectations for success. That is basically the philosophy that Mrs Chua is advocating if one overlooks the overly controlling manner in which she executes this philosophy (at least by her description).

But I also think that true success must come from within. People of all ages find the greatest satisfaction when they are inspired to work hard and succeed. The real reason I was motivated to work hard was not because my parents shamed me by telling me that I was garbage whenever I did not wholly succeed. There are two real reasons. First, I was surrounded by friends and teachers who encouraged me to tap into my imagination. Second, my parents are inspiring people in their own right.

Despite all the childhood angst I endured, I love and look up to my parents. As first generation immigrants, they found the strength to thrive and provide for their family, when many with lesser willpower would not have succeeded in their situations. That is true inspiration.

And of course, when the worst side of Asian parenting becomes too oppressive, then one runs the risk of sad endings.

In Remembrance.

Les Valiant wins ACM Turing Award

2011-03-11T10:22:00.001-05:00

The ACM recently announced that Les Valiant is the newest recipient of the Turing Award, which is the highest award in the field of computer science. Les Valiant is widely considered to be one of the main founders of modern machine learning. His concept of PAC learning revolutionized the way we reason about machine learning algorithms. The PAC framework provides a way to formally characterize hypothesis testing, and thus also gave justification to concepts like Occam's razor.

The typical way to design programs that you can "trust" to do the right thing is to both (A) empirically verify and (B) formally prove its correctness. For both (A) and (B), one must have a common language or set of benchmarks with which to compare different algorithms. For machine learning programs, ways to reason about (B) were few and far in between before the PAC learning framework. You might say that machine learning before PAC learning was like computing before Turing machines.

In honor of this occasion, I decided to finally read Valiant's seminal paper, "A Theory of the Learnable". It's definitely a good read, and understandable for anyone with working knowledge of computer science. In fact, I found it striking that the paper still felt quite "modern" even after all these years and subsequent advances.

My Favorite Procrastination Activities

2011-03-05T22:58:00.000-05:00

Like basically everyone, I am a habitual procrastinator. Ever since reading Good and Bad Procrastination by Paul Graham, I've continuously tried -- with limited success -- to manipulate or trick my procrastinating tendencies to maximize my productivity. Barring working on the most important thing possible, I hope to at least do something useful.

In cases when the urge to procrastinate from doing research becomes too strong to ignore, my list of ideal secondary activities includes:

reading papers

exercising

blogging

updating my website

practicing photography

cleaning my apartment

Of course, these secondary activities sometimes rise in prominence. For instance, if I haven't cleaned my apartment or exercised in a long time, then I feel compelled to prioritize that higher. Unfortunately, that is exactly when I also feel compelled to procrastinate from doing those activities. One would hope that my alternate activity of choice then would be doing research, but by that point I've usually completely succumbed to doing something mindless like YouTubing.

Now some of you might suspect that I'm blogging right now because I should be doing more important work. Some of you might be right. So I guess it's back to work! There are theorems to be proven!

Dynamic Ranked Retrieval

2011-02-20T10:38:00.001-05:00

One of the biggest trends in web page design is the use of rich or dynamic content. In this world of Flash, AJAX, jQuery and Firefox plugins, it is quite easy to create rich interactive layouts that can efficiently and dynamically display relevant content in response to user input.

Information retrieval services have been taking advantage these technologies by dynamically changing the search results retrieved in response to the user modifying the query or clicking on a search result. For instance, Surf Canyon is a start-up search company that allows users to expand or refine search results. Below is an example interface inspired by their design.

In the first screen capture, the interface initially displays a normal ranking of results in response to the query "SVM".

In the second screen capture, the interface displays an indented ranking in response to the user clicking on the expand button.

These indented rankings can potentially go several levels deep.

One can also imagine an alternative layout where the results are displayed in a multi-column layout, similar to the Mac Finder. The key point here is that dynamic rankings allow the user to efficiently retrieve the most relevant information (i.e. dynamic post-query personalization).

In a recent paper presented at WSDM 2011, my collaborators and I have developed the first formal model (to my knowledge) for characterizing such rankings. It is a bit crude in some respects, but is still an exciting first step. The key intuition is to treat the user experience as traversing a ranking tree.

When examining each result, the user decides whether to expand or skip that result. An expand corresponds to taking a right branch of the corresponding node in the ranking tree, and a skip corresponds to taking the left branch. In essence, the ranking tree captures all possible sequences of clicks and skips the user can make when interacting with the dynamic ranking. It is, of course, intractable to explicitly enumerate the entire tree, but we can still prove things about it. A conventional static ranking corresponds to the special case where the user always takes the left (skip) branch. The utility a user derives from a ranking tree can then be defined over the path (or distribution of paths) the user traverses in the ranking tree.

Given a modularity assumption on utility (i.e. that utility can be decomposed additively over the results), we proved that a simple greedy algorithm always performs at least as well as the optimal static ranking (i.e. dynamic personalization never hurts). Our experiments furthermore demonstrate that dynamic rankings can offer substantial gains in performance in practice. We have also made our experimental setup available for others to test with. Many thanks go to Christie Brandt, who did the lion's share of the work for this project.

There are still many interesting directions to explore. For instance, this model only deals with extrinsic ambiguity (different users have different but narrow intents when issuing the same query), but not intrinsic ambiguity (users are simultaneously interested in many different aspects of the query). The model also doesn't capture cases where the user backtracks or scans multiple results before expanding one of them. One direction that I'm particularly interested in is modeling even richer interfaces, such as those that allow users to provide redundancy feedback. Needless to say, this is an exciting research area that I expect will continue to grow in the foreseeable future.

My Trip to India: The Backlash

2011-02-18T00:43:00.002-05:00

A couple months ago, I visited Hyderabad to explore some research collaboration possibilities. Little did I know that the trip would ignite one of the more stressful periods of my life. To sum it up, I'm now finally infection-free after four weeks pain, medication, and a trip to the ER.

The origin of my malaise is unclear, since bacterial infections can happen anywhere. But at the very least, my susceptibility to contracting an infection was dramatically increased due to my trip to India. To start with, it never really gets cold in Hyderabad, so there are insects and arachnids around all year long. I never found out if the bites on my leg were caused by spiders or mosquitoes or something else entirely, but at the time I didn't really pay it much heed. After all, I'd dealt with my fair share of bug bites before, and none of them had ever caused me much trouble (queue ominous background music).

But for whatever reason, the bites didn't heal properly. I suspect that my immune system was especially vulnerable at that time due to the exhausting 30 hour trip from Hyderabad to Phoenix. And since holidays with the parents typically involve large amounts of socializing interleaved with frantic efforts to get work done during the down hours, the status of my bite wounds was never at the forefront of my mind. In fact, I engaged in multiple activities that most likely exacerbated the condition (basically anything that involved moving around).

By the time my vacation was drawing to a close, it was clear that I had an infection at the site of the bite wound. Actually, to anyone who wasn't me, it would've been clear I had an infection several days earlier -- not that anyone else knew about it (way to be a macho idiot, Yisong).

The physician I saw in Phoenix suspected a staph infection, so he prescribed ten days worth of Bactrim and Keflex. The next day, I boarded a plane back to Pittsburgh, expecting things to clear up in the next few days. (Note to everyone: DO NOT search for images of staph infections.)

Things took a turn for the worse almost immediately. I'll spare the gruesome details, but my infection starting oozing "substances". To make matters even worse, the infection had swelled to the point where even just walking hurts.

Luckily, the city of Pittsburgh is dominated by the University of Pittsburgh Medical Center, which I must say offers some of the best medical service possible. Within thirty minutes of entering the urgent care facility, a physician had inspected and operated on my infection. The procedure was fairly straightforward: after applying anesthetics, the doctor made an incision and manually drained the infection by squeezing it out. After some standard wound packing, I was discharged only an hour after I was admitted.

Over the next several days, my wound slowly healed and I finished my prescription. Thinking the issue resolved, I treated myself to a fine glass of Malbec.

A couple days after my prescription ran out, I noticed a small bump close to the site of the original infection. Using all the common sense that I could possibly muster, I decided to hold off seeing a doctor for a day or two just in case it was just a fluke bump on my skin (yes, that did make sense at the time).

Lo and behold, a second infection (or maybe it was a remnant of the first) took hold. A trip to the doctor procured me another ten days of Bactrim and Keflex. The doctor hoped that no draining would be necessary this time, due to taking the antibiotics in the early stages of the infection.

As it turns out, I'm allergic to Bactrim (or I'd developed an allergy to it). The next evening, I noticed a rash on both my upper thighs. Thinking the situation not too serious, I decided to try to sleep it off and see what the doctor had to say the next morning.

Now, one of the reasons I am able to see a doctor so promptly and without a prior appointment is because I am using the urgent care service. Unfortunately, this leads to significant variability in which doctor I get to see. In retrospect, the professional incompetence of this next doctor was pretty off-the-charts.

The doctor (a) did not prescribe new antibiotics or attempt to treat the infection in any way, instead telling me to let it heal on its own, and (b) did not prescribe any allergy medication nor provide any advice on what I should do regarding my rash. At that time, the rash on my thighs had faded, but I was developing new rashes on my arms and face. Since I have never had reason to distrust a doctor before, I simply took his advice and expected my rash to dissipate over time.

I worked at my office until late into the evening. By the time I finally went home and looked in the mirror, I realized with great dismay that my rash had intensified and now covered my entire body from head to toe. A simple web search revealed that the threat of asphyxiation was very real. Thus, for the first time in my life, I checked myself into an ER.

I was given an IV with steroids and monitored overnight. I was also admonished for not taking any Benadryl or related medication (to which I blame the incompetent doctor). The ER physician decided to incise and drain the new infection, thus giving me the impression that ER doctors are less conservative than other doctors (no other doctors had recommended draining such a small infection). I was also prescribed new antibiotics, this time clindamycin, as well as various medication for my allergy.

Luckily, my condition gradually improved from that point until the infection finally cleared up. A culture test revealed that the bacteria was not staph, but some unidentified species (which increases the probability that I contracted it from India). I am now the proud owner of a couple new scars, a gamut of wound treatment supplies, excess medication that I'm allergic to, a newfound respect for competent doctors, a distrust of doctors who recommend no treatment, a modicum of common sense, and a healthy fear of all things microbial. All in all, it was a very memorable experience.

Having search engines learn from usage data is good for everyone.

2011-02-12T19:41:00.013-05:00

As many already know by now, Google and Bing have been going back an forth regarding what Google perceives as cheating by Bing (see articles here, here, here, here, and here).

The situation can be summarized as follows.

Over the past few months, Google has discovered that many of Bing's search results have become disconcertingly similar to Google's.

Some Internet Explorer or toolbar users send usage data to Microsoft. This includes what search results they clicked when searching using Google. Google thinks perhaps Microsoft has been using this data to augment Bing's search results.

Google then conducted an experiment whereby some rare queries had bogus results manually injected into the search results. A number of Google employees then issued these searches and clicked on the bogus results while using Microsoft Internet Explorer.

After a short waiting period, a small fraction these bogus results appeared on Bing's search results as well. This confirms that, when other relevance signals are weak or non-existent, the only active signal in Bing's ranker seems to be whether or not users clicked on the search result on the Google results page.

As a result, Google is accusing Microsoft of cheating by copying Google's search results.

My response: of course Bing uses that kind of data, how could they not use such a valuable resource? I actually find it surprising that Google didn't expect Microsoft to be doing this already.

Numerous studies in the literature have provided us with overwhelming evidence that mining click data is one of the most useful signals for improving search quality. If I worked at Bing, I would be pushing to make use of such data.

Many other search companies use click data on third party search results as well, for example Surf Canyon. Surf Canyon has an installable add-on that can dynamically re-rank search results on Google, Bing, Yahoo!, and Craigslist. This re-ranker is, of course, trained in part using click data harvested via their toolbar from users issuing queries on other search engines. Surf Canyon also has a native search engine, which I expect is also optimized using click data on Google's search results gathered from their own toolbars. That is basically the exact same thing as what Bing is doing. Now, none of these other search companies come close to the size of Bing, so maybe Google just didn't care or notice until Bing started doing this in a more obvious way.

I personally think that I should own my search logs. If I am allowed to share my usage data with any company of my choosing, then I think that's a win for everyone (except perhaps for the company currently holding a monopoly over the usage data, of course). As mentioned elsewhere, this would lower the barrier for competition and innovation.

In reality, clicks on search results make up only a small part of the equation. Suppose a Google Chrome user is sending usage data to Google. Google sees a log where the user

1) issued a query on Google
2) clicked on a search result
3) immediately issued the same query on Bing
4) clicked on a search result
5) browsed around on the landing website for 15 minutes

Would Google ignore this entry simply because it contains a Bing query? I think existence of the Bing query is almost beside the point in this case, because harvesting just clicks on search results does not tell the whole story. Usage data also includes the actions users take after leaving the search results page, which can be just as valuable (see example study). Leveraging usage data of all varieties is the future, and it benefits everyone.

In Hyderabad

2010-12-15T21:20:00.005-05:00

Greetings from Hyderabad, where I've been spending the past week visiting EMRI. This is my first time traveling to India, and the trip has been both busy (due to work and travel) as well as refreshing.

Having spent considerable time in China, I figured that little would surprise me about India. This has generally held true, although there have been some eye openers.

First things first: the traffic. Given the population density and a relative lack of policing, it's not surprising that traffic conditions are a bit hectic. But it still caught me off guard to see cars driving against the flow of traffic, or the numerous pedestrians crossing the street through traffic.

The next key topic, of course, is the food. As I've come to understand, the majority of "authentic" Indian cuisine in the US are north Indian. Hyderabad is located in the state of Andhra Pradesh, which is considered south Indian. Most dishes here are so-called wet dishes, and thus must be eaten with rice or bread. The food has been delicious, although I'm not used to eating such a large amount of carbohydrates. Most meals are served with a huge bowl of rice per person (of which I typically only eat a small fraction, which feels a bit wasteful).

Almost all dishes are spicy, so we typically eat a small amount of yogurt or curd at the end of meals to help ease digestion. Even breakfast foods are spicy! I do love spicy food, so it's been largely enjoyable, with the single exception being the tandoori seekh kebab I had as a late night snack. Don't try that one at home, kids.

As part of the CMU contingent, I've had the pleasure of meeting a number of government officials of Andra Pradesh (including the collector). It seems everyone gives out business cards at such meetings, which unfortunately left me out of the loop in that particular social exchange. Being too reliant on the digital world has its drawbacks when traveling beyond its confines.

I've been having some trouble understanding the Indian version of English (or Hindish). The regular lexicon is a bit different (for example, they use "revert" instead of "reply" when writing emails), which simply takes time to get accustomed to. The more challenging part is when people speak half-Hindi half-English, or Hindi-slurred English, phrases that I have simply no hope in comprehending. I imagine I'd be in a similar boat in Singapore with the Singaporeans' Chinese/English hybrid Singlish language (even though I speak both Chinese and English).

I've also been pleasantly amused to be mistaken by some locals as being from northeastern India (basically on the border with Nepal). I don't think I look very Nepalese, but I suppose such a mistaken association is understandable with people who don't have much exposure. (Aside story: when my very Caucasian friend John Carrino visited China with me in 2008, my aunt managed to convince some poor peddler that John was a family member of ours from western China, where people are more Turkish than Chinese).

[EDIT] It turns out the joke's on me. I was likely confused for people from Arunachal Pradesh, which is located on the far northeastern handle of India past Bangladesh and bordering on China and Burma.

I travel back to the US in a few days to celebrate the holidays. Fortunately, my family now lives in Phoenix, so I can ease the transition back to cold weather climates. Winters in Hyderabad are very mild, with temperatures in the high 70s and into the 80s, which makes it a great place to visit during the winter (and I imagine a pretty miserable place during the summer).

An Implication of Heavy-Tailed Distributions

2010-10-28T18:19:00.003-04:00

What is the probability that a search engine user that issued one query will issue a second query in the same session? For example, the user could first search for "best cars to buy" and then follow up with a search for "Toyota vs Honda". Similarly, what is the probability that a search engine user that issued two queries will issue a third query in the same session? How do these two probabilities compare?

There are three possible answers to the last question posed above. The first probability could either be greater than, less than, or equal to the second probability. To answer this question, one needs to consider the distribution of issued queries in a given query session, which is known to follow a heavy-tailed distribution

One of the most prevalent phenomena in our society is abundance of heavy-tailed distributions. For those unfamiliar with the concept, a heavy-tailed distribution is simply one whose tail is not exponentially bounded. Heavy-tailed distributions arise basically everywhere. For example, the distribution of phone call activities is almost always heavy-tailed for any call network (e.g., the the number of people who made X calls per month, F(X), is heavy-tailed in a dataset I'm examining). The number of citations any research paper receives also follows a heavy-tailed distribution.

Coming back to search engine users, let P₂ denote the probability of a user issuing a second query conditioned on already having issued one query, and let P₃ denote the probability of a user issuing a third query conditioned on already having issued two queries. Then, assuming that query session length follows a heavy-tailed distribution, we can conclude that P₃ > P₂. In fact, any distribution which results in P₃ being equal to or less than P₂ must have an exponentially bounded tail.

What this means is that, the more queries a user issues, the more confident we can be that the user will continue issuing more queries in this particular search session. This certainly has implications on how we can think about automatically incorporating session-based memory into information services such as search engines. And in fact, if we can predict when the user is about to engage in a long information gathering session, then the search engine can potentially preemptively show the user a more diverse set of results (i.e., exploration) so that the user's feedback (e.g., which results she clicks on) can help the search engine produce more relevant results in down-stream queries.

This topic came up during a discussion with Paul Bennett.

Simple Experiment Design Question

2010-10-20T21:57:00.002-04:00

Consider the following hypothetical experiment to examine whether taking a headache pill reduces a headache. The 40 participants in the treatment group took a pill for their headache, and 50% of them have been cured. The 40 participants in the control group did not take the pill, and only 40% them have been cured. The results are in the table below. It seems clear that the pill helps reduce headaches.

	Headache Gone	Headache Not Gone	% Cured
Pill	20	20	50
No Pill	16	24	40

However, suppose that this experiment did not properly randomize the assignment of subjects to the treatment or control groups. How can a confounding variable (e.g., the gender of the test subjects) invalidate the outcome above?

Interactive Machine Learning

2010-10-09T11:53:00.026-04:00

Ever try to do some information gathering on a topic (e.g., buying a car, finding a good restaurant, researching some historical event), only to be frustrated by the inane way commercial search engines just spit out rankings after rankings of useless results? One reason why this happens is because existing systems, while they can make static predictions such as rankings very efficiently (witness Google Instant), ignore contextual information as users interact with the system.

Suppose I'm looking for that perfect restaurant for a first date. I might go on Yelp and do a few searches with some typical keywords (e.g., "romantic, $$$, good wine"). Sure, Yelp does a reasonable job there, if the end goal is to just efficiently provide a reasonable ranking of restaurants for the query.

But I care about so much more than that. I want to understand the ambiance (I want something romantic but also lively), parking issues (I hate not being able to find parking), service quality (I drink a lot of water, and hate having my glass unfilled), and types of food (I prefer having healthier options). Essentially, I want to interact with someone who knows the wisdom of the crowds (e.g., the reviews) and have that person help me quickly find what I'm looking for.

How would a restaurant/dating expert answer these questions? Such an expert would probably have a few top notch (and diverse) recommendations in mind, and help this aspiring young romantic to zero in on the right selection. If there turns out to be some previously unrevealed constraints (maybe I want it to be close to the lake front, so we can go for a stroll afterwards), then the expert can adapt accordingly.

Basically, if we want to achieve the same type of efficiency with an information system, we'd need algorithms that can predict a dynamic plan for the entire user session.

Given the rapid increases in computational resources and the efficiency of machine learning algorithms, it's worth asking what would be possible if one could run sophisticated learning and inference algorithms in (near) real-time. Could one design algorithms and systems that can efficiently predict good dynamic plans?

One of the major strengths of more sophisticated learning algorithms is the fact they can train more flexible models. Sure, computers can play chess in real-time, but chess is actually a very rigid environment with limited ambiguity.

The most powerful learning algorithms in our toolkit are supervised learning algorithms -- those that train models whose predictions most closely match some target labels. Historically, these target labels have been acquired through manual annotation (e.g., a bunch of experts in some domain label a document as good or bad). This is obviously expensive and limits the scope and applicability of such approaches. Commercial search companies employ raters through interfaces similar to Amazon Mechanical Turk to collect millions of these types of annotations, but that is not a viable solution for most applications. Furthermore, it would be wildly intractable to collect sufficient training data to cover all contexts and use cases.

But suppose a system could collect feedback dynamically throughout the session. For example, suppose I could flag a few results as "good" or "bad" and tell Google or Yelp to (instantly) re-train the ranker and give me a new ranking using that feedback. If the results look better but not perfect, I can unflag some of the earlier results or maybe flag some of new results, rinse and repeat. And, of course, many HCI researchers are interested in designing new ways of exploring information (e.g., Microsoft Pivot), and that can lead to different ways for users to provide feedback to the system.

The key thing to note in the above scenario is that the system receives feedback that is directly tied to the specific usage context (e.g., one search session). As such, the system doesn't have to learn to optimize for every possible context a priori, so long as it can react quickly and intelligently to the real-time feedback the user is providing.

One important modeling or theoretical question here is, given that I expect the user to flag some results as good or bad, to what extent should the system make its initial recommendations to maximize its information gain? If we think the user is willing to go through, say, three rounds of feedback, then maybe we should only care about providing the most relevant results after the third round and spend the first two rounds showing results whose feedback would cut the version space as much as possible. But on the other hand, suppose that for some queries we're very confident that the user is only interested in one of a few results. In that case, maybe we should just show those results? This is not that different from the restaurant/dating expert deciding between asking more clarifying questions about your preferences or making a few recommendations based on her understanding of what you want. But how does one quantify this trade-off between exploration and exploitation in a way that is both programmable and an appropriate characterization of user utility?

In any case, it looks like real-time and interactive machine learning is on the horizon. And when it arrives, I expect it to be here to stay.

Simple Convexity Question

2010-09-13T10:59:00.001-04:00

Suppose we have functions f : R -> R, and g : R^N -> R, and that both f and g are convex. Is f(g(x)) also convex? If not, then under what conditions is it convex?

Wordled

2010-09-12T15:38:00.001-04:00

I decided to generate a Wordle of all the papers I wrote during my grad school years. Not to state the obvious, but it seems I like to work on learning ranking/retrieval functions =)

(You can click on the image above to see a bigger version.)

Latent Variable Models for Sentiment Classification

2010-09-11T18:28:00.007-04:00

One of the major limitations to applying machine learning to train complex prediction models is the need for "training labels". For example, when classifying the sentiment of a document (e.g., a movie review, or a speech), we often base our classification on only a portion of the document. In particular, a movie review typically contains so-called "objective" components such as a summary of the plot of the movie, but such components do not provide information regarding the sentiment (e.g., positive or negative review).

It would be nice to be able to encode such structure into the prediction models we want to train using machine learning. In fact, it is known that properly incorporating such structure can improve the prediction accuracy of machine learned models. However, this is where the limitation kicks in. To train such a model using conventional machine learning techniques, you'd need to acquire manually labeled documents with both the overall sentiment, as well as the sentences of the document that explains the sentiment (e.g., the sentence, "I love this movie!"). And you'd likely have to do this separately for each different domain (e.g., the distribution of words typically used to express sentiment in movie reviews is probably different from the words congressmen use to support or oppose a bill). This type of data is rather costly to obtain, and is often inaccurately labeled by the human labelers.

This is where latent variable models come to the rescue. Latent variable models essentially assume that some of the variables in your model are hidden and cannot be observed. For example, suppose I only have training data with sentiment labels, then the hidden variables correspond to the sentences which best explain the sentiment of each document. As you might imagine, this is a much harder learning problem than knowing all the training labels to start with.

Fortunately, there's been a growing body of work showing just how to train such latent variable models in a way so as to maximize final prediction performance. In a newly accepted paper with Ainur Yessenalina and Claire Cardie (to appear at EMNLP 2010), we've shown how to apply these methods to sentiment classification with really nice results.

Without sacrificing sentiment classification performance, our approach can actually learn to extract the best supporting sentences despite not knowing that information a priori. In fact, since we make good use of this structural assumption (that there is a subset of sentences that best explains the document-level sentiment), our model actually achieves better predictive performance than previous approaches, sometimes substantially.

We've also released the source code and data, so please check them out if you're interested in playing around with our approach.

I particularly like the example given in Table 4 in the paper, which is also shown below. This speech is from the US Congressional floor debates transcripts, and was made in support of the Stem Cell Research Enhancement Act (so the sentiment classification is positive, or "yea"). The best supporting sentences identified by our model is shown in bold face, and the least "subjective" sentences are shown in underline.

Mr. Speaker, I am proud to stand on the house floor today to speak in favor of the Stem Cell Research Enhancement Act, legislation which will bring hope to millions of people suffering from disease in this nation. I want to thank Congresswoman Degette and Congressman Castle for their tireless work in bringing this bill to the house floor for a vote.

The discovery of embryonic stem cells is a major scientific breakthrough. Embryonic stem cells have the potential to form any cell type in the human body. This could have profound implications for diseases such as Alzheimer’s, Parkinson’s, various forms of brain and spinal cord disorders, diabetes, and many types of cancer. According to the Coalition for the Advancement of Medical Research, there are at least 58 diseases which could potentially be cured through stem cell research.

That is why more than 200 major patient groups, scientists, and medical research groups and 80 Nobel Laureates support the Stem Cell Research Enhancement Act. They know that this legislation will give us a chance to find cures to diseases affecting 100 million Americans.

I want to make clear that I oppose reproductive cloning, as we all do. I have voted against it in the past. However, that is vastly different from stem cell research and as an ovarian cancer survivor, I am not going to stand in the way of science.

Permitting peer-reviewed Federal funds to be used for this research, combined with public oversight of these activities, is our best assurance that research will be of the highest quality and performed with the greatest dignity and moral responsibility. The policy President Bush announced in August 2001 has limited access to stem cell lines and has stalled scientific progress.

As a cancer survivor, I know the desperation these families feel as they wait for a cure. This congress must not stand in the way of that progress. We have an opportunity to change the lives of millions, and I hope we take it. I urge my colleagues to support this legislation.

The thing I find cool is that the least "subjective" (or equivalently, most "objective") sentences could plausibly have come from a speech made in opposition to a bill limiting stem cell research. That is to say, these sentences don't actually reveal much about the speaker's stance towards the specific bill in question.

In Pittsburgh

2010-09-05T22:38:00.000-04:00

Greetings from Pittsburgh! I'm happy to report that I've successfully defended my dissertation and moved into my new apartment. This move feels significantly different from previous moves I've done. This is due to a few reasons. Most obviously, it's my first as a non-student.

Another reason is that, due to the apartment being unfurnished (a first for me), I decided to also take this opportunity to practice home-making. Now, after a three hour tour through Target and a five hour tour through IKEA, I've completed most of my furnishing tasks. This whole experience is taking much longer than I'd anticipated, but I suppose will likely provide good practice for "life".

Some other notes worthy of highlighting include an adventurous drive from Ithaca to Pittsburgh. It turns out I just happened to pick the stormiest day in weeks to move, and having gusts of wind blow upon a tall vehicle like a U-Haul truck while driving on hilly highways with very limited visibility is not the best idea.

I start work either tomorrow or the day after. I'm having some difficulty discerning whether I should go in on Labor Day. If all the administrative and tech support staff are off tomorrow, there's nothing I could do tomorrow at work anyways.

Reviewer Anonymity Risk

2010-08-21T18:40:00.007-04:00

I'm on the program committee for the upcoming ACM Conference on Web Search and Data Mining (WSDM). The reviewing process is done completely via the EasyChair system. One seemingly nice feature is that in any descriptions of submitted papers, clicking on the author's names will take you to their personal web pages (if the authors submitted such information). For example, here is the info page for a submission that I'm a co-author on:

Note the links to the authors' home pages at the bottom of the page. (FYI the primary reason the submission information is blurred out is because I have not yet obtained permission from my co-authors allowing me to broadcast this information.)

While this seems like a pretty convenient feature, using it can actually carry a non-trivial degree of risk, depending on your institution. The reason for this is because many (most?) researchers track visits to their web pages. For example, when I click on the my name in the page shown above, my webpage stat tracker logs the following event:

Location: Ithaca, New York, United States
IP Address: Cornell University (128.84.154.13)
Page Loaded: www.yisongyue.com/
Referrer: www.easychair.org/conferences/submission.cgi?submission=468064;track=24725;a=a09cfab06091
Time: 21st August 2010 17:23:23

For a variety of reasons motivated by preserving fairness, the reviewing process is supposed to be blind, i.e. the authors shouldn't know who their assigned reviewers are. But, basically, what the situation described above means is that unsuspecting program committee members and paper reviewers can reveal significant information to the authors regarding the identity of the reviewers. For example, if I see an event in my logs like the one above from a relatively small school with perhaps one or two people in my field of research, then I can actually make a very strong inference about who one of my reviewers might be.

Now, this issue can be largely avoided if one uses a less conspicuous network than that of a university or a tech company (e.g., Starbucks), but it still poses a risk nonetheless. I certainly won't be clicking on these links anytime soon.

This also speaks to the larger issue of how to make queries or collect information anonymously, and what the social implications are for breaking anonymity. Privacy is a huge topic that I cannot hope to do full justice on. But, as is my wont, I'd like to touch on another completely different setting.

Most online social networking sites do not disclose pageview information, so you can't see who has been visiting your profile. What would happen if such information becomes available?

A few sites, such as OkCupid, allow you to opt in to remove this layer of anonymity. Any users who opt in can then see which other users (that have also opted in) have visited their profile, but users who have not opted in cannot see this information. This leads to the expected behavior of people creating two profiles: one official and opted in, and the other for stalking.

But we needn't even need to use examples that involve technology (although they are more fun). Suppose you could goto parties and freely check out attractive potential dating partners without anyone noticing. How would that change your behavior? (As a side note, I was once asked by someone to lend him my sunglasses at a social gathering, and I'm almost certain it was for this exact reason.)

Visiting Yahoo!

2010-08-19T13:57:00.001-04:00

It's been a whirlwind month for me (which has been contributing to the lack of blog activity). I'm currently visiting Yahoo! Research for a week or to wrap up some projects I've been collaborating remotely on.

My opinions are mostly limited to my own experiences, but Yahoo! seems to have an incredibly collaboration friendly culture. I've already met many (machine learning) researchers here, and I also haven't had too much trouble getting set up. In fact, I've basically hit the ground running (which is, admittedly, to be expected in this case since I have a very narrow project that I'm spending a relatively short amount of time on).

While the turnover of core search to Bing has inevitably caused some turmoil, it seems that many researchers here are excited to be turning their attention to other, newer, projects. After all, virtually everything is an opportunity.

I return to Cornell in two weeks time to defend my dissertation, after which I'll be immediately starting my PostDoc appointment at Carnegie Mellon.

Towards Interactive Information Systems and Automated User Interface Design

2010-07-28T21:51:00.016-04:00

Attending conferences like SIGIR provides a good opportunity to reflect on the major (perceived) trends in the field. One current trend that I find particularly exciting is progress towards more adaptive and interactive systems.

The ten blue links metaphor has dominated the design of information retrieval systems for about twenty years now. This is really a shame given that displays are large enough and computers are fast enough to accommodate richer forms of content display. Rich interactive frameworks are particularly useful for information discovery or vaguely targeted browsing tasks, where most of the useful information is not concentrated in just a few results.

For example, suppose I wanted to learn more about the history of research progress regarding a certain family of diseases. This is very much an exploratory information need, where I don't actually even know what I'd find interesting (until I see it), and being able to interactively re-organize and visualize the data (e.g., via different clusters or histograms, etc) might prove immensely useful.

One challenge is, of course, finding the right user interface. Microsoft's Gary Flake gave a very impressive demonstration of an experimental ZUI (zoomable user interface) called Microsoft Pivot during an invited talk at SIGIR. During the demo, Flake showed how one can seamlessly navigate and re-organize a large data collection to better understand its content.

Such interfaces should become increasingly easier to develop in the future as more SDKs become available.

Flake's demonstration did come with limitations. Most obviously, there were many things that were specially engineered for a particular type of data or application, and things obviously won't work so smoothly for other settings and tasks (this was irrelevant with regards to his talk, since he was presenting the only the user interface, and not relevance models).

This leads me to consider the two-pronged problem of figuring out not only how to find the most useful information, but also how to best present it to the user. It makes me wonder if we might be headed towards an era where machine learning approaches are used to automatically test out different ways to organize and display content (i.e., automatically learning to optimize the user interface). To some extent, people are already doing this, such as for nexus pages with lots of mash ups (I believe the folks at Yahoo! call this the page optimization problem).

It remains to be seen to what extent the two aspects (content/relevance and display/interface) can be addressed separately. Given the trend towards interactive systems such as Pivot, I suspect that significant improvements to the user experience will be realized when we have effective methods that can model both content and display simultaneously (i.e., what to show and how to show it).

Ponderings on China

2010-07-09T03:26:00.004-04:00

One of my pet peeves about traveling around (many parts of) China is the disregard for outdoor maintenance. It stands as a stark contrast to the impressively upscale areas that can be found in the world-class cities such as Beijing and Shanghai.

There appears to be (at least) two inter-related forces contributing to this effect. First, there are simply too many people. Even a small city such as Fuyang (where my dad's family resides) has a population of over two million. Despite China seeming like a large country geographically, most of the western regions are uninhabited. As a result, most of the 1.4 billion Chinese folk are crammed along the eastern seaboard.

A second related reason seems to be cultural. Many people seemed to be used to (or resigned to) the idea of the public areas being under-maintained. For example, my aunt's housing complex in Beijing looks a little rundown from the outside. Litter is strewn about the parkway, the main corridors look like unfinished, and the elevator seems like on the verge of breaking down. But her actual home is a nice and tidy two bedroom unit. I asked my aunt why no one bothered to renovate at least the main corridors (imagine the front lobby of a condo building in Chicago being rundown/unfinished but the individual units are nicely finished). She replied that not only does there not exist an organization responsible for such projects, no one really cares enough to pay more for the upkeep of these more public areas.

Robin Hanson recently wrote a blog post describing what he defines at the two key disputed values in any given society. One point he makes is particularly relevant here,

Cultures where invasion was less an issue tended to evolve family oriented values, while cultures where invasion was more common focused more on larger community solidarity.

Certainly, China has experienced relatively few external invasions in its history, and I know from personal experience that the country is very family oriented. And given the huge concentration of people (both rich and poor) living in the same area, the forced mixing of the populations might necessarily encourage people to be more limited in their domain of care.

For example, even if these housing complexes in Beijing were well maintained from head to toe, most of its inhabitants would still have to pass through less savory parts of the city during their daily excursions. So it's not that big a difference said less savory parts extended a little closer to home. It's a little reminiscent of nice university communities surrounded by less affluent neighborhoods (e.g., Yale and UChicago), albeit to a much larger degree.

But of course, there are some things that are universal across all cultures. Cute toddlers, for instance.

Brief Hiatus

2010-06-11T23:13:00.001-04:00

This past month has given me relatively little free time. With all the commotion surrounding commencement (family visiting to see me in a fluffy red robe, etc), saying good-byes to friends who will soon part ways, and wrapping up some research projects before I take a long vacation in China (where I won't have easy blogger access and will otherwise be distracted by other festivities), I haven't had time to really relax and ponder random ponderings. As such, this blog's activity level will be pretty low (possibly in hibernation), until I return from China.

Running Online Experiments

2010-05-11T21:31:00.001-04:00

I've been spending some time recently thinking about the problem of running online experiments for eliciting user preferences on search engines. The setup is pretty straightforward:

1. Our search engine has a pool of K retrieval functions (a retrieval function returns a ranking of results given an input query).

2. We want to figure out which of these is the best one. The way we elicit preferences from users is by showing users an interleaving of two rankings (computed from two different retrieval functions). Then we can interpret users' clicks as a preference vote for one of the two retrieval functions.

3. However, we do pay an exploration penalty because we might be subjecting users to suboptimal results as we're performing our online interleaving experiments. Since the goal is to ultimately provide the best possible service to our users overall the lifetime of the search engine, there's an exploration versus exploitation trade-off at play here.

The mathematical framework I've been working on to model this setting is called the Dueling Bandits Problem. The linked to paper, we presented an algorithm that is optimal in the worst case under relatively mild assumptions.

Now, there are two ways to make interesting refinements to a problem statement (i.e., to consider a special case of the original problem). Not surprisingly, they are to make the problem easier or harder. Problems become easier when the refinement introduces additional structure that removes difficult to solve cases -- this is in my experience the more common setting. Problems become more difficult when the refinement restricts the strategy space with which you can design an algorithmic solution. This is especially pertinent when the previously known algorithm that solves the original problem is no longer feasible with the restriction of the strategy space.

For example, we might know a similarity metric on our set of candidate retrieval functions. For example, all the retrieval functions that pay special attention to navigational queries could be "closer" to each other than retrieval functions that do not. This might allow us to remove various worst case scenarios and prove stronger performance guarantees. Such a refinement would make the problem easier.

In the original statement of the Dueling Bandits Problem, our decision algorithm is allowed to spend as much time exploring as necessary. This is actually an undesirable property for many search engines, due to management and infrastructure issues. For example, we may only want to allocate a fixed amount of exploration budget, after which we must decide on which retrieval function to commit to for our production system. Such a refinement would make the problem harder, since it restricts the strategy space of our decision algorithm.

In this case, it seems that making restrictions (that make the problem harder) motivated by real-world concerns is the more interesting direction to pursue, since otherwise these results are much more likely to be confined in pure theory-land.

Coin Weighing Problem

2010-05-01T23:13:00.000-04:00

We are given nine coins. The coins are identical except that one coin is slightly heavier than the others. This can only be detected using a scale (i.e., given two groups of coins, the scale will tell you which group is heavier, or that both groups have equal weight). How many scale weighings are required to isolate the heavy coin?

The Multi Armed Bandits Problem

2010-04-24T18:39:00.002-04:00

Three armed bandits are in a gunfight with each other. The gunfight proceeds in a sequence of rounds. In each round, each bandit chooses one of the bandits to target, and all three shoot simultaneously. Each bandit (while alive) shoots once per round. Each bandit has a certain accuracy, and the probability of hitting or missing a target is independent of the target (i.e., it doesn't matter who the bandit is shooting), and also independent of all other shots.

Each bandit knows the others' (possibly randomized) strategy. Any randomness used by any of the bandits is hidden from the other bandits (i.e., the distribution of actions that defines any bandit's strategy is independent of the other bandits' strategies).

The gunfight ends when at most one bandit remains alive. A bandit "wins" the gunfight if he survives until the last round (he is either the last bandit remaining, or he survived until the very end when all remaining bandits are simultaneously killed). In the degenerate case where all three bandits have 0 accuracy (and thus the gunfight never ends), then all three bandits are declared winners.

What is the Nash equilibrium?

(Thanks to Vasu for helping come up with the name.)

Pittsburgh Bound

2010-04-14T09:53:00.002-04:00

I have decided, upon the successful defense of my thesis in at the end of this summer, to take up a postdoctoral position at the iLab in the Heinz College at Carnegie Mellon University, where I'll be working with Carlos Guestrin and other researchers there.

It's a great opportunity to conduct interdisciplinary research and leverage large datasets, and also to work with some of the best machine learning researchers in the world. The goal, as I see it, is to explore new problem spaces, see how machine learning can make a difference, and design novel (and principled) machine learning techniques motivated by these new problems.

It's a temporary position, so I'll be looking for another job in a couple of years, but the research prospects are definitely exciting. I'm also not ready to settle yet, which reduces the downside of it being a temporary position.

Debugging Natural Language

2010-04-12T01:48:00.001-04:00

My officemate Ryan is currently in the midst of writing a research paper. A few days ago, he remarked that his first drafts are often clunky and lacking smooth logical flow.

For example, a common mistake he (and basically everyone) makes when writing a first draft of a technical report is to start using terminology before they're properly defined. These types of logical discontinuities are often most easily recognizable by someone else, as it can be hard for the writer to tease apart what was actually written from the idea that was meant to be conveyed.

If you stop to think about it, many aspects of proofreading are basically just debugging the natural language. Sure, we already have some pretty good spell checkers and grammar checkers these days. Those are akin to the syntax checkers we have in our compilers and interpreters for programming languages. But we don't have anything that checks for higher level concepts.

One could make the argument that all aspects of proofreading can be compared to debugging at some level of the abstraction, but the generality of that argument makes it in some ways less interesting. Certainly some aspects of proofreading are easier to design automated or semi-automated services for -- such as detecting whether certain terms are defined before they are first used. Come to think of it, it now seems kind of barbaric that we still rely so much on manual proofreading for such relatively low level editing assistance.

Stream Sampling Problem

2010-03-25T20:14:00.000-04:00

Assume you are processing a large stream of N distinct elements (e.g., strings, characters, numbers, etc). Your goal is to sample a set of K elements (without replacement) uniformly at random (such that all N choose K sets are sampled equally often). Devise a strategy that requires only one pass through the stream has space complexity that does not depend on N.

Dice Rolling Question

2010-03-13T03:37:00.005-05:00

I've been spending a lot of time traveling and interviewing these days. While this has prevented me from dedicating sufficient time to writing random ponderings or research blog posts, it has exposed me to a bunch of fun and interesting math problems. So I thought I'd share another one that I encountered recently.

Suppose you had two three-sided dice D1 and D2 (they do exist), and suppose that rolling such a die gives a uniform probability of resulting in 1, 2 or 3. Our rolling procedure proceeds as follows.

Step 1. ROLL D1

Step 2. IF D1=3 GOTO Step 1

Step 3. ROLL D2

Step 4. IF D1=2 AND D2=3 GOTO Step 1

Step 5. STOP

Steps are taken in sequential order unless a GOTO command is invoked. How many total dice rolls are performed in expectation?

Another Fun Math Problem

2010-02-20T07:19:00.001-05:00

Assume we have a set S of n distinct integers and a permutation Q of S drawn uniformly from the set of a possible permutations. We wish to analyze a procedure that maintains a current largest seen value C as it iterates through the permutation of S. Initially, C is set to be negative infinity (or some practical equivalent thereof). Our procedure then iterates through the elements of S in the order defined by Q. Whenever we encounter a number larger than C, we update C to be this newly seen largest value so far. How many updates should we expect to observe from running this procedure?

Fun Math Problem

2010-02-19T02:57:00.006-05:00

Evaluate, simplify, and/or approximate the following expression

SUM[ -log( b + (1-b)/(1+b^(j-1)) ) ] , FOR j=1,2,...,d

Assume that 0 < b < 1.

This popped up from a discussion I had with Nikos.

Geeky vs Hardcore

2010-02-17T03:48:00.001-05:00

I've recently heard computer programmers being described as both geeky and hardcore (sometimes simultaneously). After pondering a bit on what both terms mean to me, I've concluded that both terms are indeed appropriate descriptors for many stereotypical computer enthusiasts. Curiously, the set of qualities which evoke the use of the two words have a very strong intersection.

After perusing various definitions from sources such as Wikipedia and Urban Dictionary, I've come up with the following definition summaries (note: I am ignoring how hardcore is used to describe a music genre, and rather am focusing on its uses as a general adjective).

A geeky person is someone who is obsessed with or focused on a subject area to that's considered outside of mainstream. These subject areas tend to be intellectual by nature. Examples include being a hacker geek or a music geek.

A hardcore person is someone who is intensely or relentlessly focused on a task or goal (this is taken almost verbatim from the first definition on Urban Dictionary). It is typically associated with physically oriented activities (e.g., hiking or camping), but has been increasingly used to refer to programming and other technical activities.

Those who are better versed in language than I will already know that the word geek originally referred to a carnival performer and shares its roots with the word freak. As such, it has historically been associated with non-mainstream activities that are low status.

On the other hand, hardcore is used to describe admirable activities that require such extreme endurance or resilience that it necessarily falls outside of mainstream (since otherwise it wouldn't be that extreme). As such, it has historically been associated with high status activities. This doesn't seem to have changed much.

I'm not an expert on sociology or cultural psychology, so I can't comment very deeply on what this all means. But at the very least, one might plausibly interpret this as evidence that we as a society are placing more value in technical skills. And as such, I do find something comforting from this ongoing merging of the two definitions.

Life is not a zero-sum game.

2010-01-29T03:04:00.002-05:00

Harvard professor Michael Sandel is well known for teaching his Justice course, which covers a number of thought provoking philosophical and moral issues. This past semester, Professor Sandel decided to record his lectures for widespread consumption, and I must say that these lecture videos are an absolute delight to watch.

It's beyond the scope of Professor Sandel's course, but one flaw in his discussions is that they all seem to be implicitly assuming some kind of zero-sum game (i.e., the prosperity pie is relatively static, and our chief concern should be on how to fairly redistribute wealth). For some issues, this hardly matters and is mostly beside the point. But in many other cases, I see this as a major limitation.

Take Episode Eight, for example. This episode deals with issues such as getting a "fair" start to life and proper taxation levels of the wealthy. Policies made based on these issues clearly have ramifications regarding how quickly we as a society can make progress and grow the aggregate prosperity pie for everyone.

After all, what's the point of expending a great deal of effort in enforcing fairness (for a static society) when (for example) a relatively poor person 10 years from now could be as wealthy as a relatively rich person from today? Maybe we should be more focused on figuring out how to get that 10 years down to 8 or 5 years.

As an engineer and technology enthusiast, I view this issue through a much different lens than people with backgrounds like Professor Sandel. Technology levels the playing field by making it easier for those who start off disadvantaged to catch up. For example, with the magic of YouTube, even the poorest among us can now watch lecture videos of Michael Sandel's Justice course at the esteemed Harvard University.

Furthermore, once you take raw materials and natural resources out of the equation, technology is then the SOLE driving force behind increasing prosperity (socio-political issues notwithstanding). In light of this, maybe enforcing static fairness shouldn't be the focal point, but rather figuring out how to grow the prosperity pie for everyone as quickly as possible. Keep in mind also that the rate of technological development is now faster than ever, and is expected to increase in the years to come.

In short, it's a good time to be an engineer =)

Sam Roweis 1972 - 2010

2010-01-17T21:26:00.006-05:00

Last week, Sam Roweis passed away after falling from his 16th floor balcony. A professor of computer science at New York University, Sam was highly regarded for his research in the field of machine learning. A memorial website has been set up in remembrance and celebration of his life.

I only just met Sam for the first time last November at a machine learning symposium in NYC, but had attended a few of his talks before that. He'd always struck me as very intelligent and enthusiastic about research -- qualities that many others (who knew him much better than I) also noted.

After reading through a number of testimonies on the memorial website, I realized that in many ways Sam was the kind of person that I hope to become. The circumstances of my discovering this are indeed tragic -- as they often are, since such personal qualities tend to become so publicly celebrated only after tragedy strikes (e.g., Randy Pausch).

Sam is survived by his wife and two children. Those wishing to help are invited to make donations to the Sam Roweis Memorial Fund.

A CERN for Information Retrieval?

2010-01-09T02:40:00.014-05:00

One major road block that information retrieval researchers presently face is a dearth of suitable live experimental testbeds. It's becoming increasingly apparent that offline data collection techniques (e.g., acquiring human judgments) cannot scale with growing demand and are insufficient for answering many of the emerging interesting research questions. After all, the ultimate goal is to design the best possible retrieval system for a particular search domain (e.g., medical, patent, etc), but asking humans to go through and label thousands or millions of documents for each domain seems impossibly inefficient.

The primary benefit of having live experimental information systems is the ability to try out different retrieval functions and subsequently analyze changes in user behavior (i.e., does the new retrieval function work better or worse?). Being able to validate under live experimental conditions can yield valuable new insights and exciting new progress in pushing the state-of-the-art in information retrieval.

At present, there exists a smattering of digital libraries which host search services that allow for online experimentation. For example, here at Cornell computer science, we manage the search service for arXiv, with the goal of developing new methods for interactive online experimentation and learning. However, for most academic researchers in information retrieval, it is very difficult to gain access to such a system.

This begs the question of whether the research community could/should build and maintain a centralized information system as a communal research tool -- much like how high energy physicists collaborate to run experiments at CERN, or how networking researchers collaborate to maintain PlanetLab. Such a system needs to be somehow centralized or consolidated in order to draw in a sufficiently large user base -- otherwise it would be uninteresting for analysis purposes.

This idea has been proposed in the past, though it's never really picked up much steam. The biggest drawback against developing such a system is, of course, the manpower required to get it off the ground. However, with the recent arrival of new platform technologies (e.g., Yahoo! BOSS), this barrier appears lower than ever.

Consider the following scenario:

1. Individual research groups use platforms such as BOSS to efficiently build their own retrieval functions and search servers. These servers have no direct users.

2. A centralized search portal has a large user base. Users are bucketed into different experimental conditions, and different research groups own different experiments. For each experiment, the portal directs queries to the appropriate research group's search server.

3. Researcher groups take turns running experiments and sharing the common user base. Some kind of cost-sharing scheme would probably be required as well.

The good news is that the above scenario doesn't sound impossibly difficult or time consuming. And of course, this scenario merely scratches the tip of the iceberg. After all, there is so much more we could be experimenting with beyond just the ranking function. But the more exotic our methods get, the more we need to be grounded in reality. That is why having such a centralized user-sharing experimental testbed seems so appealing.

Comparing Machine Learning and Statistics

2009-12-19T12:59:00.014-05:00

There's a tendency amongst the machine learning folks to view statistics as simply more of the same, except without the computational flavor. After all, the methods employed are largely the same (at least they are nowadays with the rising popularity of statistical machine learning), with things like logistic regression, decision trees, Gaussian processes, density estimation, bagging (bootstrap aggregation) and many other approaches each taking turns being the flavors of the month.

When competing for computer science style research jobs, there's likewise a tendency to view statistics grad students as being handicapped due to the difference in research style and much lower publication frequency. The basic argument is the following: computer science Ph.D. students partake in many smaller sized publishable projects (or cut up their larger research agenda into smaller publishable units) which ideally can distinguish the best students from their peers; whereas statistics Ph.D. students focus more on core mathematics and detailed analysis of scientific data, which often leads to much slower publication rates and thus less information to distinguish between higher and lower quality students.

But it now seems to me that such considerations only really matter when evaluating candidates for core research positions that push for truly automated approaches in problem solving. There are many other jobs out there which are not purely computing focused, and for such positions this handicap might not exist or even be flipped in the other direction.

People working in machine learning are no doubt aware that many scientists are suspicious of machine learning models due to the difficulty in interpreting said models. These scientists would generally rather work with models that are much simpler but perhaps loses a few percentage points in the performance measure used to train the machine learning models (e.g., 0/1 loss -- which may or may not be that informative or crucial). Machine learning people understand this as regularization (i.e., preferring simpler models using some measure of model complexity), but in this case the regularization criterion is human interpretability.

One important reason for this is that such models are meant to be used as one tool or indicator amongst a collection of indicators for measuring quality or performance along a variety of axes. Because of the difficulty in appropriately combining these different desiderata into a single performance measure, domain experts (e.g., scientists, financial traders, marketing and campaign managers, user interface designers, etc) would much rather have a suite of simpler models that they intuitively "know how to use".

Indeed, I had such an experience this past summer, when I interned with the Search Quality Evaluation Research team at Google. This team consists of a mix of developers and statisticians and is dedicated to designing and implementing new metrics (i.e., ways of measuring the quality of Google search). Given my lack of experience in this style of analysis, my fellow interns (who were statistics Ph.D. students) made much faster initial progress on their projects. From a holistic learning perspective, this experience was very beneficial for me, although the project ended up being a bit more stressful than I'd anticipated.

In summary:

-- Computer scientists seek to build systems and solve problems a principled (and thus automated) fashion. From the (current) machine learning perspective, when we have the right performance objective and sufficient quantities of data, then we can strive to build efficient and accurate models to optimize for this relatively well-formulated problem setting.

-- Statisticians seek to understand the world. The models must be interpretable and explain the primary factors contributing to some observed phenomenon. It is often very difficult to arrive at a single performance measure, in which case it is often desirable to present human practitioners with informative and intuitive models.

-- It is unclear to me that having a plethora of conference publications is actually all that useful in distinguishing between higher and lower quality students for most jobs. Due to how the economics play out, most jobs available to Ph.D. students are more applied than not, and I'm not sure if the type of papers that tend to get published at machine learning conferences are adequately informative from that perspective.

-- It also seems to me that bridging this gap between statistics and machine learning will be crucial for developing general learning algorithms an intelligent systems. Machine learning currently works quite well when the problem is well specified. The challenge is to extend such learning methods to more heterogeneous and dynamic environments.

Some Logic Puzzles

2009-12-14T14:05:00.003-05:00

1) Jack is looking at Anne, but Anne is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person?

A) Yes B) No C) Cannot be determined

---

2) Suppose four cards are placed on a table. Each card has a letter on one side and a number on the other. The visible sides of the cards are the following:

A K 8 5

Two of the cards are letter-side up, and two of the cards are number-side up. The rule to be tested is this: for these four cards, if a card has a vowel on its letter side, it has an even number on its number side. Your task is to decide which card or cards must be turned over to find out whether the rule is true or false. Indicate which cards must be turned over.

---

(taken from "Rational and Irrational Thought: The Thinking That IQ Tests Miss")

Leveling the Language Playing Field

2009-11-30T23:39:00.013-05:00

Recently, Jibbigo announced an iPhone app that can translate between Spanish and English, potentially signaling the start of a new wave of speech translation applications. While I do get incredibly geeked out about the technology itself, I'm even more excited about the emergent properties that such technologies enable.

As a general rule, I strongly support breaking down communication barriers in all of its forms -- and the language barrier is one of the largest. In this post, I am referring specifically to the non-English handicap, which can be a tremendous hurdle for non-native speakers to overcome in a world where English has become the de facto international language.

For example, students in countries such as China expend significant effort to learn passable English. Several years back, I spent the summer in Beijing as an English teacher at a small college. I was struck by the low quality of teaching. In fact, one fellow teacher asked me for help in distinguishing nouns from adverbs from prepositions, etc (I wish I was exaggerating) -- something that everyone who passed elementary school should know. What's more, these "teachers" were paid almost thirty times more than standard wage workers. It just felt like highway robbery to me.

Finding success comes from not only taking advantage of the available opportunities, but also from simply having said opportunities in the first place. Upon reflection, it seems silly, although not surprising, that knowing English can be such a competitive advantage in many parts of the world.

In a sense, one can view the fundamental problem as there simply being a scarcity or imbalance of resources. That's why such low quality teachers receive such high compensation -- because that's all that one can get to work in those situations. Technology makes it easier to enable the transfer or creation of new resources, and thus levels the playing field by growing and redistributing the aggregate wealth of our society.

I hope that, in time, this language barrier will fade into irrelevance. And of course, the idea of having universal translators is also just plain damn cool.

Scientific Blogging University Writing Competition

2009-11-01T15:26:00.002-05:00

Scientific Blogging is currently holding a University Writing Competition, which invites graduate students from the "top science universities" to submit popular science blog articles on the topics of their choosing.

I'm happy to announce that my submission, Self-Improving Systems that Learn Through Human Interaction, has been named one of the finalists. The final winner is determined through popular voting (such as by people like you). So I encourage everyone to peruse all the submissions and, if motivated, cast your vote =)

(The voting mechanism is the grey box located across from the university seal in each of the submissions. It appears to be a bit buggy.)

If you know others who would enjoy reading my article (or any of the others), let them know!

The Internet: Our Telescope Into Human Society

2009-10-18T10:51:00.043-04:00

Imagine that we've been developing telescopes to capture increasingly detailed snapshots of an alien civilization. At first, we could only detect their large structures such as buildings and roads. Based on how their cities are organized, we might infer that certain buildings or city regions are important. We might even be able to tease apart different communities just based on infrastructure information. From this, we start to formulate theories of their social behavior and organization.

As our telescopes grew more powerful, we became able to detect traces of activity, such as from transportation vehicles and eventually even from individual people. With this new wealth of information, we can further refine our understanding of this blossoming extraterrestrial society, and paint an amazingly vivid picture of their social dynamics.

Of course, we could just as easily tell the same story about our own society. Our telescope is the internet. In the early days, we relied on relatively the static hyperlink structure to infer the authoritative or trustworthy websites. We could even detect online communities by analyzing link density.

As the internet (and our dependence on it) expanded, much of its content became more dynamic (e.g., blogs, forums, Yahoo! Answers). Nowadays, we can even trace real-time online activities on sites such as Google, Twitter, Facebook, Amazon, and many others. Our numerous online activities all leave digital footprints which reflect the fine grained dynamics of our own society. Such information can be incredibly useful. For example, a Nature article published earlier this year showed how analyzing search behavior on Google can provide a faster turnaround time to detecting influenza outbreaks. As another example, a recently published PNAS article showed how mobile phone data can be used to infer friendships. That is the power of the digital medium.

We now also have an opportunity to introduce new levels of empirical rigor to many social science disciplines. In years past, acquiring social data was a very labor intensive task, often requiring months or years to collect a modestly sized dataset. Nowadays, sociologists can mine all of Livejournal or Facebook to study the global structure of things like social influence and gossip. Companies like Google constantly run auctions to determine which ads to show whenever someone issues a query -- this is an economist's dream.

Here at Cornell, Professor Jon Kleinberg is one of the leaders in studying the convergence of social and technological networks. As part of an undergraduate course he and Professor David Easley have been teaching the past few years, they have written a new textbook. From their website: "Drawing on ideas from economics, sociology, computing and information science, and applied mathematics, it describes the emerging field of study that is growing at the interface of all these areas, addressing fundamental questions about how the social, economic, and technological worlds are connected."

Incidentally, this was the topic I initially wanted to write about for the University Writing Competition hosted by Scientific Blogging. But seeing as how any meager offering of mine would pale in comparison to Jon Kleinberg's fantastic CACM article, I decided to change topics to something I'm more of an "expert" on: Self-Improving Systems that Learn Through Human Interaction.

John Hopcroft Symposium

2009-10-09T22:58:00.203-04:00

I spent most of today attending a symposium held in honor of John Hopcroft's 70th birthday. Symposium covered an incredible gamut of John's achievements as revealed through his former students and colleagues. It was a remarkable experience for the younger faculty and current students such as myself.

John has been a faculty member in the computer science department at Cornell since its early days. In many ways, he helped define the field of computer science as a separate discipline from mathematics and electrical engineering, and is best known for his seminal work on algorithms and data structures with Robert Tarjan (for which they won the Turing Award in 1986).

In addition to his early algorithmic exploits, John has also delved into a myriad of other areas such as complexity theory, scientific simulations, robotics, and now large scale data processing. It was incredibly inspiring to hear others describe John's ability to repeatedly identify important emerging areas and the subsequent boldness he displays in rapidly adjusting his own research agenda. I have personally witnessed his most recent push to develop algorithms suitable for the information age (as evidenced by his recent talks on the subject). But I hadn't realized until today just how many times he'd successfully foraged in new research directions in the past.

In all, it was a wonderful event. Many thanks to Jon and Eva for organizing it.

Our minds linked together: the case for being Borg

2009-10-04T11:49:00.047-04:00

Those who've watched Star Trek: The Next Generation will know the Borg as a race of mindless cybernetic drones bound by a collective consciousness and lacking any trace of individuality. The Borg travel across the galaxy, forcefully assimilating any civilizations they encounter into their collective. As such, they must be mortal enemies of humanity, since our sense of self (and by extension our liberty) is a large part of what makes us human -- or so we believe.

What I find so intriguing about the Borg is the fact that they can communicate so efficiently. In essence, all their minds are directly connected to each other via some kind of super internet. The most optimistic among us believe that we might also achieve such an ability within our lifetimes.

I don't think anyone would argue that explosion of information made available by the internet is a bad thing. But humans are limited both by how much information we can send, receive and manage, as well as by the amount of computation or reasoning we can perform (not to mention all the inherent biases embedded in our neural hardware). It's why advertising companies and politicians care so much about marketing slogans and grabbing mind share. It's why we can benefit from playing games with indirect social and status signaling. It's essentially why we still have so much misunderstanding in this world.

While we shouldn't hope to completely eliminate our limitations (as that would violate our current understanding of physics), there's no reason to think that we cannot create a more efficient and freer society by making it easier and faster to communicate and organize information. In fact, we are already experiencing a change in attitude regarding what we take for granted communication-wise. For instance, I already have developed an intrinsic feeling of being connected to others through the internet. When GMail went down this past summer, I actually felt lonely (probably much like how Hugh felt in Star Trek episode "I, Borg"). Whenever I don't have my phone with me, I often feel somehow naked or incomplete.

Contrary to how the Borg are portrayed, expanding communication bandwidths will likely increase our diversity of thought rather than suppress it. I think the general principle of having robustness (e.g., promoting diversity) is a lesson that's been well learned (with notable exceptions). Forward thinking companies such as Google (where I interned this past summer) and Microsoft (3 summers ago) consistently emphasize diversity of thought and creativity in problem solving. Having a more connected society will make it easier for the good ideas to surface.

There are, of course, concerns over whether we could or should integrate our minds with "machines". One reason why we're supposed to find the Borg so repulsive is due to their cybernetic nature. Rather than debating what it means to be "human", I think it's sufficient for this discussion to note that such technologies can be a life-changer for people suffering from paralysis. In addition, there is still much left to accomplish with non-invasive interfaces (such as our phones), so we still have a ways to go before we need to cross that tricky bridge of having immersive virtual environments.

Are most numbers big or small?

2009-09-26T19:37:00.001-04:00

Consider the following statement:

For any fixed positive integer threshold K, there exists a finite number of positive integers less than K and an infinite number of positive integers greater than K.

One can plausibly interpret the above statement in two opposing ways.

1) Most numbers are big. Given any threshold K, there are always many more numbers bigger than K as opposed to less than K. So if we define big using some kind of absolute threshold, then most numbers are big.

2) Most numbers are small. For any number K, there are always more numbers bigger than K as opposed to less than K. So if we define big using some kind of percentile threshold, then most numbers are small.

Just another little quirk of them good old countably infinite sets.

Turing Tests for the Internet Age

2009-09-25T17:33:00.020-04:00

In a conventional Turing test, a human judge tries to discern which of two text chat conversations is with another human and which is with a computer program. Since it was first proposed in 1950, designing computer programs that can pass the Turing test has become a milestone target of sorts for artificial intelligence researchers.

Now that the World Wide Web has enabled us to spend significant time in online environments, web-based variants of the Turing test are quickly becoming research goals with immediate or near-term practical interest. For example, when we send emails to customer service and receive coherent (and hopefully helpful) responses, we typically assume we're being serviced by a human agent. As we all know, living in a information economy drives up the price of labor. A significantly cheaper alternative would have computer programs that can automatically understand and resolve each customer's specific issues while also providing a pleasant service experience. There would imply no more bitchy emails from disgruntled human workers, much faster turn-around time on service requests, and overall increased efficiency in this sector of society.

Currently, we are still quite far from having programs that are of practical value. One way to generate interest and spur innovation is through competitions, very much like how the X PRIZE Foundation awards prize money for demonstrating various feats of technological prowess. While we should, of course, also pursue other avenues of research, I echo John Langford's sentiment that competitions give people who know how to do things a chance to distinguish themselves. In fact, there already exists a Turing test competition for computer game bots within the gaming community.

Regardless of whether we promote this research direction through competitions or other means, it would certainly be useful to have automated online services with rich interactive components (such as the aforementioned customer support service). What might be some other interesting Turing test variants that we can/should focus on?

One idea is some kind of social networking bot. For example, one might have a profile on Facebook, MySpace, or OkCupid that is controlled automatically by a computer program. Consider the OkCupid scenario, which is reminiscent of the great Mark V. Shaney experiment but taken one step further. We can actually test two experimental settings in this space, male and female. In both settings, the learning component could, in principle, be controlled by some kind of reinforcement learning algorithm with a clever strategy exploration routine. I'd expect the male version to be more proactive since males are typically expected to initiate contact with females. One could start with the obvious reward function of maintaining message threads with other people and go from there. We even have some guidance on how we might craft a suitable strategy space.

A second idea, which takes the game bot idea to its logical extreme, is a bot in a sustained virtual world that tricks human users into thinking it is also human. For example, consider World of Warcraft (WoW), which has a sustained virtual universe that features many social components that we also find in the real world (such as a vibrant trading economy). Suppose we could design a bot player which can learn to operate autonomously in WoW and interact naturally with human players. Suppose the bot player could go on quests, team up with other players, even join a guild, all while under the pretense of being a human player. This is certainly an exciting setting to experiment in since the penalty of failure is limited (we can just create another bot character), and there already exists an enormous amount of data from usage logs of normal humans that we can use to bootstrap the learning algorithm.

In both the aforementioned scenarios, it should be noted that some kind of agreement with the service provider (e.g., OkCupid or Blizzard Entertainment) is required. Even if we ultimately weren't allowed to run live experiments, I'd imagine there is already sufficient data collected that can be used to generate interesting simulations. That data is just locked up behind corporate walls.

Thoughts on District 9

2009-09-16T09:49:00.000-04:00

If you haven't seen District 9 yet, I highly recommend that you do, and do it soon. While some elements of the movie were a little outlandish (and there were a few technology loopholes that I had trouble reconciling with), the movie overall was very well crafted and engrossing. Beyond using this post to endorse a movie that I'm sure everyone will watch anyways, I wanted to share some random ponderings of my own.

1. The filmmakers did an excellent job of balancing the foreign elements of the aliens with enough recognizably human (i.e., anthropomorphic) aspects to exploit our human empathy. A lot of it is subtle, but we can take away enough from their body language, facial expressions, and desire to go home to identify their plight with other acts of oppression in human history.

2. I'm sure many other people have pondered and researched this issue before, but watching the movie makes me wonder what intelligent extra-terrestrial species would likely look like. The aliens in the movie seem like they were possibly adapted for a heavier gravity. Would we expect aliens to breathe oxygen? It seemed like the aliens in the movie had lungs (and some kind of vocal chords), but arthropods on Earth don't have these capabilities. In fact, I don't think a terrestrial arthropod that large could survive these days since they absorb oxygen through diffusion and the the oxygen density of our atmosphere is currently only about 21%.

3. The latter part of the movie reminded me of Paul Graham's essay on great hackers. The main idea in the essay is that inequality is not necessarily a bad thing. Technology amplifies people's skills and productivity, so we should expect that some people can do amazing things when given the proper tools... while other people do nothing but watch YouTube all day.

4. In retrospect, the whole Nigerian element was kind of ridiculous, but they were so well integrated with the texture and feel of the movie that my suspension of disbelief remained intact (until the very end when a series of events made things extremely ridiculous).

5. I like how the movie doesn't really have an isolated "boss villain". It fits with the general theme of how normal people making (relatively) normal decisions can lead to a situation where other people can be oppressed and knocked down by society. Sure, there was the colonel, the big wigs at MNU and the Nigerians, but I'm pretty sure lots of people would've wanted to capture the main character for fun and profit.

Rowling's Commencement Address Revisited

2009-09-02T11:45:00.019-04:00

Over the past year or so, I've become gradually more enamored by J. K. Rowling's 2008 Harvard commencement address, to the point where I now revisit it about once a month. There's a lot to like about her speech: the eloquent phrasing, a clear message, excellent delivery, and with nary a wasted sentence to be found.

In the last couple of years, I've also become increasingly interested in stripping away (or at least explicitly identifying and acknowledging) various [human] biases which cloud our reasoning. It recently occurred to me that a rationalist's interpretation of Rowling's commencement address might yield some interesting insights.

The first half of her address is relatively straightforward. Rowling discusses how experiencing real failure can help us reset our value system and acquire (or perhaps regain) a more informed and thus less biased perspective. Often in life, we allow various non-essentials to gain in significance and consequently warp our value system to detrimental effect. This happens especially often during our high school and college years, in part due to our limited experience at that stage. Many students live or die by their academic performance; the idea of "failing" at school can seem absolutely terrifying. And sadly, as Rowling mentioned, some people continue to equate their CV with their life well into adulthood.

Although Rowling pitches this lesson as one of self-discovery and finding inner peace, one can also consider the more general point of identifying and reducing the effects of institutional and cultural biases. I still remember the first time I did poorly in a class -- I cried and pouted for hours. After a while, I started realizing that school is but a conduit for gaining education and wisdom, and grades are but a proxy measure for quantifying performance. Of course, such things are correlated with our actual goals (or what would be our goals if we knew better). For example, grades are useful indicators of aptitude and can be utilized to help predict future success. But institutionalization and/or widespread cultural acceptance can often warp values to center too much on the proxies and not enough on what they strive to achieve.

Rowling's second message seems a bit more interesting. By asking us to be more empathetic, she is basically using one human bias against another. The detrimental bias is our "Monkeysphere". We humans have evolved to maintain relatively small inner social groups (see Dunbar's number). We might feel immense anguish for the death of a loved one, but barely bat an eye when told that millions of children die from malnutrition every year. By many objective standards, this is inexcusable -- as a side note, I encourage everyone to at least donate to organizations such as UNICEF (I donate regularly).

To overcome this bias, Rowling asks us to utilize our powers of imagination and empathy. By imagining ourselves in another person's shoes, we might start to understand their troubles and place value in solving them. Actually meeting such people in distress can be very eye-opening. The first time I visited my father's old home in semi-rural China was such an experience. I was struck by their impoverished state and, because of their poverty, how limited their ambitions were. Furthermore, I was keenly aware that my life could very well have been no different. For those who lack such experiences, one can try more scalable (though less effective) tactics such as watching Hotel Rwanda.

In many ways, our empathy is very biased towards humans. This point is a bit more abstract, but we do not directly experience what other people experience. For example, I can only consider how I personally would've felt growing up in poverty, because my feelings are the only ones I've ever known. More fundamentally, I only know for certain that I'm a sentient being with feelings and values. I can only infer sentience in other humans based on their observed similarities with myself, such as how their facial expressions resemble mine when I experience certain emotions. Currently, this has no practical implications, but it will become profoundly important if and when artificial intelligence systems become sufficiently general and autonomous. Most humans accept other humans as being sentient at face value (literally), but cannot imagine giving rights to "a machine" because "machines just can't possibly ever be self-aware -- or have a soul". While this empathy bias towards other humans can be powerfully leveraged for good causes in the world today, its implications might become more complex in the (possibly near) future.

Cat Sitting Adventures

2009-08-29T14:57:00.012-04:00

Ladies and gentlemen, today I wish to discuss a most dire task which was put before me, and how I, by the noblest and bravest of acts, did make right again a little place in this world -- a place we call Babbs' ass.

As far as cats go, Babbs is quite exceptional. With her unbelievably friendly and active demeanor, one might think she's but a young kitten still cutting her teeth -- or claws -- in this brave new world. When John and Robin both had to leave town for a couple weekends earlier this summer, I gladly agreed to look in on her.

Most cats maintain an air of aloofness around us lowly humans. If they greet us or even deign to acknowledge our existence, they do so only grudgingly. But Babbs is always friendly and curious.

But alas, not all is well with sweet little Babbs. Some years ago, before John and Robin rescued her from the humane society, a tragic accident took from Babbs that which she no longer holds dear:

But losing a tail is just the beginning of the story. Adding insult to injury, Babbs has also been deprived of any autonomous defecation capabilities. That's right folks, cat sitting implies cat pooping. At a high level, it sounded simple enough: wait for Babbs to enter the litter box, hold her firm and gently guide the wannabe droppings into actual droppings. It's analogous to squeezing out toothpaste -- except for the squirming feline container and the anti-cleansing properties of the secreted substance, of course.

The consequences of not relieving Babbs can be dire. Maintaining homeostasis in any organism is a delicate process, and congestion build up in Babbs' lower abdominal area can cause no end of trouble. Vomiting is the most common symptom, but prolonged congestion can lead to stool poisoning, necrosis, and possibly even death.

But surely, I thought, squeezing some droppings out of the little bugger shouldn't be too difficult a task. I mean, just look at those eyes. How could I let Babbs down?

Unfortunately, it turns out Babbs and I don't speak the same language. While I could infer fecal matter amassing via the swelling on her bottom, I could not effectively communicate to her that yes, Yes!, YES! we should go and poop together!

All cats are quite finicky, even ones as friendly as Babbs. No sooner did she feel me start to grab her rump would she jump away or, even more annoying, fall over on her side or back. I made the best of the situation by either taking some nice candid photos (see above) or tickling her belly, but thoughts of pooping lurked ever in my mind.

Seeing as how I had plans for the weekend and couldn't stick around for all of eternity, I had to leave Babbs be, unpooped and congested. Since I basically had no experience with this type of situation, I could only hope that she won't suffer too much ere I return at nightfall.

I went about my business for the day, though my productivity was somewhat hampered due to concerns over the poor feline. That evening, I returned to John & Robin's place with renewed vigor, determined to make amends for my earlier failures. Unfortunately, I was no more successful the second time around. Desperation got the better of me for a time, and I tried to force her hand (or her rump, I guess) by the force of Yisong. That endeavor lasted but a short while and shall be forever filed away as a shameful chapter of my life.

Faced with a dilemma, I decided to bite the bullet and stay over for the night. After all, the last thing I wanted was to explain to John and Robin how they came back to find a sick or, even worse, dying cat.

It was a dark and dry summer night (it doesn't really get stormy in the Bay Area). I hunkered down on the living room sofa across from the litter box, prepared to instantly step into action should there be any action worthy of my instantly stepping into. I was woken up a few times during the night, twice when I heard her rummaging around the litter box and once when she jumped on me, but my attempts to poop her remained in vain.

Sunrise. The Yisong awakens. The cat was still alive. Her rump was bulging. Time to get down to business. Photography business.

In all seriousness, I wish this story had a more climactic ending. But truth be told, she was ready to go when she was ready to go. Later that day, she waited patiently in the litter box while I gently, but firmly, turned her internal fecal matter into honest-to-God cat feces.

I stuck around and played with her for a little while. She loves to nuzzle, so whenever I place my face just above hers, she would almost invariably rise up on two legs to greet me. Sometimes, I played hard to get and took pictures instead.

I'm not sure whether there's a lesson to be learned here. But if there is one, it's probably that cats do what they want. And we love them for it, especially the sweet ones like Babbs.

Beautiful Mathematics

2009-08-21T01:42:00.012-04:00

Everyone has encountered the type: that nerdy and perhaps arrogant guy down the hall who rants and raves incessantly about the elegance, purity and sheer beauty of mathematics (think Hardy). While my position was never quite that extreme, I did often find myself being quite taken with just how wonderfully mathematics fits together.

As noted earlier, I no longer consider mathematics to have intrinsic beauty. However, I cannot help but admire certain results in mathematics due to their sheer elegance (as perceived by me) and profound profoundness (a somewhat more objective measure). And so, I thought it would be fun to list some of my favorite results. In some sense, one can think of this as the math analogue to Paul Graham's essay on heroes (although I've also written a more conventional version).

My basic test for inclusion is to ask myself, "would I use this as an example of how math can be awesome in every way possible?" The answer need not be a definitive yes, although they all happen to be for the five items listed below. OK, that's enough preambling, let's get on with the show!

Central limit theorem -- I would be hard pressed to name a result from formal mathematics which impacted society more profoundly than the central limit theorem. In a nutshell, the theorem states that everything looks like a bell curve given enough (independent) samples. As such we can use the same type of confidence intervals when making measurements in a wide range of practical settings. The central limit theorem is the foundational principle which underpins basically all of modern hypothesis testing. Science stands proudly on the shoulders of this giant.

Stokes' theorem -- I first encountered Stokes' theorem in a tensor analysis course during college. Having taken calculus for years and years, I found it mind boggling that so many cherished results I'd learned in high school were (sometimes trivial) special cases of Stokes' theorem. Green's theorem? Yeap, it's a corollary. Gauss' divergence theorem? Also a special case. The fundamental theorem of calculus? Trivial. It's not every day that you can distill about a third of basic and multivariate calculus into a single result. The day I found Stokes' theorem, now that was quite a day.

Lagrange multipliers -- the method of Langrange multipliers is perhaps the most well known technique for solving and analyzing constrained optimization problems. This has immense practical importance since basically every practical problem, if appropriately formalized, can be stated as a constrained optimization problem. For example, airline companies use optimization software to help manage their daily activities, which might otherwise turn into a logistical nightmare. While various special (and extremely useful) cases such as integer and non-linear programming do not directly use Lagrange multipliers, the influence is clearly present via analysis techniques such as Fenchel's duality theorem and the KKT optimality conditions.

Bayes' Theorem -- as an aspiring rationalist, I find Bayes' theorem to be one of the most useful tools at my disposal. Bayes' theorem basically shows how we should update our belief of the world given observational data and a proper formulation of our prior belief. As one might expect, it is also a fundamental design principle in modern machine learning and artificial intelligence, since it provides a formal framework for dealing with belief and rationality (which is very important when programming such concepts on a computer). Just because we humans often have trouble formalizing our intuitions and gut feelings doesn't mean it cannot be done.

Mathematical induction -- induction is arguably the most useful proof technique for analysis problems in computer science and other disciplines which deal primarily with discrete mathematics. The basic idea is to identify some invariant property that should be true, decompose the problem into smaller pieces, and show that having the invariant hold in the subproblems implies the same in the original. This iterative decomposition is effective for analyzing very complex problems starting from fairly simple base cases. Because the proof itself is typically prescriptive, it often comes hand-in-hand with an algorithmic solution. This makes induction very attractive since it can provide not only intuitive proofs of correctness, but also guidance for algorithm design (see dynamic programming and divide and conquer).

Most of my honorable mentions are very biased towards my field of study (machine learning) and the broader area of studying principled models with interesting computational AND representational aspects. Honorable mentions: Nash equilibria, Taylor's theorem, Gödel's incompleteness theorems, Arrow's impossibility theorem, de Finetti's theorem, Hammersley-Clifford theorem, Cook-Levin theorem, max-flow min-cut theorem. Euler's theorem, boosting and the strength of weak learnability.

Got school?

2009-08-20T03:32:00.010-04:00

"School sucks." -- Dan Urman, circa 5 hours ago (and perhaps for all of eternity)

In the eleven plus years that I've known Dan, he has time and again stated his distaste for the current educational system. In contrast, I've instead maintained a perhaps puzzling allegiance to academia. As our most recent discussion is still fresh in my mind, I thought I'd share my thoughts regarding some of the tidbits we touched on, as well as some random ponderings I've had since then.

Ideally, one attends school in order to explore a variety of different disciplines, make connections with professors and fellow students, and ultimately develop a marketable skill set and embark on becoming an informed citizen of society. Unfortunately, reality can be a rude bitch, especially for those who never really accept (or rather, put up with) the existing institutional inefficiencies and, one way or another, fall through the cracks. Why isn't there more incentive to reach out to these students?

If we consider schools as service providers and the students as clients, then we might ask whether we live in a buyer's or a seller's market, since that determines who has more leverage. For most "highly ranked" schools, it's almost certainly a seller's market since they are perceived as being high status and receive far more applicants than they have openings. As such, institutions of higher education can often be overly arrogant regarding their role in society. (Side note: Robin Hanson's take.)

For instance, it might seem quite paternalistic and condescending to force students to enroll in courses, complete homework, and jump through other hoops such as maintaining good attendance all in the name of education (I somewhat disagree). Nonetheless, schools do this all the time in some form or another. Why not implement more flexible criteria for measuring and certifying aptitude? Well, if we could afford it, we'd probably all hire the best personal or small group tutors available. But due to the aforementioned rude bitch, compromises must be made and large classes must be managed. Things certainly become easier if we can discover stronger and more diverse (i.e., less correlated) indicators of educational success.

Perhaps schools would have greater incentive to innovate and improve educational efficiency (as measured in the cost of the student's time and money versus benefit realized) if they did not own a monopoly over their incumbent students. Analogously, a big reason for government waste stems from the effective monopolies they hold over their citizens. The Seasteading Institute hopes to work towards more efficient governments by removing the monopoly aspect via floating cities (on water) with detachable components that are free to join with other cities. Suppose it were much easier for students to transfer between schools. If that were the case, then the market might shift to favor the buyers (the students), which would in turn spur competition and innovation between the different educational institutions (Patri Friedman's take). Sometimes, that's all you need.

Photography Update

2009-08-11T08:32:00.006-04:00

I decided to take up photography to explore, for lack of better phrasing, ways of capturing moments in time. Now, half a year later, some noticeable progress has been made.

Due to a number of reasons, I've found it much easier to take photos which contain human subject matter. Obviously, humans are very dynamic and active, and thus create more original photo opportunities. I've also found it easier to elicit emotional responses with candids and portraits and the like.

In fact, some of my earliest accidental portraits are still among my favorites (perhaps with a little boost from nostalgia):

Around the midway point between then and now, spring arrived, and with that comes many a picnicking and graduating. Since I prefer to shoot without flash (and since I don't even own a decent flash for my camera), the bountiful sunlight available during outdoors shooting was a welcome change. And, as one might expect, I took advantage of the moments which presented themselves with some timely and deft camera wielding:

Nowadays, I try to make my own destiny by either pseudo-posing with other people's pets, or just plain photo stalking:

My efforts with landscape photography have been met with mixed results. I've found that they usually require more attention to composition (and a tripod, which I still don't have), since the subject matter is typically less intrinsically interesting to us humans. Hopefully, I'll find more success in the future. One of the advantages of using flickr is its social networking aspects. It can help you find some truly breathtaking photos.

Let's all be Elves!

2009-08-08T03:01:00.013-04:00

...Lord of the Rings style Elves, that is. What's so special about them Elves? Well, they live forever, of course! To be more specific, they are immune to the effects of aging and disease, but can be killed via more direct means.

The SENS movement views the human body as essentially a machine (albeit a very complicated one). Under this view, we can, at least in principle, develop techniques to repair all age-related damages that our bodies accumulate over the years. As we diagnose and solve more exotic forms of cellular damage and diseases (which might be currently irrelevant since we still die from more obvious causes), we might perhaps actually achieve longevity escape velocity. You can read an excerpt from Aubrey de Grey's book here (or if I like you enough, you can borrow my autographed copy).

For those who are curious, here are a bunch of recent articles showing a very limited snapshot of our current progress. It's certainly a very exciting time to be alive (although being born later would be even better).

So let's assume we're amongst the lucky ones who can live, if not indefinitely, then at least for a very long time. What does that imply? Well, for one, the Elves in LOTR rarely produced offspring. While many would consider this tragic, I expect almost everyone (at least in the developed world) to agree that having around two kids is preferable to having ten kids. The arguments in favor of two over ten tend to revolve around quality of life (both for the parents and for the potential children). Once our standard of living increases dramatically, one can make the same arguments for preferring to have almost zero children over having two.

But fear not, for I expect we'll be having a decent number of kids for many years to come. This leads to another interesting situation. In LOTR, Elves spend most of their lives in healthy adulthood. As such, parents and children are more or less peers (trivia: Galadriel is Elrond's mother-in-law and also his first cousin thrice removed). We see a hint of this effect already in our society; these days, parents and children might jointly enjoy 10-20 years of healthy adulthood together. So I expect this dimension of society to make a relatively smooth transition.

A particularly interesting issue is power (and career paths). In the LOTR, all the elder Elves stay in power until they are killed. Tolkien clearly wasn't trying to predict future social norms when creating his mythological world and its long-lived Elvish peoples, but it's interesting (at least to this Tolkien fan) to use as a point of contrast. A static chain of command doesn't sound very appealing. On the flip side, it seems unlikely that we could all retire (until we die) unless our productivity skyrockets at nigh-unbelievable rates, so some other solution is probably required. The most obvious one is for people to choose a line of work for a few years, take a few years off, and then rinse and repeat. Sounds pretty enticing, eh?

I hate to end this post on a downer, but the last issue we'll be covering is inequality. In the LOTR, many Elves looked down on mortal Men, whose lives are but a flicker compared to the eternal flame. On our side, not everyone has enjoyed the fruits of capitalism and its remarkable ability to spur innovation and progress. How can we bridge this gap? Despite the various disparities found within the developed world, social mobility is very possible for people with sufficient skill and dedication. But such opportunities rarely present themselves for those stuck in the bottom billion. Will we also live in a world with second class citizens? Will privilege of birth determine whether one might live forty or a thousand years? The wisest Elves looked beyond the frailties which plagued Mortal Men and formed bonds of friendship for the benefit of both races (e.g., the Half-Elven). Perhaps we can do the same.

SIGIR 2009 Recap

2009-07-29T02:31:00.019-04:00

I've finally gotten a chance to rest up a little after an exhausting week at SIGIR 2009. Compliments to James Allan and Javed Aslam for a very well-run conference. For those who don't know, Professor Allen obtained his Ph.D. from Cornell under Gerry Salton. Cornell requires all Ph.D. students to also have a graduate minor. Back in the day, Professor Allen chose theater as his minor under the supervision of David Feldshuh, who also happened to direct me in Cornell's production of Bernstein's Mass (I was just a backup singer in the liturgical choir). As you might imagine, James and I had an interesting conversation when I visited UMass Amherst a few months ago.

Moving back on topic, congratulations are also in order for Sue Dumais, who became the newest recipient of the Salton Award. For her acceptance talk, she presented a very nice retrospective on the field, although a couple of her desktop search demos were a tad on the slow side (courtesy of Vista).

I attended a very interesting tutorial by Rosie Jones and Vik Singh. Vik is the architect behind the design and development of Yahoo!'s BOSS (Build your Own Search Service) open web search platform. BOSS makes it very easy for regular folks like you and me to quickly develop our own search service on top of Yahoo!'s search infrastructure. For example, Vik wrote tweetnews, which re-ranks standard news results using Twitter feeds to provide more relevant (or at least more interesting) fresh news articles. Thanks to BOSS, the core logic required only about 10 lines of code.

The main conference itself was a gigantic blur, since I spent my copious free time preparing for the two talks I was giving on the workshop day (which thankfully is always the last day). Nonetheless, I did manage to attend a few interesting paper talks, such as Paul Bennett's on large-scale taxonomy classification and Pinar Donmez's on the local optimality of LambdaRank. I probably also tacked on some extra weight considering how little attention I paid towards my diet and exercise. The quality of the Boston area seafood (as well as a few nice gestures from various tech companies) didn't help either.

Taking a page from the human computation book, Microsoft has released a new labeling game called Page Hunt (Silverlight required). The folks manning the Microsoft table at SIGIR insisted that I should play and gave me a ridiculously oversized Bing t-shirt as a token of appreciation. Congratulations Microsoft, I now use your shirt to sleep in; the Google shirt I was using has been callously cast aside.

Given my impending talk at the diversity workshop, I was particularly interested in any papers which even remotely discussed promoting diversity in search results. Current practical search services score the relevance of each web result (with respect to a query) independently of other results. As such, the top two results could both be highly relevant but (nearly) completely redundant with each other. We currently aren't even sure how to accurately measure diversity, not to mention design models and algorithms for appropriately diversifying search results. Unfortunately, I didn't find any of the conference papers very satisfying, but the workshop stirred some very interesting discussion (as is typically the case). For those not familiar with the terminology, conferences serve as the primary computer science publication venues, and are thus more prestigious. On the other hand, workshops (which often come attached to a conference) are typically used to present more preliminary results and/or promote discussion for potential research directions.

You might be wondering about the above photo. Let's just say that someone lost a weight loss bet to someone else, and thus had to wear a tutu to the conference banquet. Luckily, he did lose enough weight to actually fit into the tutu (would've been pretty catastrophic otherwise). As a side note, the ensuing online discussion caused Gmail to start displaying tutu web ads. Thank you contextual advertising.

(As a final aside, I think I make too liberal use of parenthetical elements.)

Pondering Intelligence

2009-07-20T17:30:00.012-04:00

What is intelligence? It's a question that can be approached from numerous angles. Many (most?) people treat intelligence as a uniquely human property, one that is intricately bound to notions of tremendous intrinsic value, such as having souls, unalienable rights, self-awareness, etc. Chimps might be pretty smart (for a non-human), but they're still just chimps.

The AI community, having recovered from the failures of the 1960s and the ensuing AI winter, has now developed many techniques which have greatly improved (and in some cases enabled) a wide range of applications. Currently, our most successful AI systems use very narrow AI (e.g., chess, web search, spam filters, fraud detection, the Mars rovers), but that holy grail of general AI (AGI) still lurks in our thoughts.

Most discussions I have regarding AGI are met with stiff resistance. One of strongest biases I've noticed is a very homo-centric view that only humans can be truly intelligent or sentient. Moreover, it seems many believe, at some fundamental level, certain properties -- our sentience, for instance -- extend only to other beings who are like us. After all, I only experience my own thoughts. As such, I don't know for certain if you are sentient; I can only infer based on your behavior and my own mental model of how sentient beings (aka, myself) should behave.

So what does that imply? Well, for one, we accept other humans as being sentient, intelligent beings despite being given a very little evidence. In many cases, we're simply projecting (or empathizing, if you prefer). Does that also imply that, at least in principle, we can devise a comprehensive suite of tests or measurements to characterize the general notions sentience or intelligence?

Currently, I'm inclined to consider intelligence in very practical terms. Suppose, for the sake of argument, that there does indeed exist a suite of tests to measure all levels of intelligence (at least up to human level). It's clear that our AI systems are becoming more general every year (although they're still a far cry from human-level intelligence). However, if we dare imagine that the exponential growth curves achieved by many other currently pervasive technologies (e.g., the internet, mobile) might also apply to AI development, then we might quickly see systems of far greater complexity and generality. Some more realistic milestones include a general secretary program that can learn to perform personalized appointment/travel management tasks, and a general household robot that can learn to perform various chores for different home and lifestyle configurations.

One of the greatest conceptual difficulties when contemplating intelligence arises from the fact that no other extant beings (natural or artificial) even comes close to rivaling human intelligence. The situation might be much more intriguing had many of our pre-modern cousins and ancestors survived to present day. I think it's reasonable to assume that different hominid species would display varying degrees of intelligence. Given that type of evidence, it might be more reasonable to believe that AI could one day also climb up the general intelligence ladder.

Summer Snapshot 2009

2009-07-11T21:26:00.009-04:00

Now that I've been hanging around the Bay Area for a while, I thought I'd share some adventure stories and other tall tales.

I'm having a great time interning at Google. I've been very impressed with their amazing infrastructure, which in turn gives them the agility to quickly try out new ideas. The amount of data I've handled so far this summer is already greater than the total amount of data I've ever dealt with in the past.

In my spare time, I've been trying to balance my other priorities, such as conferences, other research activities tied to my thesis, staying healthy, and, of course, spending time with friends and family. In fact a whole host of family members paid a visit a couple weekends ago. We made hotel reservations at Fisherman's Wharf and spent the weekend seeing San Francisco, taking a cruise, and hiking in Muir Woods.

Here we see my cousin Cecillia with her mom and husband Afrim preparing to dine at the Crab House at Fisherman's Wharf. She was very excited.

I was noticeably less excited. Luckily, my mom helped pick up the slack on this side of the table.

We took to the water with a cruise around the bay. It was quite windy.

The water was very cold (which is very typical in NorCal). Nonetheless, one can still find a few brave (or stupid) souls every now and then who decide to take a tumble into the bay. This here might be the best candid shot I've ever taken.

After exploring the city a bit (including lunch in Chinatown), we meandered over to Golden Gate Park and eventually the beach.

Photography in action.

The next day, we took a trip north of the bay to Muir Woods.

Steve, Eric and John came as well.

This shot made me feel a little bit like we were in the Lord of the Rings.

Eric being Eric.

Eric: my hands, they have a mind of their own!
Steve: I'm not impressed.

The following weekend (which was 4th of July weekend), Ryan took a break from his start-up in Seattle to visit. We settled down next to the Shoreline Ampitheatre to watch some fireworks. It can get quite chilly at night in the Bay Area. As such, it's usually wise to bring along a sweatshirt or jacket. Here we see Ryan performing the requisite maneuvers.

None of my fireworks shots came out well since I didn't have a tripod with me. So I'll just conclude by showing some pictures where I flashed other people, such as Todd

Todd again.

Ryan.

I've also uploaded a few photos to my Flickr photostream for your viewing pleasure. This one is my favorite so far from this summer.

Thoughts on ICML/UAI/COLT 2009

2009-06-25T01:17:00.003-04:00

Last week, McGill University (up in Montreal) hosted the co-located ICML/UAI/COLT conferences. With over 700 participants, this was the largest gathering of machine learning researchers we'll see this year until NIPS.

First off, I'd like to congratulate my advisor, Thorsten Joachims, for winning the Best 10-Year Paper Award at ICML, which is given to a "paper published in ICML 1999 the committee feels has had the most significant and lasting impact". And not only is he a brilliant researcher, but he's also an unbelievably nice person.

As one might expect, the conferences featured a number of invited speakers. The three invited talks I enjoyed the most were given by Yoav Freund, Adam Kalai, and Yoshua Bengio.

Yoav presented a new robust boosting method he'd been working on. Boosting is one of the most successful machine learning approaches ever proposed. Those who are interested can try out his updated software package.

Adam presented new results in agnostic learning as well as shared some thoughts on interactive learning (a topic which interests me greatly). One interesting direction Adam mentioned is to somehow inject learning into human interaction protocols such as those developed by Luis von Ahn. In Thorsten's retrospective talk for his 10-Year award, he also touched on the benefits of learning from human-computer interactions. The interface between machine learning and the real world is still quite narrow. Developing an understanding of interactive learning can definitely help in this regard.

Yoshua's talk provided a nice overview of recent developments with deep learning. Deep learning distinguishes itself from (most of) the rest of machine learning by attempting to learn many layers of feature abstractions. In particular, Yoshua stressed this as being essential for jointly learning over many different tasks (i.e., transfer learning). Perhaps the greatest strength of human intelligence is our ability to reason at many levels of abstraction and thus generalize to new unseen tasks. Transfer learning is an attempt to push machine learning algorithms towards that goal.

John Langford has already highlighted a number of interesting papers from the conferences, so I'll just conclude by commenting on a few he didn't mention.

Deep Transfer via Second-Order Markov Logic by Jesse Davis and Pedro Domingos. What excites me about this paper is the fact that the authors succeeded (to some extent) at applying transfer learning to seemingly unrelated domains with different feature representations. For example, reasoning about the higher level semantic structure of protein interactions in yeast might help us do the same with the social network structure of Facebook. After all, both can be represented abstractly as graphs where edges indicate some type of interaction between the nodes. Studying networks which exhibit similar emergent properties is an ongoing and active area of research (cf. Jon Kleinberg's work). It's nice to see that we're now able to leverage these properties to improve learning and prediction across domains that are difficult to integrate via feature engineering.

Learning From Measurements in Exponential Families by Percy Liang, Michael Jordan, and Dan Klein. This paper presents a unified Bayesian framework for reasoning about different types of measurements that one can extract (e.g., labels versus label proportions). I wonder if we can see further improvements by adopting a more discriminative approach, and also whether aspects of this framework can be leveraged for real-world active learning problems such as maximizing the utilization of human judges for some labeling task. Services such as the Mechanical Turk open the door for a tremendous amount of creativity in how we acquire labels (or more generically, measurements) for training data. It would be nice to have principled ways to reason about the trade off between labeling difficulty and information gain (e.g., tracing a perfect outline of a person in an image versus drawing a bounding box).

Metric Transform Question

2009-06-24T05:11:00.003-04:00

Let d(x,y) be a metric on some space X, and let w(x) be a non-negative weight function defined on X. Define a new metric d2(x,y) on X such that d2(x,y) is monotonically increasing in d(x,y) and monotonically decreasing in w(x) and w(y).

This is a relatively open ended question. You may make further assumptions if it improves the simplicity, elegance, or usefulness (e.g., computational efficiency) of your solution.

Learning Interactively from Partial Relative Feedback

2009-06-09T23:34:00.011-04:00

In recent years, much of computer science research has focused on developing algorithms which operate in the presence of uncertainty. Indeed, from a low-level technical standpoint, statistical machine learning is essentially about designing models which have strong predictive power as well as finding efficient algorithms for processing such models.

One such area is the so called multi-armed bandit problem, which is particularly appropriate for modeling many real-world problems where a learning must be done on-line (paying higher penalty for exploring worse strategies) and feedback is partial (does not know the outcome of strategies not explored). Example settings include clinical trials, advertising, adaptive routing over networks, and basically any scenario where one can only try out a limited number of strategies at a time.

A critical modeling choice used in conventional bandit formulations is to assume the availability of absolute, or cardinal, feedback. In many (and I believe most) settings, such feedback is either unavailable or difficult to obtain. For example, when conducting a blind taste test, rather than asking people to rate the quality of two food items separately on an absolute scale, one typically would ask participants which product they prefer in a side by side relative comparison. Generally speaking, I believe it is often much easier to obtain relative, or ordinal, feedback rather than absolute feedback.

In collaboration with Thorsten Joachims, Josef Broder and Bobby Kleinberg, we've been working a bandit formulation -- called the dueling bandits problem -- where feedback is relative rather than absolute. So rather than, as in conventional bandit settings, trying out one strategy and seeing how it works out (which assumes absolute feedback), we propose instead to compare a pair of strategies and obtain a (possibly noisy) observation regarding which strategy is superior (which thus assumes relative feedback). We have recently obtained some results for both the discrete and continuous cases. The two papers will be presented later this month at COLT and ICML, respectively.

Our motivating application is the problem of learning retrieval models for search services. Search users leave behind an abundant amount of implicit feedback, most notably the results they click on. However, simply counting clicks (i.e., using clicks as absolute feedback) will lead to very biased results. A recent paper by Filip Radlinski, Madhu Kurup and Thorsten Joachims proposes a new way of comparing two different retrieval functions in an interleaving experiment that is minimally obstructive to the end users, and also allows clicks to reliably indicate which retrieval function is superior (i.e., relative feedback). This development in turn motivated our work on the dueling bandits problem. The interleaving experiment can then serve as an implementation of the comparison oracle required for obtaining relative feedback.

There is still a lot of work left to do, including how to make the methods more generally practical as well as developing more sophisticated and realistic formulations. It seems to me that a necessary (but not sufficient) condition for general AI is the ability to intelligently (and possibly interactively) explore an unknown environment. While the application setting I'm focusing on is painfully shallow compared to that lofty goal, I'm nonetheless pleased by this direction of research.

Jon Kleinberg Interviewed

2009-05-28T23:15:00.004-04:00

As the newest recipient of the ACM-Infosys Foundation Award, Jon Kleinberg was recently interviewed by the ACM. The sound quality is a little spotty, but the 45 minute interview touches on many interesting aspects of his research and teaching interests, which center around the analysis of large scale information networks and more generally the interplay between computational and social aspects of modern systems. Some selected excerpts are also available for those who do not wish to listen through the whole interview.

I had the great pleasure and privilege of being a teaching assistant for the inaugural Networks course taught jointly by Jon and David Easley (most recent semester). I also got to witness first-hand the tremendous popularity that the Rebel King garners amongst Cornell undergraduates. Although the course is not required for any major, the first day of class saw standing room only in a large lecture hall (with well over 200 students in attendance). It's not hard to see why that's case; the material covered is interesting, intuitively presented, surprisingly deep for an introductory course, and very novel to most of the students. I highly recommend the course to anyone with even a passing interest on the topic.

Paint Job

2009-05-26T20:50:00.004-04:00

I'm spending a few days with the family in Phoenix before heading out for my internship at Google. My parents recently moved into a new house and wanted my sister and me to do a bit of touch up work on our old home. In particular, we needed to paint over a few stains on the walls and ceilings. In the fairest division of labor possible, Lena got to do the painting while I provided the moral support.

Mixing the paint.

Thanks for helping, Yisong.

Prepping the paint roller.

Almost ready to paint.

Here we go...

... and still going...

... and finally done.

Phew, how's my hair?

Fun Math Puzzles

2009-05-15T10:30:00.003-04:00

Following a day of fun in NYC (which included seeing KOOZA and visiting Yahoo! New York), I spent much of the five hour drive back to Ithaca exchanging fun math puzzles with Ymir. And so, what else could I do but share them here? Some of these are harder than others.

Problem 1. There are N prisoners (N even) and a sadistic warden. The warden places N boxes in a room, all identical except that each is labeled with a unique number from 1 to N. Each box contains exactly one of the prisoner's names (assume distinct names) and each prisoner's name is in one of the boxes (a bijection). When the game starts, each prisoner is allowed to open half of the boxes and observe the names. One cannot tell from observing the room which boxes had been opened previously. Afterward opening half the boxes, without communicating with any other prisoners, each prisoner must guess which box (i.e., which number) contains their name. If all the prisoners guess correctly, then they win and are set free. Devise a strategy such that the probability that all the prisoners win is bounded below away from zero (as N grows), or prove that one does not exist.

Problem 2. Suppose we have an NxN checkerboard, and suppose that two diagonally opposite corner tiles have been removed. Devise a way to exactly tile the remainder using 2x1 domino pieces, or prove that it is impossible to do so.

Problem 3. Suppose you are placing N lemmings on a flat and finite one dimensional surface (i.e., the lemmings can fall off the edges). All lemmings behave the same: they move at a fixed speed in their direction of travel (since this is a one dimensional surface, there are only two possible directions). If two lemmings bump into each other, then they both reverse their direction of travel. There are no other obstacles on this surface. You are allowed to control the initial position and direction of travel for each lemming. Devise a strategy that maximizes the total time that lemmings spend on the surface (i.e., the sum of the time that each lemming spends on the surface before falling off).

Problem 4. Suppose you have two strings of different length and an unlimited number of matches (as in, for burning). Each string burns in exactly one hour, but the burn rate of each string is non-uniform and unknown in advance. For example, one string might burn through 99% of its length in 1 minute and the remaining 1% in 59 minutes. Devise a way of counting 45 minutes using the two strings and the matches.

Number Theory Problem

2009-04-30T10:20:00.004-04:00

Let Q_n denote the smallest number such that for any set of Q_n integers, there exists a subset of n integers that sum to zero mod n (i.e., a multiple of n). For example, for n = 3, any set of Q₃ integers contains a subset of size exactly 3 that sum to a multiple of 3, and Q₃ is the smallest such number.

What are Q₃ and Q₉?

(Hint: proof of sufficiency uses the pigeon-hole principle and proof of minimality uses a constructed counter-example.)

Summer Plans

2009-04-21T13:40:00.005-04:00

With the school year winding down and summer quickly approaching, I bet you're all wondering where little old Yisong will be come June. For those in the Bay Area, get ready for some Yisong goodness. I'll be spending this summer interning at Google and generally having a good time in Mountain View and the likes. Everyone who won't be in NorCal this summer will be missing out.

My official title is "Decision Support Engineering Analyst Intern" but you can basically think of my job as helping Google design smarter search services. I was very fortunate to make contact with a team there that's working on projects closely related to my current research (and supported by Google's amazing infrastructure). I'd already turned down Google once before in favor of some other tech company, so I don't think I can pass up the opportunity again. I'm looking forward to a very exciting and productive summer.

In other news, some of my recent research has been accepted for publication at ICML 2009, so I'll be traveling to Quebec in June to present it. I'll also be making a trip to Boston in July to attend SIGIR 2009, along with a possible brief detour to Chicago to see a certain Silk Road production. I love summers, as they're always so full of excitement.

Bernstein's Mass

2009-04-20T10:55:00.007-04:00

Ever wonder how my evenings are spent way up yonder in isolated Ithaca? This week, I'll be at the Schwartz Center doing my bit for Cornell's production of Bernstein's Mass. As part of the Cornell Glee Club, I'll be singing in the background liturgical choir (which basically means I'm a backup singer).

Bernstein's Mass is one of the most ambitious productions in existence, mostly due to it sheer size. Luckily, Cornell has the luxury of giving out course credit (as well as letters of recommendation) instead of monetary compensation to student performers (who comprise the majority of the cast).

Having rehearsed for a few weeks now (including tech rehearsal every night last week), I can safely say that it should be a great show. Our lead actor, Dominic Inferrera, is quite amazing -- he has that rich baritone voice that every young choir boy dreams about having. So if you happen to be in town this week, unlikely as that is, be sure to catch one of the shows. We'll be performing every night from Tuesday (tomorrow) through Sunday with an additional matinee performance on Saturday (details here).

Cornell CS Foosball World Cup

2009-04-19T22:37:00.003-04:00

The computer science department recently received a foosball table courtesy of Yahoo!. And so, what else could we do but host our very own foosball world cup tournament?

Due to limited representation from many countries, some creativity was required in order to construct a good-sized bracket. Many teams had to enlist expatriates from other countries in order to form a full team (of 2). I ended up being recruited by the Slovenian team, and scrubbed out pretty quickly. Overall, the tournament was very entertaining and had plenty of highlights, even though it was never in doubt who the champions would be.

Here we see either Team Britain or Bulgaria (there's one member from each country) versus one of the American teams (there were three).

Good game.

The agony of defeat.

The action got pretty heated at times.

Even some professors got in on the action. Here we see Andrew Myers and Steven An go up against Team Vietnam (the eventual champions).

hehe...

2009-04-08T18:45:00.003-04:00

(from Facebook Pages We'd Like to See)

Best thank you card ever?

2009-04-01T18:11:00.001-04:00

What do you think?

Research in the Engineering Sciences (Machine Learning)

2009-03-31T15:58:00.010-04:00

Research in the engineering sciences differs from more established fields in that it's focused on the methodology rather than the search for the truth (in the natural sciences), or the TRUTH (in pure mathematics). As such, it's more empirically driven than research in mathematics, since (almost) no one cares about methods that don't work in practice, but a model need not be perfect or even understandable in order to be useful and very widely applicable. One can think of engineering research as primarily concerned with analyzing trade offs with respect to things like computational efficiency, accuracy, robustness, etc.

Since the computer science field is relatively young and experiments are usually done completely on computers, most research projects produce publishable results fairly quickly (usually within 3-9 months). This has (at least) two consequences. First, the ease of technology transfer from research to consumer software makes it very attractive for companies to establish flexible collaborations with academics. Second, the condensed publication cycle makes computer science a conference driven field (as opposed to journal driven).

Being a conference driven field with strong industry participation has its advantages and disadvantages, though I think the advantages strongly outweigh the disadvantages for engineering fields like computer science. First, this means that Ph.D. students will typically have several publications by the time they graduate, thus affording them many chances to attend conferences, give talks, share ideas, and network with other researchers. For a field that is rapidly growing, this offers grad students a great opportunity to distinguish themselves (by having clear ownership over their research projects) and thus facilitating finding productive collaborations with other researchers.

My particular area, machine learning, intersects strongly with statistics. In fact, many statistics students compete for machine learning positions in industry. However, given that statistics is mathematics field and has a much longer publication cycle with different criterion for judging research quality, it can be more difficult for a statistics student to land a good position in industry (c.f. the statistics handicap). The good news, as stated before, is that the field is growing and thus requires more people.

But not all is roses. Computer science, and machine learning in particular, suffers from having an extremely long tail in terms of publication quality. This year alone, the major machine learning conferences will publish around a thousand papers, and this is not even counting all the workshops or papers in application areas that heavily use machine learning such as computer vision, natural language processing, computational biology and information retrieval. As you might imagine, reviewing so many paper submissions can be quite a chore, and most published papers are rarely heard from again.

I think having such a long tail lowers the barrier of entry for machine learning research (which is good), but it can also degrade the scientific rigor of the field (which is bad). Many researchers effectively use their papers to convey chapters of their ongoing research story. This, of course, is completely reasonable. The problem arises when individual papers convey very little novel content. One interesting solution would be to incorporate blogs as an accepted component of the research culture. Who knows, maybe I'll start my own research blog some day.

At any rate, despite its imperfections, machine learning is a booming research area with many exciting problems to tackle. The style of research in the engineering sciences is dramatically different from that in the natural sciences or in mathematics. Publishable results are generally easier to come by and, given the technology infrastructure in place, typically have quick turn-around times in terms of making an impact. The condensed publication cycle also keeps things fresh, since I find myself working on at least two new projects every year (although they're usually related to my previous projects) . Of course, it's not for everyone. You have to like working with computers and be sufficiently motivated (or at least satisfied) by the types of impact progress in this field will have. But it sure is fun!

Lame Math Question

2009-03-25T01:49:00.000-04:00

Evaluate the following infinite sum:

1/1 + 1/4 + 1/9 + 1/16 + 1/25 + 1/36 + ...

Math Professor Personalities

2009-03-23T12:53:00.003-04:00

We've all heard stories about nutty math professors. You know, the ones who have their heads lost in the clouds of higher mathematics. For them, math is beautiful and proofs come naturally (i.e., easily). In fact, they generally think that it's sufficient to merely state the theorem. Why bother with the proof?

We've also heard stories about snooty math professors. For them, conquering mathematics is a rite of passage. Math is hard and they are the shit(s) for being masters at it. They chortle with contempt whenever any of their pathetic students dares to ask a (stupid) question. Why bother trying to explain proofs in class? They won't get it anyways.

And then of course, there are the bipolar math professors. They like to dabble in both...

My First Poster

2009-03-17T09:59:00.002-04:00

Check it out!

Don't worry dad, I'm not going to quit my day job.

An Open Mind

2009-03-03T22:28:00.006-05:00

Here are some quotes (to the best that I can remember them) that I've encountered in recent years:

"How can anyone do better than Google?"

"A black person won't become president of the United States within my lifetime."

"Without Albert Einstein, it would've taken another 30 years to develop general relativity."

"Without Microsoft Internet Explorer, the World Wide Web would've been set back for years."

"I don't believe we will ever create true artificial intelligence."

"The human body is so complicated, I don't see how we can dramatically extend human lifespans in the foreseeable future."

The first quote is a question that a lot of smart people have good answers for, the second is now proven to be false, and I strongly disagree (in some way) with the remainder. The third and fourth quotes deal with alternative history, while the last two express opinions about the future.

The common theme, as I see it, is a combination of pessimism and close mindedness. The two are related but not necessarily correlated. For example, one can be extremely imaginative about all the possible doomsday scenarios that might come to pass. I won't discuss first two quotes in detail (except to say that they were opinions expressed by well educated professionals) and will instead deal with the last four.

I'm not very knowledgeable about general relativity, so I won't comment on it much. But Einstein clearly collaborated with and built off the work of many others including Lorentz, Hilbert, and Riemann. But we humans tend to funnel the joint accomplishments of many people into just a few (or in this case one person). Yes, Einstein was a genius. But I highly doubt that he was smarter than the collective intelligence of all the other physicists and mathematicians working on the problem at that time. We tend to deify Einstein for the accomplishments of that era without sufficiently acknowledging all the hard work that everyone else did.

The Internet Explorer opinion seems impossibly difficult to defend. IE wasn't even the first commercial web browser. That honor falls to the Netscape Navigator. The idea that the World Wide Web would not charge full steam ahead without one software product seems a little absurd. It was already clear at that time that the internet was going to be a huge deal. Where there is a will, there is a way. Someone was going to develop a dominant browser, and it just so happened that Microsoft did it (some might argue that IE's dominance was more the result of Microsoft's market leverage rather than good design, but let's not split hairs).

In essence, we get so locked into what happened, and how monumental the end result was, that we fail to put the full process into a proper perspective. This has two effects. First, it makes us think that the world's fate (or at least various aspects of it) balances on a knife's edge -- that our average quality of life might have been much worse had a few events happened differently (this is not necessarily in conflict with ideas like the butterfly effect or chaos theory). Second, it makes us more pessimistic about the future. If we only focus on the final results and not the incremental journey we took to arrive at those results, then it might seem impossibly hard for us to make more breakthroughs in the future.

It is true that I am overly optimistic compared your average Joe. However, I would argue this is mostly due to all them Joe's not fully reflecting on our technological and scientific prowess, where we are, and how we got here. Rather than addressing the last two quotes directly (which would require a lot of speculation), I'll simply remind everyone of some recent breakthroughs.

Suppose you had asked someone in 1985 whether, in 20 years, the developed world would be connected via a global information network that transforms almost every aspect of our lives, that person will probably think you're either nuts or spent way too much time watching bad sci-fi movies. And yet, that is exactly what has happened.

Within the same time period, we have now developed a variety of methods to efficiently map the human genome, as well as perform some fairly complex gene manipulations. Again, most people back in 1985 would probably think you're nuts for imagining so much progress in such a short time.

As we continue to gather more knowledge, conceptual shifts are often required to more effectively model and understand the world around us. Unfortunately, that makes it very difficult to foresee how we should approach currently unsolved problems from a technical standpoint (if we did, then it wouldn't require research). But such transformational shifts in thinking have happened in the past (sometimes gradually and other times violently), and they will continue to occur in the future.

So I want you to ask yourself a couple simple questions. First, how long do you think it will take us to develop general purpose programs that can interactively learn and adapt to heterogeneous environments (e.g., an off-the-shelf robot that can learn to perform all sorts of different house-hold tasks)? Second, how long do you think it will take before medical advances allow us to healthily live to 120 years? Would you answer 25 years? 100 years? 500 years?

I expect most people would give a relatively small answer (perhaps in the 20-30 year range). Keep in mind that, as new technology areas mature and enable new businesses, the rate of progress often experiences a huge explosion. That's what happened with automobiles, airlines, computers, cell phones, the internet... and basically every cool new technology in the last 200 years. So it's what I think will happen with smart programs (and machines) and life extension technologies.

I'll conclude with a couple of my favorite quotes:

"The best scientist is open to experience and begins with romance - the idea that anything is possible."
-- Ray Bradbury

"If you can dream it, you can do it."
-- Walt Disney

A Good Life

2009-02-23T23:37:00.009-05:00

Life is what you make of it. So says my sagely friend Dan Urman.

So what, then, would make a good life for you? By this question, I don't mean simply your current living situation -- I'm asking about your entire life, a hypothetical best case scenario that is still realistic.

It's very difficult for me to answer this question (although the difficulty does seem to vary depending on how "realistic" is interpreted). I can think of two reasons for this. The first is pretty straightforward: I implicitly operate under the assumption of free will and non-determinism. Any attempt to define my entire life trajectory would undermine my inherent sense of how life should be lived. In other words, I prefer to view my life as an ongoing journey, where the road ahead is to some degree cloaked in mystery and thus still uncertain.

This first issue can be partially addressed by being less detailed or precise when describing a hypothetical good life. But even if it is desirable to employ such a tactic to answer this question, the second concern remains.

To the extent that it is possible, I am fundamentally an optimist. So if you propose to me a hypothetical scenario, then I will consider ways to improve upon said scenario. As such, any instance of a (realistic) hypothetical life will ultimately be unsatisfying to me, given sufficient time for contemplation.

For example, while I don't expect to live forever, any pre-specified lifespan is completely unsatisfying due to notions such as longevity escape velocity. One can easily relate this idea to some simple mathematics. Consider, for example, a positive random variable X that follows a folded Cauchy Distribution (a folded version F' of F basically means P(0 < F' < K) = P(|F| < K)). The probability of X taking on larger and larger values shrinks to 0, but its expected value is infinite. This is basically following the idea that the function f(x)=1/(1+x) vanishes as x increases, but its integral over the positive reals is unbounded.

Another way to look at this issue is by considering the two envelopes problem. Conditioned upon knowing a realistic scenario for what might be, I can always imagine something better. Not surprisingly, the solution proposed by Chalmers uses a distribution that has an infinite expected value.

But maybe I'm just weird and severely over-thinking the situation (happens pretty often). Maybe this question is relatively easy for other people to answer.

Guess who?

2009-02-16T22:04:00.002-05:00

Hint: his face looks much bigger in full lighting.

Random Walk Revisited

2009-02-12T23:17:00.009-05:00

(revisited from the post Random Walk Puzzles)

Consider random walks on a graph with N+1 nodes (numbered 0,1,2...,N) where node j transitions to a lower numbered node i (0 <= i < j) with equal probability 1/j. Node 0 is an absorbing node, and there are no other transitions. We will consider only random walks that start at node N and end at node 0. Define X_j to be a binary random variable which indicates whether a random walk visits node j.

What independence properties can we state about X_0,...X,_N?

What is P(X_j = 1)?

Ponderings on Progress

2009-02-07T09:44:00.011-05:00

As part of a user study, I recently read through a number of old papers from NIPS, which is a major machine learning conference. Some of these papers were almost twenty years old. Although it isn't really surprising in retrospect, I was struck by how shoddy and ad-hoc many of these papers were by today's standards. What's more, many were written by researchers who are now very well known in the field.

Any hybrid or empirically driven field undergoes a "mathematization" or formalization process as it matures (some fields take longer than others). And while we're still very, possibly impossibly, far off from general artificial intelligence, we've at least learned to pose more precise questions. Progress, it's a good feeling.

My Favorite Quotes

2009-02-02T23:05:00.025-05:00

I've received a few questions/comments regarding my favorite quotes page over the years, so I thought I'd take the opportunity now to elaborate on my choices there.

I created the quotes page when first constructing my website over spring break in 2005. The page is as much (or more so) for myself than it is for others. I wanted to have a repository of useful quotes and figured that it could also be useful for other people as well. The basic criterion for inclusion are elegant phrasing and applicability to broad contexts. Beyond that, I just include quotes if they feel right.

Upon examination, there appears to be three main themes which can be fuzzily defined as being truthful to ourselves, showing empathy towards others, and having general optimism about the human potential. Those who know me well won't find this particularly surprising, since these themes resonate very strongly with my general views on life, the universe, and everything.

The remaining "rogue" quotes, as well as few mainstream ones, stem from notable moments from younger days. And since I'm presently in the sharing mood, brace yourselves for some Yisong trivia! Let's proceed in alphabetical order.

"Any fool can criticize, condemn and complain -- and most fools do. But it takes character and self-control to be understanding and forgiving." -- Dale Carnegie. This comes from the book, How to Win Friends and Influence People, which I read in the summer of 2005. The somewhat pretentious title notwithstanding, I found the book quite enlightening, as it very succinctly discussed rules of thumb for understanding and successfully collaborating (in a very general sense) with other people.

"Don't cry because it's over, smile because it happened." -- Dr. Seuss. As was the case for many of us back in the day, I became very emotional during the weeks leading up to my high school graduation (June 2001). Our yearbooks allowed seniors space for a personal quote or some such thing. I simply came across this Dr. Seuss quote while reading the material other seniors had used, and it's stuck with me ever since.

"Some men are born great, some achieve greatness, and some have greatness thrust upon them." -- William Shakespeare, "Twelfth Night". I did my fair share of acting back in high school, and our production of "Twelfth Night" was by far my favorite. To me (and probably the entire cast), this was the defining quote of the play.

"It's wisdom to recognize necessity, when all other courses have been weighed, though as folly it may appear to those who cling to false hope." -- J. R. R. Tolkien, The Lord of the Rings. It's pretty obvious I'm a Tolkien fanatic (see three posts down). While the book contains many memorable quotes, very few actually translate well beyond the context of Middle Earth. And so we'll just have to settle for this one.

"You treat a disease, you win, you lose. You treat a person, and I guarantee you'll win no matter the outcome." -- Robin Williams, "Patch Adams". For a long time, "Patch Adams" was far and away my favorite movie. In part due to being overly romantic at the time and in part due to dealing with hormonal shifts, I always teared up at this line, which occurs towards the end of the movie. I did, in fact, use this as my senior quote for my high school yearbook.

Joining the Trend Towards Photography Snobbery

2009-02-01T08:29:00.006-05:00

My new digital SLR camera arrived two days ago. Since then, I've practically forsaken all other activities in favor of rampant photo shooting -- with now approximately ten hours spent and a thousand pictures taken. Since I'm still learning how to use the camera (and also don't know much about photography), the percentage of good pictures is quite low.

Nevertheless, I did get a few beauties like the one below (post-processing done using Adobe Lightroom). You can find the rest on my Flickr photostream.

25 Random Things About Me

2009-01-31T09:00:00.009-05:00

There's currently a posting phenomenon happening on FaceBook where people write down 25 random things about themselves, and then ask other people to do the same thing. While I declined to subject others to this silliness, I humored my former theater/drama mentor Tony DiSanto and made a list of my own. So here goes.

1. I also prefer my own company. I typically like to spend a good chunk of my time being alone.

2. When I was a wee lad, I wanted to be an astronaut, then a paleontologist, then a basketball player, then a famous musician, and finally I gave up all hope.

3. I am a compulsive gum-chewer. I like to chew gum when I'm trying to concentrate.

4. I think people who try to make everything "deep", or try to always sound "deep" are retarded.

5. As such, I think literature snobs who bash The Lord of the Rings for not being "deep" or something related to "deep" are, in fact, retarded.

6. To all those people who think scientists are ideologically against religion: science is not the Truth, but merely the body of methods by which we uncover the truth. It has been evolving over the years, and will continue to evolve to complement our growing understanding of the universe and technological progress.

7. I'm a little disturbed by how much blind hope Barack Obama has generated (despite his attempts to be realistic). It speaks to a great inefficiency in human nature.

8. I believe morality can exist in the absence of religion and even emotions.

9. I secretly yearn for the days when I could pick up a random fiction novel without having weird feelings that I should be doing something more productive.

10. Unlike most coders who listen to trance or techno, I like listening to cheerful, sometimes folksy music when I code (think Great Big Sea).

11. People growing old makes me sad.

12. I wish I was a better public speaker. I have to give a few research talks every year, many of which will find themselves on the internet somewhere.

13. One of the biggest adjustments in my life has been accepting (and even embracing) failure as part of the learning process.

14. But still, it would be nice if we could just plug our brains into some system and suddenly knew Kung-Fu.

15. I used to think math was intrinsically beautiful.

16. I still think that to a degree, but my view of elegant mathematics has been shaped by what I think is useful mathematics.

17. I like to work on math problems when taking extended dumps. It sure as hell beats reading magazines.

18. Girls are hot.

19. Sometimes, I wish I was an android so I could work and work without suffering from any distractions whatsoever.

20. I currently weigh 155 pounds butt naked (I've lost about 10 pounds since entering grad school).

21. I have extremely dry skin. I'm constantly putting on lotion.

22. One of my legs is noticeably shorter than the other.

23. "I have been and always shall be your friend."

24. I sometimes wonder to what degree other animals (e.g., a dog) can appreciate beauty (of a sunrise or something).

25. I'm up this early because I'm going to try (and probably fail) to take pictures of the sunrise.

As for that last item... I failed miserably (what a surprise).

A Classic Revisited

2009-01-27T16:11:00.003-05:00

I recently started re-reading my favorite book from younger days: The Lord of the Rings by J. R. R. Tolkien. It's been many years since I last visited the written world of Middle Earth, so it's high time for a little reunion.

I think the movies have done wonders for bringing the sheer breadth, creativity and passion of his story to a broader audience. I was one of the few people who relished the numerous "fake endings" at the end of The Return of the King (Peter Jackson was really in a bind here, since purists would've probably burned him at the stake if he'd made the ending shorter). I used to maintain a pseudo-tradition of re-reading the book every summer (I naively thought I could keep that up indefinitely). Every time I finished the book, I'd try to conjure up my own depiction of the endings. So for me, it was a thrill to see it all on the big screen.

But as with nearly all cinematic adaptations, the movies fall far short of the depth offered by the original text. It's hard for me to believe, but I'd actually forgotten just how elegantly the book is written. The book is best read slowly and with care. Those who just want to quickly absorb some gimmicky plot to pass the time will probably find the book quite boring and somewhat old fashioned. While some aspects of the writing style do feel a little dated, I think the linguistic shifts of the last 50 years have left the book largely unscathed. What is required is simply greater commitment from the reader.

Anyhow, enough of writing blog posts. There's serious reading to be done!

Morning Internet Hypnosis

2009-01-18T10:56:00.007-05:00

Unless I have something pressing to do in the morning (e.g., a meeting), I usually loiter on my sofa and spend some time surfing the great World Wide Web for shits 'n giggles. In this semi-hypnotized state, I often lose track of time -- half an hour, occasionally even an hour might go by.

Inevitably, I am yanked back into reality by the sound of one of my roommates entering the bathroom. Like most people, my kidneys have been hard at work during the night, thus requiring me to relieve myself in morning. Not surprisingly, this need becomes only more pressing after spending an extra 30 minutes not urinating. The worst case scenario arises when when my roommate decides to also take a shower, which is what happened earlier today.

At this point, not even the great WWW can block out the urgent warning signals my bladder is sending to my brain. Unfortunately, there isn't too much I can do other than to try to return to my semi-hypnotized state. You see the dilemma I'm faced with (it's tragic, I know). Of course, the obvious solution is to use the bathroom before I commence my daily loitering. But believe me, on most mornings (especially this morning), that is much easier said than done.

(no, I have never made water in my pants -- at least not since I was a young lad still earning my second set of teeth)

The Enright Hotel

2009-01-09T20:50:00.002-05:00

The dynamic duo, Bill Prokopow and Steve Hackman, have formed a new group called The Enright Hotel. Their website contains a number of sample songs and videos, most of which are quite good. Their MySpace page describes their music as sounding like "the Postal Service but with Simon and Garfunkel harmonies and a taste of Chopin." They were also finalists in the John Lennon Songwriting Contest with their entry The Pendulum Song.

Both Bill and Steve were musical directors for The Other Guys during their respective years at UIUC. Bill and I actually sung together with the Glee Club for over two years, where his musical leadership was widely appreciated by everyone involved. With any luck, we'll be hearing much more from these guys in near future.

Automatically Generating Blog Post Labels

2008-12-30T11:47:00.007-05:00

I recently converted to the new layout provided by Blogger. As part of the face-lift process, I thought it would be fun to label my posts. You can find all the labels I came up with on the right sidebar.

In reality, these labels represent clusters of blog posts that are similar to each other in some way. Posts can belong to multiple labels or belong to none. Some labels might be very similar (so many posts belong to both) -- some might even be nested labels (although that doesn't happen in this blog).

Furthermore, these labels are very view-dependent. For example, while I have separate labels for computer science and machine learning (which is primarily a computer science sub-area), I have one big label for science and technology in general. But given my particular interest in life extension, I decided that could be kept separate from the rest.

I started wondering about how one might automatically cluster my blog posts. It's possible that conventional clustering techniques will yield interesting results, but I find that unlikely. That might work OK on very structured and high volume blogs like Overcoming Bias. But conventional techniques ignore the fact that these clusters are view dependent, so we probably need to leverage some amount of background knowledge (and possibly the network structure) for most blogs. In addition, we need a clustering model which can handle the fact that some posts belong to multiple labels and some posts belong to no labels. There is also a time-dependency factor that might play a big role.

Why is this interesting? First of all, I think it's compelling enough to be able to discover a person's view (or projection) of the global topic/discipline hierarchy. It might also help us discover more about ourselves and how we view the world (since the clusters discovered by any algorithm will inevitably be different from our own manually generated labels). From a technical standpoint, depending on the approach, tackling this problem might yield insight on designing new clustering techniques or utilizing the background information of the internet.

Like most ideas, this one probably won't lead to any interesting results. But it's fun to think about. In case anyone is interested, I harvested my blog posts (with labels and all) and it's available here.

Combinatorics Question

2008-12-28T02:20:00.007-05:00

Suppose we are sampling permutations of the first N positive integers from a uniform distribution. For any two integers x and y, what is the probability that they lie in same cycle?

For example, suppose we sampled the following permutation using the first 5 positive integers:

2 3 1 5 4

Then {1,2,3} form a cycle because position 1 points to 2, position 2 points to 3, and position 3 points to 1. Likewise {4,5} also form a cycle.

Machine Learning Video Lectures

2008-12-27T19:31:00.009-05:00

I've been spending a lot of time this week watching videos on videolectures.net (my way of celebrating Christmas). I found several that are very interesting as well as accessible to general audiences, and so I thought I'd share. The first few are short interviews with prominent machine learning researchers.

Bernhard Schölkopf (Max Planck Institute)

Tom Mitchell (CMU, chair of machine learning department)

Fei-Fei Li (UIUC -> Princeton)

Jon Kleinberg (Cornell)

Benardo Huberman (HP Labs / Stanford)

Some themes harvested from these interviews:
1) as any field matures, mathematical rigor inevitably becomes more important
2) scientific disciplines must evolve in order to pose (and answer) more challenging questions...
3) ... as such, interdisciplinary research tends to be the most compelling
4) it's difficult to predict the future, so make sure to stay flexible
5) general AI is probably still very far away

The next two are invited talks at large machine learning conferences and are a bit longer. Both are fairly high level and very well presented.

Andrew Ng (Stanford) invited talk at ICML 2008

Jon Kleinberg (Cornell) invited talk at KDD 2007

NIPS 2008

2008-12-21T00:13:00.011-05:00

I recently spent a week in the Vancouver area attending NIPS 2008 (which stands for Neural Information Processing Systems). While the name might be slightly misleading, it is in fact the largest machine learning conference in existence. The main conference is a marathon that stretches across four days, with each day starting from the early morning and ending in poster sessions that often last until midnight.

NIPS has a strong tradition of tying together machine learning and biology-related research. This year featured invited talks by Daniel Wolpert on computational methods for human-motor control, Sabastian Seung on mapping every neural connection in the human brain, and by Rebecca Saxe whose talk I unfortunately missed. Attending research conferences can be quite invigorating. As one of the great enabling fields, machine learning offers remarkable promise in designing useful models for all types of application domains. I often find myself very inspired after listening to success stories.

After the main conference concluded, we gathered onto shuttle buses for a two and a half hour ride to the Whistler ski village, where the workshops are held. The NIPS workshops are the best research workshops I have ever attended or heard about. The location is great, the venue (the Hilton Ski Resort at Whistler) is fantastic, the audience is littered with prominent machine learning researchers. All in all, the NIPS workshops offer a great forum for discussion regarding exciting new machine learning research.

I'm currently working on algorithms that an information retrieval system (such as a search engine) might use for interactively learning from its users. My results are largely theoretical at the moment (i.e., proofs), but I hope to implement learning experiments on real search systems in the near future. You can catch a video of my workshop talk using the link below.

Scooping the Poop

2008-12-02T23:22:00.002-05:00

Barack Obama has put his foot down. He's not going to scoop the poop.

Computational Sustainability

2008-11-24T08:24:00.003-05:00

Cornell is currently leading an NSF Expeditions in Computing project titled Computational Sustainability. The NSF also funded three other such projects, which are described in their announcement.

In a nutshell, computational sustainability is about developing techniques to formulate and solve large scale problems which mix together environmental, economic and societal needs both for the present and the future. Both the NSF and members of the project hope to create a new interdisciplinary field in a manner analogous to computational biology.

As the computer science field matures, various sub-areas will likely spin off into their own fields (something I've pondered about before). Some sub-areas will join forces with sub-areas from other fields. Cornell is currently home to a department of biological statistics and computational biology, which mixes together (not surprisingly) relevant areas of biology, statistics and computer science.

As I see it, the creation of these fields is important for two main reasons. First, it bridges the communication gap between different fields. While attending the kick-off meeting for the Institute for Computational Sustainability, I was surprised by the lack of awareness even here at Cornell. Researchers from many other fields were seeking answers to problems that computer scientists have already contemplated (at least in some idealized setting).

These institutions are also particularly important for enabling disciplines such as computer science. I've seen enough assistant professors be denied tenure because their research was deemed to have insufficient fundamental impact within computer science. We must have a system that recognizes the scientific value of their work.

A Message From Aubrey

2008-11-10T14:05:00.004-05:00

I recently received this plea for help from my new FaceBook friend Aubrey de Grey. It's too bad I can't help out, since my family does have a history of cancer. But hopefully some of my esteemed readership will take the initiative.

---------------------------------------

Hello everyone,

My long-time friend and colleague Chris Heward was recently diagnosed with terminal oesophageal cancer. With standard treatment he has a life expectancy of only a few months. However, he is in the fortunate position to be able to access very novel and promising therapies, and the one he is working to get is very promising indeed, even for terminal cases. I'm writing this message because YOU can help - the treatment needs donations of white blood cells. For more details, please see the FaceBook page that has been set up for this purpose:

http//www.facebook.com/note.php?note_id=33488382415

and join the group "Friends of Chris Heward":

http://www.facebook.com/group.php?gid=37638890926

This treatment is very new and experimental, but I believe it has a real chance of giving years - maybe even decades - of extra life to those like Chris who otherwise are at death's door. Without Chris's efforts to obtain this treatment for himself, it would probably have been years before anyone in the West received it. Thus, Chris is blazing a trail here that may save a very large number of lives. Please do what you can to help.

Cheers, Aubrey

Yet Another Post About Politics

2008-10-29T10:02:00.007-04:00

I never intended for this blog to become a reservoir for political commentary. But it's that crazy time of the 4-year cycle again, so I can't help but read and ponder about it.

All across the internet, the criticism directed at the McCain campaign has been severe, most notably by the scientific community. A common complaint is that the Republicans come off as being anti-intellectual.

It has been noted that oftentimes, people are anti-intellectual in defiance to elitism in all its forms. By definition, the elites are a minority. And thus, what else can the Republicans do but try to appeal to a majority? Yes, McCain's and Palin's public comments deriding science are troubling. But, sadly, they are effective for a large segment of the population.

Most entries in A Vote for Science feel (at least partially) like they're preaching to the choir. Some are blatantly dismissive of anti-intellectual stances. Tell me, do you think this clip will actually convince anyone in the middle?

It's important to look beyond Barack Obama's gifts as an orator. I do think debating and speaking skills are important, since they reflect one's ability to dissect, organize, and subsequently present information. But one man can't run this nation alone, so we should listen to what his advisers have to say.

And if we still can't decide, then maybe we should just have a dance off.

On the Election

2008-10-25T16:25:00.006-04:00

It seems the United States has been on a path of polarization for the past eight years. Those who watched the debates were probably disappointed by how insubstantial they were. Personally, I found the SEA's forums on health and energy policy much more informative. It should also help quiet some doubts people might have about the quality of John McCain's advisers.

As an aspiring scientist, it's probably not surprising that I believe America's science and innovation platforms are drivers of growth and improving quality of life. Ultimately, I don't see too much difference between McCain's and Obama's goals in this regard. Obama has certainly been more organized about getting his message to the public. While late to the game, McCain's advisers are probably every bit as competent as Obama's. In fact, I wouldn't be surprised if many of them share views that they cannot publicly acknowledge due to political affiliations. From watching the SEA's forums and reading both candidates' current policies (as stated on their websites), the general sense I get is that the McCain campaign feels we should be a little more pragmatic in solving these issues, while the Obama campaign emphasizes a more holistic view.

A science activist site, A Vote for Science, has launched a challenge to gather scientists' endorsements. The response has overwhelmingly been in favor of Obama. In fact, 65 American Nobel Prize winners signed an open letter in support of Obama, while not a single scientist has (to date) publicly endorsed McCain on the site.

It's not very surprising that scientists (including social scientists such as economists) generally support Obama. Being a professor (albeit in law), he probably understands the intellectual rigor involved with scientific inquiry. Obama comes across as a man more in touch with the needs and ideals of the scientific community.

Nonetheless, many of the endorsements on A Vote for Science leave much to be desired. As an extreme example, Vin Cerf endorsing Obama because of net neutrality is a little ridiculous. Yes, he was probably overemphasizing the point in order to make a point (that I happen to agree with). But net neutrality is probably not something that should sway votes.

My friend Lev recently sent around a letter explaining his support for McCain. In it, Lev emphasizes ignoring a candidate's public remarks (largely made to appeal to various segments of the population) and looking at the track record. Having bipartisan success is a strong indicator of being a successful president, which is something McCain has had plenty of. If a candidate doesn't have a proven track record (as in the case of Obama), then it might be wise to assume the worst or something close to it. I highly recommend that everyone read his letter since he makes a pretty compelling argument. Another important point is that foreign policy, which is widely viewed as one of McCain's strengths, is an area where the president is fairly autonomous.

McCain is considered unappealing and erratic partly because of the pressure to win over the ultra-conservatives. But it works both ways. For example, Obama is on record for wanting energy independence in 10 years. This is something even one of his energy advisers acknowledged was very unlikely. Is Obama a pioneering visionary, or just making empty promises?

At this point, it probably sounds like I'm going make an endorsement for McCain. In truth, I am relatively indifferent about the two candidates. If you read between the lines of what they and their advisers say, both candidates seem open minded and will put the needs of the country first.

Supporters of McCain typically say he stands for principles, is realistic and pragmatic, and doesn't make empty promises. I think the next president should be the one who can get the most done to resolve the many complex issues he'll inherit, as well as deal with unforeseen developments. Under the watchful eye of public scrutiny, I think Barack Obama is better equipped to manage the complexities of 21st century governance. For all intents and purposes, John McCain has mismanaged his campaign. Can you imagine John McCain hosting this Economic Competitiveness Summit? Then again, maybe he doesn't have to in order to be a good president -- after all, one of his economic advisers would suffice.

I admit to being moved by his eloquence when I first heard Barack Obama give a speech. It alarms me how much people make their decisions based on almost irrelevant emotional factors. Ultimately, this campaign has put a spotlight on very important issues we must deal with. Given the increase in scrutiny due to eight years of mismanagement under Bush, I believe either administration would tackle these issues responsibly. I do worry a little about the prospects of having both a Democratic White House and Congress. Of course, my opinion doesn't really matter, since I didn't even register to vote.

[Edit 10/27/2008] - Peter Norvig's views

Fill in the Blank

2008-10-22T01:15:00.002-04:00

42, 46, 52, 60, 110, 132, 222, 1120, ?

No cheating!

Rejecting Racism and Elitism

2008-10-21T00:09:00.001-04:00

Many people probably have heard the stories, but this clip is really something else. While, at a high level, I appreciate many Republican ideals, this whole social conservatism thing is getting a little out of hand. Even Colin Powell agrees.

Having spent a number of years living in the south side of Chicago, I've been through my share of racism. It's not pretty, and a little bit of a shock when you first hear someone call you a "chink" or make generic Asian-accented sounds in your direction.

While we see that racism is still quite prevalent in many parts of the United States, I'm happy to say that my days of enduring racial harassment are more or less over. Working in a meritocratic environment can work wonders in dispelling tasteless and counterproductive mindsets.

Given the current social and political landscape, ultra-conservatism is closely linked to other perceived intellectual deficiencies such as creationist dogmatism. It's easy to put down others as being uncivilized or somehow intellectually challenged. But we are living in a society that's highly interconnected and increasingly places more value on mental flexibility. We should therefore be educating others.

But it's not enough to just pay lip service while still maintaining that aloof (alienating) elitism. Having spent time now at both a public and an ivy-league university, I can definitely sense an elitist undercurrent at Cornell. Alienation works both ways; everyone would have a far easier time realizing the value of keeping an open mind if those of us who are privileged keep an open mind (and open arms) as well.

I particularly liked this bit on the Guardian (although the Ghana entry was very sad).

A Pleasant Surprise

2008-10-06T22:14:00.006-04:00

I came home today to be greeted by a small package from the Methuselah Foundation. To my surprise, the packaged contained a recent book by Aubrey de Grey.

Somewhat incredibly, the book was autographed by the man himself.

I never realized how much return I could expect from a rather small donation. Well, an earlier post of mine did get picked up by their blog scans a while back. That the post starts by discussing urinals was simply gravy.

On the notion of heroes

2008-10-05T14:25:00.010-04:00

Paul Graham describes his personal heroes as those who've had a tremendous positive influence on his life. When making decisions, he'd ask what they would do when faced with the same situation. This, he states, is a stricter definition than just admiration. I generally agree with his description. His list of heroes (and justification thereof) was a fun read. I imagine it was also fun coming up with the list, so I figured I'd do the same.

Given that my accumulated wisdom cannot hope to match his, this list will probably be far less impressive. I'll also use a slightly different definition of heroism. My quarter-century of experience has been fraught with personal paradigm shifts. As a research oriented academic (or maybe it's academically oriented researcher), I probably value clarity of thought and purpose more than most people. The people I list below are those who have positively influenced my outlook and approach on life, the universe, and pretty much everything. It's impossible to provide a comprehensive list, so think of these as my personal all-stars.

My high school friends - I've been very fortunate to have found lasting friendships dating back to my IMSA days. I've known Steve, John and Navreet for over ten years now. We also went to college together. They each pulled me towards different perspectives and have thus (in my humble opinion) made me a better person.

John is almost certainly the most competitive person I've ever known. He's also one of the most practical. He has accomplished much in his brief career, and is always excited to tackle the next problem or learn about the next big thing. He is, in some ways, the most driven person I know -- because he genuinely loves everything he does.

Navreet's basic approach to life (as I understand it) is pretty straightforward: you have to do what makes you happy, and you have to be prepared for when you change your mind. He has always been active about discovering the opportunities that await him. He convinced me to join ACM@UIUC, encouraged me to work on research projects, and asked me to help with Illini Book Exchange. Looking back on my undergraduate experience, those were probably the most meaningful things I'd done during that time.

Steve is the most cerebral of the bunch. We've spent many hours discussing a wide range of topics. I find that talking to him keeps me honest. He challenges my core assumptions and embraces my faults. Indeed, I sometimes treat him as a personal oracle for bullshit detection. We also share very similar hopes for the future. His outlook is generally sunny, and his optimism has proven resilient to life's many challenges.

J.R.R. Tolkien - this is perhaps an acknowledgment to childhood nostalgia. When it comes to quality story-telling, I always use The Lord of the Rings as the standard to compare against. Tolkien started writing these stories as a teenager in the trenches of World War I, and he continued to work on them until his death. He quite literally infused his essence into the lore of Middle Earth. The Tale of Beren and Luthien reflected his own pursuit of the his love, Edith. Like Paul Graham's description of Leonardo, Tolkien did not initially write these stories for public consumption. He wrote them as a tool for personal exploration.

Coincidentally, my love for The Hobbit sparked a love for theater and subsequently choral music. I joined my first choir at the age of 12. To this day, I'm still an active choral member (now with the Cornell University Glee Club).

Isaac Newton - I found Paul Graham's description of Newton very compelling, and it's worth repeating it here:

Newton has a strange role in my pantheon of heroes: he's the one I reproach myself with. He worked on big things, at least for part of his life. It's so easy to get distracted working on small stuff. The questions you're answering are pleasantly familiar. You get immediate rewards—in fact, you get bigger rewards in your time if you work on matters of passing importance. But I'm uncomfortably aware that this is the route to well-deserved obscurity.

To do really great things, you have to seek out questions people didn't even realize were questions. There have probably been other people who did this as well as Newton, for their time, but Newton is my model of this kind of thought. I can just begin to understand what it must have felt like for him.

You only get one life. Why not do something huge? The phrase "paradigm shift" is overused now, but Kuhn was onto something. And you know more are out there, separated from us by what will later seem a surprisingly thin wall of laziness and stupidity. If we work like Newton.

Daring to dream big is almost a necessity in many areas of research. Our intellectual and technological progress now defines our evolution. Which leads me to...

Aubrey de Grey - A computer scientist turned biologist. The leader of the Methuselah Foundation. Champion of the SENS platform. History will judge him to be either a hero or a fool. At least he cares enough to try.

Professors at Cornell - While dreaming big is great in the abstract, the implementation requires attention to details and an understanding of the fundamentals. Some people get caught up in the big ideas and are out of touch with reality. Others are mired in the details and cannot contemplate the bigger picture.

Since arriving at Cornell, I've received some of the best education and mentoring that can be reasonably expected, and have learned to somewhat bridge this gap. Most of the credit, of course, goes to my advisor, but I'm also appreciative of the Kleinberg brothers and others at Cornell as well.

Dwight Schrute - oh yeah.

Live Forever or Die Tomorrow?

2008-09-24T00:29:00.011-04:00

Every year at the glee club retreat, we play a game called the truth circle (or "Who's here?"). We all form a circle and step forward when a question is asked that is true for us. It's intended to be a low-pressure way to share personal information with rest of the group.

This year, I asked the awesome question, "Who expects to be alive in 100 years?" To my delight, quite a few members (though clearly a minority) stepped forward. This in itself probably wasn't that surprising or interesting. But then someone asked the follow up question, "Who would rather live forever than die tomorrow?" I didn't think too hard before stepping forward (the questions asked in this game are often under-specified and thus open to interpretation). About an equal number of the other members (though not necessarily the same ones) also stepped forward.

Initially, I assumed that more people would step forward, since I'd certainly take my chances with eternal life given the alternative. Thinking back, I find this result less surprising. It wasn't too long ago when I proudly declared that I'd rather be dead than feeble and senile. But thanks to Aubrey de Grey and others, I now envision eternal life as one of constant rejuvenation, learning, and (dare I hope) joy. How can you go wrong with that?

Stupid Math Problem

2008-09-17T00:35:00.004-04:00

What are the values of A and B in the diagram below?

Make Way For Progress (Link Dump)

2008-09-09T17:33:00.005-04:00

This post serves to provide my small (but dedicated!) readership with some recent popular science articles I've come across lately. As you might expect, the following articles are related to life-extension efforts in its many flavors.

The progress is really quite remarkable. Here's the list:
1. robot with biological brain
2. fighting cancer with your own immune system
3. stopping the aging process in mice livers.
4. long-life gene discovered
5. understanding telomerase.
6. natural tumor suppressor enzyme

Of course, these are largely initial or intermediate results in the ongoing efforts to fight aging and death. So one shouldn't think that a miracle is just around the corner. It seems to me that progress will take the form of baby steps like the articles listed above. We don't don't need a big miracle, just a million little ones.

Reading too much into popular news articles can also lead to grossly mis-interpreting the actual research. For example, this news article on BrowseRank provides a fairly poor description of the goals and results as stated in the actual scientific paper. The comments left by other readers don't make me feel any better. To be fair, the article used BrowseRank mainly as an example of recent research to segue into a discussion about industrial research and competition in the search business -- so it did form a cohesive story.

Nonetheless, the overall trend is pretty clear. I'm pretty excited about the future, and I think you should be too.

Lost in Translation

2008-09-02T00:32:00.007-04:00

Having returned from the Far Orient, I can report with great confidence that globalization and market growth is alive and well over yonder. In order to accommodate the rising number of foreign visitors, the Chinese have appended English to many informative signs across the country. As we shall see, some meaning was inevitably lost during translation, often to hilarious effect. Most of the following photos are courtesy of John Carrino.

It seems like they just strung words together for this one.

Now we all know the legal rights violation hotlines.

Taller children, hooray!

That's deep...

...but this is even deeper.

Probably nothing was lost in translation on this one, but funny nonetheless.

Yeah...

Remember kids, no striding!

How much to pay?

2008-08-30T20:52:00.008-04:00

How much should I pay? It's one of the great cognitive dissonances involved with traveling to countries such as China. After some lively bargaining at the flea markets, John and I were able to grab most of our souvenirs at around $2-$6 per item. While we at first enjoyed the process, the novelty effect wore off towards the end. It even became a chore at times, trying to knock off that last dollar or fifty cents. All the while, surrounding vendors are screaming at us to buy their knock-off products and/or sisters.

On the flip side, I spent $8 per drink at bars and nightclubs with nary a moment's pause. In fact, John and I probably spent more money on drinks (and karaoke rooms) than we did at the flea markets, and we went out only a few nights during our action-packed Sino-Olympic adventure.

As I finished off the remainder of my Chinese currency at the duty-free (i.e., expensive) shops, it struck me how absurd the whole situation was. It's one of the hallmarks of pronounced economic disparity.

Location:	Ithaca, New York, United States
IP Address:	Cornell University (128.84.154.13)
Page Loaded:	www.yisongyue.com/
Referrer:	www.easychair.org/conferences/submission.cgi?submission=468064;track=24725;a=a09cfab06091
Time:	21st August 2010 17:23:23