Learning on Big Data has been one of the main driving forces behind the staggering advances in artificial intelligence. For many application problems, the fear of overfitting by complex models has largely yielded to the practice of optimizing such models on a large volume of data.
While exciting, we should also bear in mind that there are equally many other, if not more, application scenarios that the data volume is inherently limited. Exemplar cases include predicting the effect of treatment on a rare disease, identifying infrequently appearing visual object categories, and learning from data by interacting with the physical world. In short, we also need to develop methods for “small data”.
In this talk, I will describe several research work in my lab along that direction. I will exemplify them with 3 vignettes: multi-task learning, domain adaptation and zero-shot learning. The theme is to investigate learning models for small data by soliciting help from other tasks and related (big) datasets. I will describe our efforts in both developing methodologies and applying them to real-world problems.