[Kim+ ICML12] Dirichlet Process with Mixed Random Measures

We held a private reading meeting for ICML 2012.
I took and introduced [Kim+ ICML12] “Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data.”
This is the presentation for it.

DP-MRM [Kim+ ICML12] is a supervised topic model like sLDA [Blei+ 2007], DiscLDA [Lacoste-Julien+ 2008] and MedLDA [Zhu+ 2009], and is regarded as a nonparametric version of Labeled LDA [Ramage+ 2009] in particular.

Although Labeled LDA is easy to implement (my implementation is here), it has a disadvantage that you must specify label-topic correspondings explicitly and manually.
On the other hand, DP-MRM can automatically decide label-topic correspondings as distributions. I am very interested in it.
But it is hard to implement because it is a nonparametric bayesian modal.
Hence I don’t want infinite topics but hierarchical label-topic correspondings, I guess that it will become very useful and handy and fast to replace DPs into normal Dirichlet distributions in this model… I am going to try it! 😀

Advertisements
This entry was posted in LDA, Machine Learning, Nonparametric Bayesian. Bookmark the permalink.

6 Responses to [Kim+ ICML12] Dirichlet Process with Mixed Random Measures

  1. Hello,

    Thanks a lot for all your open-source help. I have been trying to use your LLDA.py. I was hoping if I could somehow find out how the dataset looks like and how do we provide the preset labels.

    Thanks

  2. Hello again,

    How can I effectively estimate alpha and beta ? The literature on the topic is not very helpful..

    • shuyo says:

      I don’t know it much because I have no motivation to estimate hyper parameters…
      I feel that it is effective in perplexity improvement, but not in quality improvement, e.g. topic-word distribution.
      If you want the latter, I guess it is much more effective to try various pre-processings.

  3. Pingback: Python implementation of Labeled LDA (Ramage+ EMNLP2009) | Shuyo's Weblog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s