I wanted a chronological table or a brief history of machine-learning but couldn’t find it.

So I make it with famous models and algorithms.

For each obscure item, I select its date by introducing its name in principle.

Please tell me if there are mistakes or shortages.

1805 | Method of Least Squares |

1901 | PCA (Principal Component Analysis) |

1905 | Random Walk |

-1925 | Logistic Regression |

1936 | Fisher’s Linear Discriminant Analysis |

1946 | Monte Carlo Method |

1950 | RKHS (Reproducing Kernel Hilbert Space) |

1950s | Markov Decision Process |

-1957 | Perceptron |

1958 | Kalman Filter |

1960s | Hidden Markov Model |

-1961 | Neural Network (Multi-Layer Perceptron) |

1962 | L2 regularity |

1967 | Viterbi Algorithm |

-1970 | Metropolis-Hastings algorithm |

1973 | Dirichlet Process |

1975 | Mean Shirt Clustering |

1975 | Genetic Algorithm |

1977 | EM Algorithm |

1980 | Graphical Modeling |

1982 | SOM (Self-Organizing Map) |

-1984 | Gibbs sampling |

1984 | CART (Classification And Regression Tree) |

1984 | PAC learning |

1984 | Markov Random Field |

1985 | Bayesian Network |

1986 | Back-Propagating Algorithm |

1988 | Variational Bayesian method |

1989 | Graph Cut |

1994 | L1 regularity |

1994 | ICA (Independent Component Analysis) |

1995 | Support Vector Machine |

1995 | AdaBoost |

1995 | Particle Filter |

1999 | LSH (Locality Sensitive Hashing) |

2000 | FP-Growth |

2001 | Item-based Collaborative Filtering |

2001 | Random Forest |

2001 | CRF (Conditional Random Field) |

2001 | Expectation Propagation |

2003 | Slice sampling |

2003 | LDA (Latent Dirichlet Allocation) |

2006 | Deep Learning |

- 11/08/2011: Add Back-Propagating Algorithm and Deep Learning.

Advertisements

For neural network, could add “back-propagating algorithm – 1986” and “Deep learning – 2006”.

References: ‘Learning representations by back-propagating errors’, D. Rumelhart et al., 1986, Nature. and ‘Reducing the dimensionality of data with neural networks’, G. Hinton et al., 2006, Science.

I see!

I’ll add them. Thanks!!

genetic algorithm 1975

Thanks! I added it!

Look ; I may be going a bit out here. I was eventually led to this site, looking for something I found in Discover Magazine, in either the late eighties, or early 90s, an article about a fellow w/ a team who had some funding to sort of (it seemed to me) start over on the whole ai problem. He was creating a computer that turned data into info and asked questions from it’s knowledge base. He estimated that the thing had the iq of a 2 year old. I don’t expect you to be expert about Discover Magazine, of course. Could it be the beginning of CYC? If you can help me with this, even indirectly, please write me: Thaddeus.Slamp@gmail.com. I really want to find out what happened to the darned thing, and have for a long time. I figure an interval of about 20 to 25 years, is enough to make some judgement. I realize you are interested in the theoretical algorithms of the subject (or I should say thats how it seems to me your time-line orients). Nice if you could help me track this down. (more on the device: They were doing actual new hardware, I believe, concerned w/lack of commonsence knowledge base, they were teaching it these sorts of things [if you jump in the air you fall down; really simple stuff]). It would process data in it’s database in spare time, and have new questions to ask from all its data, especially the newly organized stuff, generally the nest day (like sleep and dreams in humans). Ok…I’ve rambled enough. Hope you can help. Thanx.

I suppose I cannot help you because I don’t know of CYC at all. Sorry.

Though I am also interested in Question Answering a little, as you guessed,

I would like to tackle more primitive tasks for the present. (e.g. Sentiment Analysis, Text Summarization and so on)

Thanks.

So what about Reinforcement Learning?

I intend that Markov Decision Process includes it.