Incremental Learning


Machine learning for real-world applications often confronts such a difficulty that a complete set of training samples is not given in advance. Actually in most cases, data are presented as random chunks illustrated in the following figure, streaming continuously and possibly without an end.  For system updating, classic batch approach is to construct a provisional system by learning the collected data so far. Such computing is memory and computational time expensive, because a huge size memory might be required for the storage of data either previously learned or newly presented. Incremental learning is an optimal solution because it can retain the knowledge acquired in the past, and increment the system knowledge with just a single presentation of data. In the field of Incremental learning, we developed earlier several algorithms for one-pass incremental online feature extraction. The work includes Incremental Principle Component Analysis (IPCA), Incremental Linear Discriminant Analysis (ILDA), LDA Merging and Splitting, Chunk Incremental IDR/QR LDA, and curiosity-driven ILDA.


tl_files/paper_upload/New Bitmap Image (2).jpg

Related articles:

  • N. Shimo, S. Pang, N. Kasabov and T. Yamakawa, "Curiosity-Driven Multi-Agent Competitive and Cooperative LDA Learning," International Journal of Innovative Computing, vol. 4, no. 7, pp. 1537-1552, 2008.|PDF|Bibtxt|
  • S. Pang, S. Ozawa and N. Kasabov, "Incremental linear discriminant analysis for classification of data streams," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, no. 5, pp. 905 -914, 2005.|PDF|Bibtxt|
  • S. Ozawa, S. L. Toh, S. Abe, S. Pang and N. Kasabov, "Incremental learning for online face recognition. ," IEEE International Joint Conference on Neural Networks, 2005. IJCNN '05, vol. 5, pp. 3174 - 3179, 2005.|PDF|Bibtxt|
  • S. Ozawa, S. Pang and N. Kasabov, "Incremental Learning of Chunk Data for Online Pattern Classification Systems," IEEE Transactions on Neural Networks, vol. 19, no. 6, pp. 1061 -1074, 2008.|PDF|Bibtxt|
  • S. Pang, T. Ban, Y. Kadobayashi and N. Kasabov(2010) LDA Merging and Splitting with Applications to Multi-agent Cooperative Learning and System Alteration, IEEE Trans. on System, Man, and Cybernetics-Part B , In press.