:::

詳目顯示

回上一頁
題名:擴充屬性資訊以提升小樣本分類之效果
作者:劉巧雯
作者(外文):Liu, Chiao-Wen
校院名稱:國立成功大學
系所名稱:工業與資訊管理學系
指導教授:利德江
學位類別:博士
出版日期:2010
主題關鍵詞:分類支持向量機模糊理論屬性建構屬性選取classificationsupport vector machine (SVM)mega-trend diffusion (MTD)fuzzy set theoryattribute constructionattribute selection
原始連結:連回原系統網址new window
相關次數:
  • 被引用次數被引用次數:期刊(0) 博士論文(0) 專書(0) 專書論文(0)
  • 排除自我引用排除自我引用:0
  • 共同引用共同引用:0
  • 點閱點閱:1
建構小樣本資料之機器學習模式是一件困難的課題。小樣本資料發生在許多的領域中,例如在醫學界,針對某些特殊疾病的DNA的基因資料,或是在工業界,早期製造系統的產品生產排程等。一般 而言,過少的資料會使得模式在訓練的過程中產生對訓練資料過度配適的情形,而過多的資料屬性又會影響(或干擾)模式學習的正確度以及模式的計算效率。為了改善小樣本資料的學習問題,本研究提出一個處理資料屬性的方法來提升小樣本分類問題的正確率。方法包括建立屬性類別可能值,其藉由模糊裡論的基礎建立資料其各屬性於各類別的可能值,利用這些各類別的可能值來增加小樣本資料的分析資訊以提升分類正確率;方法另一策略為屬性建構,目的在分析既有的屬性間關係,以發掘隱藏的有效屬性。而且為避免因資料屬性過多而干擾模式學習的成果,本研究利用資料間各屬性的相關程度,將高度相關的屬性作合併以整合出指標屬性來提昇學習的正確度。
本研究利用兩個醫學相關資料與一個工業中的排程資料來驗證所提出之屬性擴充之方法不僅能提升分類器之分類正確率並且有較佳的分析效率。
Learning from small data sets is fundamentally difficult. In many data sets such as gene in medicine field or scheduling in the early manufacturing process, the data sizes are often not only small, but they also have high dimensions. Generally, a too small data size will detract modeling accuracy, and too many data attributes will affect the efficiency of the analysis. This research proposed a method for attribute analysis to enhance the analysis efficiency and accuracy for small data set. The proposed method includes two techniques; one called the class possibility method which uses a fuzzy membership function to build up the class possibility value for each data point in every attribute. The other technique called attribute construction aims to non-linearly create hidden attributes and combine attributes with high correlation value into principal attributes.
Three data sets, an early flexible manufacturing system, Pima Indians diabetes data set, and Wisconsin breast cancer data set, are employed to prove the proposed method having better classification performance than other studies.
Aizerman, M. A. Braverman, E. M. and Rozoner, L. I., 1964, The probability problem of pattern recognition learning and the method of potential functions. Automation and Remote Control. 25, 1175-1190.
Altaye, M., Donner, A., and Eliasziw, M., 2001, A general goodness-of-fit approach for inference procedures concerning the kappa statistic. Statistic in Medicine, 20(16), 2479-2488.
Amari, S., and Wu, S., 1999, Improving support vector machine classifiers by modifying kernel functions. Neural Networks, 12, 783-789.
Andrew, P. B., 1997, The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145-1159.
Anthony, J. V., Joanne, M. G., 2005, Understanding interobserver agreement: the kappa statistic. Family Medicine, 37(5), 360-363
Balcan, M. F. and Glum, A., 2006, On a theory of learning with similarity function. In Proc. of the 23rd International Conference on Machine Learning, 73-80.
Basu, S., Micchelli, C. A., and Olsen, P., 2000, Maximum entropy and maximum likelihood criteria for feature selection from multivariate data. Proc. IEEE Int’l Symp. Circuits and Systems. III-267-III270.
Baudat, G.., and Anouar, F., 2000, Generalized discriminant analysis using a kernel approach. Neural computation, 12, 2385-2404.
Bazaraa, M. S., Jarvis, J. J., and Sherali, H. D., 1990, Linear Programming and Network Flows, 2nd, New York, Wiley.
Bensusan, H., and Kuscu, I., 1996, Constructive induction using genetic programming. In T. Fogarty and G. Venturini, editors, (ICML)’96, Evolutionary computing and machine learning workshop.

Bishop, C. M., 2006, Pattern Recognition and Machine Learning, Springer, New York.
Campbell, C., 2002, Kernel methods: a survey of current techniques. Neurocomputing, 48, 63-84.
Chen, S., and Zhang, D., 2004, Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure. IEEE Trans. Systems, Man and Cybernetics-Part B, 34, 1907-1916.
Cheng, T. C. Edwin, Gupta, Jatinder N. D., Wang, G., 2000, A Review of Flow shop Scheduling Research with Setup Times. Production and Operations Management, 9(3), 262-282 .
Cohen, J., 1960, A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20,37-46.
Cortes, C. and Vapnik, V. N., 1995, Support Vector networks. Machine Learning, 20, 273-297.
Devijver, P. A., and Kittler, J., 1982, Pattern Recognition: A Statistical Approach. Engledwood Cliffs: Prentice Hall.
Demšar J., 2006, Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1-30.
Dubois, D., Prade, H., 1980, Fuzzy Sets and Systems Theory and Applications, New York, London, Toronto.
Dy, J. and Brodley, C., 2000, Feature subset selection and order identification for unsupervised learning. Proc. 17th Int’l. Conf. Machine Learning.
Filippone, M., Camastra, F., Masulli, F., and Rovetta, S., 2008, A survey of kernel and spectral methods for clustering. Pattern Recognition, 41, 176-190.
Fodor, I. K., 2002, A survey of dimension reduction techniques. LLNL Technical Report, UCRL-ID-148494.
Furey, T., Cristianini, N., Duffy, Bednarski, N., Schummer, D. M., and Haussler, D., 2000, Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics, 16, 906-914.
Ghazavi SN and Liao TW, 2008, Medical data mining by fuzzy modeling with selected features. Artificial Intelligence in Medicine ;43:195-206.
Gomez, G., and Morales, E. F., 2002, Automatic feature construction and a simple rule induction algorithm for skin detection. Proceeding of the ICML workshop on machine learning in computer vision, 31-38.
Guyon, I., and Elisseeff, A., 2003, An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157-1182.
Harville, D., 1977, Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72, 320-338.
Hastie, T., Tibshirani, R., and Friedman, J., 2001, The Elements of Statistical Learning. Springer series in statistics. Springer, New York.
Hong G, Asoke KN., 2006, Breast cancer diagnosis using genetic programming generated feature. Pattern Recognition 39:980-987.
Hu, Y. J., 1998, A genetic programming approach to constructive induction. In Proc. Of the Third Annual Genetic Programming Conference, 146-157. Morgan Kauffman, Madison, Wisconsin.
Huang, C. F., and Moraga, C., 2004, A diffusion-neural-network for learning from small samples. International Journal of Approximate Reasoning, 35, 137-161.
Ikeda, K., 2006, Effects of kernel function on Nu support vector machines in extremen cases. IEEE Trans. Neural Networks, 17, 1-9.


Jahangirian, M., and Conroy, G. V., 2000, Intelligent dynamic scheduling system: the application of genetic algorithms. Integrated Manufacturing Systems, 11(4), 247-257.
Jennrich, R. I., and Schluchter, M. D., 1986, Unbalanced repeated-measures models with structured covariance matrices. Biometrics, 42, 805-820.
Jeong, K. C., and Kim, Y. D., 1998, A real-time scheduling mechanism for a flexible manufacturing system: using simulation and dispatching rules. International Journal of Production Research, 36(9), 2609-2626.
Jolliffe, I. T., 2002, Principal Component Analysis. 2ed, New York, Springer.
John, S. T. and Nell, C., 2004, Kernel Methods for Pattern Analysis. Cambridge University Press.
Kari T., 2003,Feature extraction by non parametric mutual information maximization. Journal of Machine Learning Research 3:1415-1438.
Kim, C. O., Min, H. S., and Yih, Y., 1998, Integration of inductive learning and neural networks for multi-objective FMS scheduling. International Journal of Production Research, 36(9), 2497-2509.
Kohave, R., John, G., 1997. Wrappers for feature subset selection. Artificial Intelligence, 273-324,
Koza, J. R., 1992, Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge Massachusetts.
Krawiec, K., 2002, Genetic programming-based construction of feature for machine learning and knowledge discovery tasks. Genetic Programming and Evolvable Machines, 3(4), 329-343.
Kudo, M., and Sklansky, J., 2000, Comparison of algorithms that selects features for pattern classifiers. Pattern Recognition, 33, 25-41.

Laird, N. M., Ware, J. H., 1982. Random-effects models for longitudinal data. Biometrics, 38, 963-974.
Langley, P., Zytkow, J. M., Simon, H. A. and Bradshaw, G. L., 1986, The Search for Regularity: four aspects of scientific discovery, Machine Learning: An Artificial Intelligence Approach, 2, Morgan Kaufman, Los Altos, CA, 425-470.
Li, D. C., and Liu, C. W., 2009, A Class Possibility Based Kernel to Increase Classification Accuracy for Small Data Sets Using Support Vector Machines. Expert systems with applications, in Press.
Li, D. C., Wu, C. S., Tsai, T. I, and Lina, Y. S., 2007, Using mega-trend-diffusion and artifical samples in small data set learning for early flexible manufacturing system scheduling knowledge. Computers and Operations Research, 34, 966-982.
Li, D. C., Hsu, H. C., Tsai, T. I., Lu, T. J., and Hu, S. C., 2007, A new method to help diagnose cancers for small sample size. Expert Systems with Applications, 33, 420-424.
Li, D. C., and Lin, Y. S., 2006, Using virtual sample generation to build up management knowledge in the early manufacturing stages. European Journal of Operational Research, 175, 413-434.
Li, D. C., and Yeh, C. W., 2008, A non-parametric learning algorithm for small manufacturing data sets. Expert Systems with Applications, 34, 391-398.
Li, D. C., and Yeh, C. W., A novel forecasting method for small data sets, under review at Expert Systems with Applications.
Liu, H. and Motoda, H., 1998b, Feature extraction, construction and selection: a data mining perspective, Kluwer Academic Publisher, Norwell, MA.


Liu, J., and MachCarthy, B. L., 1999, General heuristic procedures and solution strategies for FMS scheduling. International Journal of Production Research, 37(14), 3305-3333.
Luukka P., 2008, Similarity classifier in diagnosis of bladder cancer. Computer Methods and Programs in Biomedicine ;89:43-49.
Matheus, C. J., and Rendell, L., 1989, Constructive Induction in decision trees, Proc. Eleventh IJCAI, Morgan-Kaufman, San, Mateo, CA, 645-650.
Mercer, J., 1909 Functions of positive and negative type, and their connection with the theory of integral equations. Transactions of the London Philosophical Society (A), 209, 415-446.
Mika, S., Schölkopf, B., Smola, A. J., MJuller, K. –R., Scholz, M., and RJatsch, G., 1999, KPCA and de-noising in feature spaces. Advances in Neural Information Processing Systems, 11, 536-542.
Mitra, P., 2002, Unsupervised feature selection using feature similarity. IEEE transactions on pattern analysis and machine intelligence. 24(3), 301-312.
Muharram, M. A., Smith, G. D., 2004, Evolutionary feature construction using information gain and gini index. Keijzer, M., O’Reilly, U.M., Lucas, S. M., Costa, E., Soule, T. EuroGP 2004. LNCS, 3003, 379-388. Springer, Heidelberg.
Niyoqi, P., Girosi, F., Tomaso, P., 1998. Incorporating prior information in machine learning by creating virtual examples. Proceeding of the IEEE, 275-298.
Neshatian, K., Zhang, M., and Johnston, M., 2007. Feature construction and dimension reduction using genetic programming. Springer-Verlag Berlin Heidelberg, 160-170.
Otero, F. E. B., Silva, M. M. S., Freitas, A. A., Nievola, J. C., 2003, Genetic programming for attribute construction in data mining. EuroGP 2003 LNCS, 2610, 384-393. Springer, Heidelberg.
Pagallo, G., 1989, Learning DNF by decision trees. Proc. Eleventh IJCAI, Moargan-Kaufman, San Mateo, CA, 639-644.
Pal, S. K., De, R. K. and Basak, J., 2000, Unsupervised feature evaluation: a neuro-fuzzy approach. IEEE Transaction on Neural Network, 11, 366-376.
Piramuthu, S., Ragavan, H., and Shaw, M., 1998, Using feature construction to improve the performance of neural networks. Management Science, 44(3), 416-430.
Rama Bhupal Reddy, K., Na Xie, and Velusamy Subramaniam, 2004, Dynamic scheduling of flexible manufacturing systems. Innovation in Manufacturing Systems and Technology (IMST);URI: http://hd1.handle.net/1721.1/3903.
Rau, K. R., and Chetty, O. V. K., 1996, Production planning of FMS under tool magazine constraints: a dynamic programming approach. International Journal of Advanced Manufacturing Technology, 11, 366-371.
Reyes, A., Yu, H., Kelleher, G., and Lloyd, S., 2002, Integrating Petri nets and hybrid heuristic search for the scheduling of FMS. Computers in Industry, 47, 123-138.
Romdhani, S., Gong, S., and Psarrou, A., 1999, A multi-view nonlinear active shape model using KPCA. In Proceedings of BMVC (483-492), Nottingham, UK.
Rudin, W., 1973, Functional analysis. New York, McGraw-Hill.
Sabuncuoglu, I., 1998, A study of scheduling rules of flexible manufacturing systems: a simulation approach. International Journal of Production Research, 36(2), 527-546.
Schwarz, G., 1978, Estimating the dimensions of a model. Annals of Statistics, 6, 461-464.
Schőlkopf, B., Mika, S., Burges, C. J. C., Knirsch, P., MJuller, K. –R., RJatsch, G., et al., 1999, Input space versus feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5), 1000-1016.
Schőlkopf, B., Smola, A., and Műller, K. –R., 1998, Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5), 1299-1319.
Schölkopf, B., Sung, K. –K., Chris, J. C., Girosi, F., Niyogi, P., Poggio, T., and Vapnik, V., 1997, Comparing Support Vector Machines with Gaussian Kernel to Radial Basis Function Classifiers. IEEE Trans. Signal Processing, 45(11), 2758-2765.
Schölkopf, B., Tsuda, K., Vert, J.-P., 2004, Kernel Methods in Computational Biology, MIT Press, New York.
Setiono R., 2000, Generating concise and accurate classification rules for breast cancer diagnosis. Artificial Intelligence in Medicine ;18: 205-219.
Shawkat, A., Kate, A., Smith, M., 2006, A meta-learning approach to Automatic Kernel Selection for Support Vector Machines. Neurocomputing, 70, 173-186.
Smith, M. G., Bull, L., 2005, Genetic programming with a genetic algorithm for feature construction and selection. Genetic Programming and Evolvable Machines, 6(3), 265-281.
Srebro, N., 2007, How Good is a Kernel When Used a Similarity Measure? Proc. 20th Annual Conference on Learning Theory, 4539, 1611-3349.
Takahashi, N., and Nishi, T., 2006, Global convergence of decomposition learning methods for support vector machines. IEEE Trans. Neural Network, 17, 1362-1369.
Tingting M, Asoke KN, Rangaraj MR., 2007, Classification of breast masses via nonlinear transformation of features based on a kernel matrix. Medical and Biological Engineering and Computing 45(8):769-780.
V. D. Sánchez A., 2003, Advanced support vector machines and kernel methods. Neurocomputing, 55, 5-20.
Vafaie, H., and DeJong, 1998, Feature space transformation using genetic algorithms. IEEE Intelligent systems, 13(2), 57-65.
Weston, J., Elisseff, A., Schoelkoph, B., and Tipping, M., 2003, Use of the zero norm with linear models and kernel methods. JMLR, 3, 1439-1461.
William H, Wolberg MD, Ellen P, Romsaas MS, Martin A, Tanner PhD, James F, Malec, 1989 PhD. Psychosexual adaptation to breast cancer surgery. Cancer ;63(8):1645-1655.
Wu, S., and Amari, S., 2002, Conformal Transformation of Kernel Functions: A Data-Dependent Way to Improve Support Vector Machine Classifiers. Neural Processing Letters, 15, 59-67.
Yang, D. S., Rendell, L., and Glix, G., 1991, A scheme for feature construction and a comparison of empirical methods. Proc. Twelfth IJCAI, Sydey, Australia, Morgan-Kaufman, San Mateo, CA, 699-704.
Yu, L., Shih, H. M., and Sekiguchi, T., 1999, Fuzzy inference-based multiple criteria FMS scheduling. International Journal of Production Research, 37(10), 2315-2333
Zadeh, L., 1978, Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems, 1, 3-28.
 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top
QR Code
QRCODE