用户:Atry/特征选择
机器学习与数据挖掘 |
---|
![]() |
在机器学习和统计学中,特征选择 也被称为变量选择 , 属性选择 或变量子集选择 。它是指:为了构建模型而选择相关特征(即属性、指标)子集的过程。使用特征选择技术有三个原因:
要使用特征选择技术的关键假设是:训练数据包含许多冗余 或无关 的特征,因而移除这些特征并不会导致丢失信息。[2] 冗余 或无关 特征是两个不同的概念。如果一个特征本身有用,但如果这个特征与另一个有用特征强相关,且那个特征也出现在数据中,那么这个特征可能就变得多余。[3]
特征选择技术与特征提取有所不同。特征提取是从原有特征的功能中创造新的特征,而特征选择则只返回原有特征中的子集。 特征选择技术的常常用于许多特征但样本(即数据点)相对较少的领域。特征选择应用的典型用例包括:解析书面文本和微阵列数据,这些场景下特征成千上万,但样本只有几十到几百个。
介绍
[编辑]特征选择算法可以被视为搜索技术和评价指标的结合。前者提供候选的新特征子集,后者为不同的特征子集打分。 最简单的算法是测试每个特征子集,找到究竟哪个子集的错误率最低。这种算法需要穷举搜索空间,难以算完所有的特征集,只能涵盖很少一部分特征子集。 选择何种评价指标很大程度上影响了算法。而且,通过选择不同的评价指标,可以吧特征选择算法分为三类:包装类、过滤类和嵌入类方法[3]
包装类方法使用预测模型给特征子集打分。每个新子集都被用来训练一个模型,然后用验证数据集来测试。通过计算验证数据集上的错误次数(即模型的错误率)给特征子集评分。由于包装类方法为每个特征子集训练一个新模型,所以计算量很大。不过,这类方法往往能为特定类型的模型找到性能最好的特征集。
过滤类方法采用代理指标,而不根据特征子集的错误率计分。所选的指标算得快,但仍然能估算出特征集好不好用。常用指标包括互信息、[3]逐点互信息,[4] 皮尔逊积矩相关系数、每种分类/特征的组合的帧间/帧内类距离或显著性测试评分。[4][5]
过滤类方法计算量一般比包装类小,但这类方法找到的特征子集不能为特定类型的预测模型调校。由于缺少调校,过滤类方法所选取的特征集会比包装类选取的特征集更为通用,往往会导致比包装类的预测性能更为低下。不过,由于特征集不包含对预测模型的假设,更有利于暴露特征之间的关系。许多过滤类方法提供特征排名,而非显式提供特征子集。要从特征列表的哪个点切掉特征,得靠交叉验证来决定。过滤类方法也常常用于包装方法的预处理步骤,以便在问题太复杂时依然可以用包装方法。
嵌入类方法包括了所有构建模型过程中庸道德特征选择技术。这类方法的典范是构建线性模型的LASSO方法。该方法给回归系数加入了L1惩罚,导致其中的许多参数趋于零。任何回归系数不为零的特征都会被LASSO算法“选中”。LASSO的改良算法有Bolasso[6]和FeaLect[7]。Bolasso改进了样本的初始过程。FeaLect根据回归系数组合分析给所有特征打分。 另外一个流行的做法是递归特征消除(Recursive Feature Elimination)算法,通常用于支持向量机,通过反复构建同一个模型移除低权重的特征。这些方法的计算复杂度往往在过滤类和包装类之间。
传统的统计学中,特征选择的最普遍的形式是逐步回归,这是一个包装类技术。它属于贪心算法,每一轮添加该轮最优的特征或者删除最差的特征。主要的调控因素是决定何时停止算法。在机器学习领域,这个时间点通常通过交叉验证找出。在统计学中,某些条件已经优化。因而会导致嵌套引发问题。此外,还有更健壮的方法,如分支和约束和分段线性网络。
Subset selection
[编辑]Subset selection evaluates a subset of features as a group for suitability. Subset selection algorithms can be broken up into Wrappers, Filters and Embedded. Wrappers use a search algorithm to search through the space of possible features and evaluate each subset by running a model on the subset. Wrappers can be computationally expensive and have a risk of over fitting to the model. Filters are similar to Wrappers in the search approach, but instead of evaluating against a model, a simpler filter is evaluated. Embedded techniques are embedded in and specific to a model.
Many popular search approaches use greedy hill climbing, which iteratively evaluates a candidate subset of features, then modifies the subset and evaluates if the new subset is an improvement over the old. Evaluation of the subsets requires a scoring metric that grades a subset of features. Exhaustive search is generally impractical, so at some implementor (or operator) defined stopping point, the subset of features with the highest score discovered up to that point is selected as the satisfactory feature subset. The stopping criterion varies by algorithm; possible criteria include: a subset score exceeds a threshold, a program's maximum allowed run time has been surpassed, etc.
Alternative search-based techniques are based on targeted projection pursuit which finds low-dimensional projections of the data that score highly: the features that have the largest projections in the lower-dimensional space are then selected.
Search approaches include:
- Exhaustive
- Best first
- Simulated annealing
- 这些理论包括:
- Greedy forward selection
- Greedy backward elimination[8][9][10]
- Particle swarm optimization[11]
- Targeted projection pursuit
- Scatter Search[12]
- Variable Neighborhood Search[13][14]
Two popular filter metrics for classification problems are correlation and mutual information, although neither are true metrics or 'distance measures' in the mathematical sense, since they fail to obey the triangle inequality and thus do not compute any actual 'distance' – they should rather be regarded as 'scores'. These scores are computed between a candidate feature (or set of features) and the desired output category. There are, however, true metrics that are a simple function of the mutual information;[15] see here.
Other available filter metrics include:
- Class separability
- Error probability
- Inter-class distance
- Probabilistic distance
- Entropy
- Consistency-based feature selection
- Correlation-based feature selection
Optimality criteria
[编辑]The choice of optimality criteria is difficult as there are multiple objectives in a feature selection task. Many common ones incorporate a measure of accuracy, penalised by the number of features selected (e.g. the Bayesian information criterion). The oldest are Mallows's Cp statistic and Akaike information criterion (AIC). These add variables if the t -statistic is bigger than .
Other criteria are Bayesian information criterion (BIC) which uses , minimum description length (MDL) which asymptotically uses , Bonferroni / RIC which use , maximum dependency feature selection, and a variety of new criteria that are motivated by false discovery rate (FDR) which use something close to .
Structure Learning
[编辑]Filter feature selection is a specific case of a more general paradigm called Structure Learning. Feature selection finds the relevant feature set for a specific target variable whereas structure learning finds the relationships between all the variables, usually by expressing these relationships as a graph. The most common structure learning algorithms assume the data is generated by a Bayesian Network, and so the structure is a directed graphical model. The optimal solution to the filter feature selection problem is the Markov blanket of the target node, and in a Bayesian Network, there is a unique Markov Blanket for each node.[16]
Minimum-redundancy-maximum-relevance (mRMR) feature selection
[编辑]Peng et al. [17] proposed a feature selection method that can use either mutual information, correlation, or distance/similarity scores to select features. The aim is to penalise a feature's relevancy by its redundancy in the presence of the other selected features. The relevance of a feature set S for the class c is defined by the average value of all mutual information values between the individual feature fi and the class c as follows:
- .
The redundancy of all features in the set S is the average value of all mutual information values between the feature fi and the feature fj:
The mRMR criterion is a combination of two measures given above and is defined as follows:
Suppose that there are n full-set features. Let xi be the set membership indicator function for feature fi, so that xi=1 indicates presence and xi=0 indicates absence of the feature fi in the globally optimal feature set. Let and . The above may then be written as an optimization problem:
The mRMR algorithm is an approximation of the theoretically optimal maximum-dependency feature selection algorithm that maximizes the mutual information between the joint distribution of the selected features and the classification variable. As mRMR approximates the combinatorial estimation problem with a series of much smaller problems, each of which only involves two variables, it thus uses pairwise joint probabilities which are more robust. In certain situations the algorithm may underestimate the usefulness of features as it has no way to measure interactions between features which can increase relevancy. This can lead to poor performance[18] when the features are individually useless, but are useful when combined (a pathological case is found when the class is a parity function of the features). Overall the algorithm is more efficient (in terms of the amount of data required) than the theoretically optimal max-dependency selection, yet produces a feature set with little pairwise redundancy.
mRMR is an instance of a large class of filter methods which trade off between relevancy and redundancy in different ways.[18][19]
Global optimization formulations
[编辑]mRMR is a typical example of an incremental greedy strategy for feature selection: once a feature has been selected, it cannot be deselected at a later stage. While mRMR could be optimized using floating search to reduce some features, it might also be reformulated as a global quadratic programming optimization problem as follows:[20]
where is the vector of feature relevancy assuming there are n features in total, is the matrix of feature pairwise redundancy, and represents relative feature weights. QPFS is solved via quadratic programming. It is recently shown that QFPS is biased towards features with smaller entropy,[21] due to its placement of the feature self redundancy term on the diagonal of H.
Another global formulation for the mutual information based feature selection problem is based on the conditional relevancy:[21]
where and .
An advantage of SPECCMI is that it can be solved simply via finding the dominant eigenvector of Q, thus is very scalable. SPECCMI also handles second-order feature interaction.
For high-dimensional and small sample data (e.g., dimensionality > 10 and the number of samples < 10 5), the Hilbert-Schmidt Independence Criterion Lasso (HSIC Lasso) is useful. 3[22] HSIC Lasso optimization problem is given as
where is a kernel-based independence measure called the (empirical) Hilbert-Schmidt independence criterion (HSIC), denotes the trace, is the regularization parameter, and are input and output centered Gram matrices, and are Gram matrices, and are kernel functions, is the centering matrix, is the m-dimensional identity matrix (m: the number of samples), is the m-dimensional vector with all ones, and is the -norm. HSIC always takes a non-negative value, and is zero if and only if two random variables are statistically independent when a universal reproducing kernel such as the Gaussian kernel is used.
The HSIC Lasso can be written as
where is the Frobenius norm. The optimization problem is a Lasso problem, and thus it can be efficiently solved with a state-of-the-art Lasso solver such as the dual augmented Lagrangian method.
Correlation feature selection
[编辑]The Correlation Feature Selection (CFS) measure evaluates subsets of features on the basis of the following hypothesis: "Good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other".[23][24] The following equation gives the merit of a feature subset S consisting of k features:
Here, is the average value of all feature-classification correlations, and is the average value of all feature-feature correlations. The CFS criterion is defined as follows:
The and variables are referred to as correlations, but are not necessarily Pearson's correlation coefficient or Spearman's ρ. Dr. Mark Hall's dissertation uses neither of these, but uses three different measures of relatedness, minimum description length (MDL), symmetrical uncertainty, and relief.
Let xi be the set membership indicator function for feature fi ; then the above can be rewritten as an optimization problem:
The combinatorial problems above are, in fact, mixed 0–1 linear programming problems that can be solved by using branch-and-bound algorithms.[25]
Regularized trees
[编辑]The features from a decision tree or a tree ensemble are shown to be redundant. A recent method called regularized tree[26] can be used for feature subset selection. Regularized trees penalize using a variable similar to the variables selected at previous tree nodes for splitting the current node. Regularized trees only need build one tree model (or one tree ensemble model) and thus are computationally efficient.
Regularized trees naturally handle numerical and categorical features, interactions and nonlinearities. They are invariant to attribute scales (units) and insensitive to outliers, and thus, require little data preprocessing such as normalization. Regularized random forest (RRF)[27] is one type of regularized trees. The guided RRF is an enhanced RRF which is guided by the importance scores from an ordinary random forest.
Overview on metaheuristics methods
[编辑]A metaheuristic is a general description of an algorithm dedicated to solve difficult (typically NP-hard problem) optimization problems for which there is no classical solving methods. Generally, a metaheuristic is a stochastics algorithm tending to reach a global optima. There are many metaheuristics, from a simple local search to a complex global search algorithm.
Main principles
[编辑]The feature selection methods are typically presented in three classes based on how they combine the selection algorithm and the model building.
Filter Method
[编辑]![](http://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Filter_Methode.png/300px-Filter_Methode.png)
Filter-based feature selection has become crucial in many classification settings, especially object recognition, recently faced with feature learning strategies that originate thousands of cues.[28] Filter methods analyze intrinsic properties of data, ignoring the classifier. Most of these methods can perform two operations, ranking and subset selection: in the former, the importance of each individual feature is evaluated, usually by neglecting potential interactions among the elements of the joint set; in the latter, the final subset of features to be selected is provided. In some cases, these two operations are performed sequentially (first the ranking, then the selection); in other cases, only the selection is carried out.[28] Filter methods suppress the least interesting variables. These methods are particularly effective in computation time and robust to overfitting.[29]
However, filter methods tend to select redundant variables because they do not consider the relationships between variables. Therefore, they are mainly used as a pre-process method.
Wrapper Method
[编辑]![](http://upload.wikimedia.org/wikipedia/commons/thumb/0/04/Feature_selection_Wrapper_Method.png/300px-Feature_selection_Wrapper_Method.png)
Wrapper methods evaluate subsets of variables which allows, unlike filter approaches, to detect the possible interactions between variables.[30] The two main disadvantages of these methods are :
- The increasing overfitting risk when the number of observations is insufficient.
- The significant computation time when the number of variables is large.
Embedded Method
[编辑]![](http://upload.wikimedia.org/wikipedia/commons/thumb/b/bf/Feature_selection_Embedded_Method.png/300px-Feature_selection_Embedded_Method.png)
Recently, embedded methods have been proposed to reduce the classification of learning. They try to combine the advantages of both previous methods. The learning algorithm takes advantage of its own variable selection algorithm. So, it needs to know preliminary what a good selection is, which limits their exploitation.[31]
Application of feature selection metaheuristics
[编辑]This is a survey of the application of feature selection metaheuristics lately used in the literature. This survey was realized by J. Hammon in her thesis.[29]
应用领域 | 算法 | 我们的工作方式 | classifier | Evaluation Function | {0}参考号{1}:016596{/1}{/0} |
---|---|---|---|---|---|
SNPs | Feature Selection using Feature Similarity | 过滤 | ŕ | Phuong 2005 [30] | |
SNPs | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | Decision Tree | Classification accuracy (10-fold) | Shah 2004 [32] |
SNPs | HillClimbing | Filter + Wrapper | Naive Bayesian | Predicted residual sum of squares | 长 |
SNPs | Simulated Annealing | Naive bayesian | Classification accuracy (5-fold) | Ustunkar 2011 [33] | |
Segments parole | Ants colony | {0}wrapper{/0}{1}.{/1} | Artificial Neural Network | MSE | Al-ani 2005 [来源请求] |
营销 | Simulated Annealing | {0}wrapper{/0}{1}.{/1} | 回归 | AIC, r2 | Meiri 2006 [34] |
经济 | Simulated Annealing, Genetic Algorithm | {0}wrapper{/0}{1}.{/1} | 回归 | BIC | Kapetanios 2005 [35] |
Spectral Mass | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | Multiple Linear Regression, Partial Least Squares | root-mean-square error of prediction | Broadhurst 2007 [36] |
Microarray | Tabu Search + PSO | {0}wrapper{/0}{1}.{/1} | Support Vector Machine, K Nearest Neighbors | Euclidean Distance | Chuang 2009 [37] |
Microarray | PSO + Genetic Algorithm | {0}wrapper{/0}{1}.{/1} | Support Vector Machine | Classification accuracy (10-fold) | Alba 2007 [38] |
Microarray | Genetic Algorithm + Iterated Local Search | EMBEDDED | Support Vector Machine | Classification accuracy (10-fold) | Duval 2009 [31] |
Microarray | Iterated Local Search | {0}wrapper{/0}{1}.{/1} | 回归 | Posterior Probability | Hans 2007 [39] |
Microarray | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | K Nearest Neighbors | Classification accuracy (Leave-one-out cross-validation) | Jirapech-Umpai 2005 [40] |
Microarray | Hybrid Genetic Algorithm | {0}wrapper{/0}{1}.{/1} | K Nearest Neighbors | Classification accuracy (Leave-one-out cross-validation) | 我 |
Microarray | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | Support Vector Machine | Sensitivity and specificity | Xuan 2011 [41] |
Microarray | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | All paired Support Vector Machine | Classification accuracy (Leave-one-out cross-validation) | Peng 2003 [42] |
Microarray | 这些理论包括: | EMBEDDED | Support Vector Machine | Classification accuracy (10-fold) | Hernandez 2007 [43] |
Microarray | 这些理论包括: | 混合式 | Support Vector Machine | Classification accuracy (Leave-one-out cross-validation) | Huerta 2006 [44] |
Microarray | 这些理论包括: | Support Vector Machine | Classification accuracy (10-fold) | Muni 2006 [45] | |
Microarray | 这些理论包括: | {0}wrapper{/0}{1}.{/1} | Support Vector Machine | EH-DIALL, CLUMP | Jourdan 2004 [46] |
Object Recognition | Infinite Feature Selection | 过滤 | Support Vector Machine | Mean Average Precision (mAP) | Roffo 2015 [28] |
Feature selection embedded in learning algorithms
[编辑]Some learning algorithms perform feature selection as part of their overall operation. 这些心理素质包括:
- -regularization techniques, such as sparse regression, LASSO, and -SVM
- Regularized trees,[26] e.g. regularized random forest implemented in the RRF package[27]
- Decision tree[来源请求]
- Memetic algorithm
- Random multinomial logit (RMNL)
- Auto-encoding networks with a bottleneck-layer
- Submodular feature selection[47][48][49]
参见
[编辑]参考文献
[编辑]- ^ 1.0 1.1 Gareth James; Daniela Witten; Trevor Hastie; Robert Tibshirani. An Introduction to Statistical Learning. Springer. 2013: 204.
- ^ 2.0 2.1 Bermingham, Mairead L.; Pong-Wong, Ricardo; Spiliopoulou, Athina; Hayward, Caroline; Rudan, Igor; Campbell, Harry; Wright, Alan F.; Wilson, James F.; Agakov, Felix; Navarro, Pau; Haley, Chris S. Application of high-dimensional feature selection: evaluation for genomic prediction in man. Sci. Rep. 2015, 5.
- ^ 3.0 3.1 3.2 Guyon, Isabelle; Elisseeff, André. An Introduction to Variable and Feature Selection. JMLR. 2003, 3.
- ^ 4.0 4.1 Yang, Yiming; Pedersen, Jan O. A comparative study on feature selection in text categorization. ICML. 1997.
- ^ Forman, George. An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research. 2003, 3: 1289–1305.
- ^ Bach, Francis R. Bolasso: model consistent lasso estimation through the bootstrap. Proceedings of the 25th international conference on Machine learning. 2008: 33–40. doi:10.1145/1390156.1390161.
- ^ Zare, Habil. Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis. BMC Genomics. 2013, 14: S14. doi:10.1186/1471-2164-14-S1-S14.
- ^ Figueroa, Alejandro. Exploring effective features for recognizing the user intent behind web queries. Computers in Industry. 2015, 68: 162–169. doi:10.1016/j.compind.2015.01.005.
- ^ Figueroa, Alejandro; Guenter Neumann. Learning to Rank Effective Paraphrases from Query Logs for Community Question Answering. AAAI. 2013.
- ^ Figueroa, Alejandro; Guenter Neumann. Category-specific models for ranking effective paraphrases in community Question Answering. Expert Systems with Applications. 2014, 41: 4730–4742. doi:10.1016/j.eswa.2014.02.004.
- ^ Zhang, Y.; Wang, S.; Phillips, P. Binary PSO with Mutation Operator for Feature Selection using Decision Tree applied to Spam Detection. Knowledge-Based Systems. 2014, 64: 22–31. doi:10.1016/j.knosys.2014.03.015.
- ^ F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. Solving feature subset selection problem by a Parallel Scatter Search, European Journal of Operational Research , vol. 169, no. 2, pp. (2006年).
- ^ F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. Solving Feature Subset Selection Problem by a Hybrid Metaheuristic. In First International Workshop on Hybrid Metaheuristics , pp. 59–68, 2004.
- ^ M. Garcia-Torres, F. Gomez-Vela, B. Melian, J.M. Moreno-Vega. High-dimensional feature selection via feature grouping: A Variable Neighborhood Search approach, Information Sciences , vol. 326, pp. {0}8.{/0} {1}{/1}
- ^ Alexander Kraskov, Harald Stögbauer, Ralph G. Andrzejak, and Peter Grassberger, "Hierarchical Clustering Based on Mutual Information", (2003) ArXiv q-bio/0311039
- ^ Aliferis, Constantin. Local causal and markov blanket induction for causal discovery and feature selection for classification part I: Algorithms and empirical evaluation (PDF). Journal of Machine Learning Research. 2010, 11: 171–234.
- ^ Peng, H. C.; Long, F.; Ding, C. Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2005, 27 (8): 1226–1238. PMID 16119262. doi:10.1109/TPAMI.2005.159.教学大纲
- ^ 18.0 18.1 Brown, G., Pocock, A., Zhao, M.-J., Lujan, M. (2012). "Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection", In the Journal of Machine Learning Research (JMLR). [1]
- ^ Nguyen, H., Franke, K., Petrovic, S. (2010). "Towards a Generic Feature-Selection Measure for Intrusion Detection", In Proc. International Conference on Pattern Recognition (ICPR), Istanbul, Turkey. [2]
- ^ Rodriguez-Lujan, I.; Huerta, R.; Elkan, C.; Santa Cruz, C. Quadratic programming feature selection (PDF). JMLR. 2010, 11: 1491–1516.
- ^ 21.0 21.1 Nguyen X. Vinh, Jeffrey Chan, Simone Romano and James Bailey, "Effective Global Approaches for Mutual Information based Feature Selection". Proceeedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'14), August 24–27, New York City, 2014. "[3]"
- ^ M. Yamada, W. Jitkrittum, L. Sigal, E. P. Xing, M. Sugiyama, High-Dimensional Feature Selection by Feature-Wise Non-Linear Lasso. Neural Computation, vol.26, no.1, pp.185-207, 2014.
- ^ M. Hall 1999, Correlation-based Feature Selection for Machine Learning
- ^ Senliol, Baris, et al. "Fast Correlation Based Filter (FCBF) with a different search strategy." Computer and Information Sciences, 2008. ISCIS'08. 23rd International Symposium on. 978-1-4244-2108-4/08 /©2008电气与电子工程师协会[4]
- ^ Hai Nguyen, Katrin Franke, and Slobodan Petrovic, Optimizing a class of feature selection measures, Proceedings of the NIPS 2009 Workshop on Discrete Optimization in Machine Learning: Submodularity, Sparsity & Polyhedra (DISCML), Vancouver, Canada, December 2009. [5]
- ^ 26.0 26.1 H. Deng, G. Runger, "Feature Selection via Regularized Trees", Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), IEEE, 2012
- ^ 27.0 27.1 RRF: Regularized Random Forest, R package on CRAN
- ^ 28.0 28.1 28.2 Roffo, Giorgio; Melzi, Simone; Cristani, Marco. Infinite Feature Selection. www.cv-foundation.org. International Conference on Computer Vision. 2015 [2016-01-25].
- ^ 29.0 29.1 J. Hammon. Optimisation combinatoire pour la sélection de variables en régression en grande dimension : Application en génétique animale. 2014年11月10日
- ^ 30.0 30.1 T. M. Phuong, Z. Lin et R. B. Altman. Choosing SNPs using feature selection. Proceedings / IEEE Computational Systems Bioinformatics Conference, CSB. IEEE Computational Systems Bioinformatics Conference, pages 301-309, 2005. PMID:
- ^ 31.0 31.1 B. Duval, J.-K. Hao et J. C. Hernandez Hernandez. A memetic algorithm for gene selection and molecular classification of an cancer. In Proceedings of the 11th Annual conference on Genetic and evolutionary computation, GECCO '09, pages 201-208, New York, NY, USA, 2009. (u A/cm2)
- ^ S. C. Shah et A. Kusiak. Data mining and genetic algorithm based gene/SNP selection. Artificial intelligence in medicine, vol. 31, no. 3, pages 183-196, July 2004. PMID:
- ^ G. Ustunkar, S. Ozogur-Akyuz, G. W. Weber, C. M. Friedrich et Yesim Aydin Son. Selection of representative SNP sets for genome-wide association studies : a metaheuristic approach. Optimization Letters, November 2011.
- ^ R. Meiri et J. Zahavi. Using simulated annealing to optimize the feature selection problem in marketing applications. European Journal of Operational Research, vol. 171, no. 3, pages 842-858, Juin 2006
- ^ G. Kapetanios. Variable Selection using Non-Standard Optimisation of Information Criteria. Working Paper 533, Queen Mary, University of London, School of Economics and Finance, 2005.
- ^ D. Broadhurst, R. Goodacre, A. Jones, J. J. Rowland et D. B. Kell. Genetic algorithms as a method for variable selection in multiple linear regression and partial least squares regression, with applications to pyrolysis mass spectrometry. Analytica Chimica Acta, vol. 348, no. 1-3, pages 71-86, August 1997.
- ^ Chuang, L.-Y.; Yang, C.-H. Tabu search and binary particle swarm optimization for feature selection using microarray data. Journal of computational biology. 2009, 16 (12): 1689–1703. PMID 20047491. doi:10.1089/cmb.2007.0211.
- ^ E. Alba, J. Garia-Nieto, L. Jourdan et E.-G. Talbi. Gene Selection in Cancer Classification using PSO-SVM and GA-SVM Hybrid Algorithms. Congress on Evolutionary Computation, Singapor : Singapore (2007), 2007
- ^ C. Hans, A. Dobra et M. West. Shotgun stochastic search for 'large p' regression. Journal of the American Statistical Association, 2007.
- ^ T. Jirapech-Umpai et S. Aitken. Feature selection and classification for microarray data analysis : Evolutionary methods for identifying predictive genes. BMC bioinformatics, vol. 6, no. 1, page 148, 2005.
- ^ Xuan, P.; Guo, M. Z.; Wang, J.; Liu, X. Y.; Liu, Y. Genetic algorithm-based efficient feature selection for classification of pre-miRNAs. Genetics and Molecular Research. 2011, 10 (2): 588–603. PMID 21491369. doi:10.4238/vol10-2gmr969.
- ^ S. Peng. Molecular classification of cancer types from microarray data using the combination of genetic algorithms and support vector machines. FEBS Letters, vol. 555, no. 2, pages 358-362, December 2003.
- ^ J. C. H. Hernandez, B. Duval et J.-K. Hao. A genetic embedded approach for gene selection and classification of microarray data. In Proceedings of the 5th European conference on Evolutionary computation, machine learning and data mining in bioinformatics, EvoBIO'07, pages 90-101, Berlin, Heidelberg, 2007. SpringerVerlag.
- ^ E. B. Huerta, B. Duval et J.-K. Hao. A hybrid GA/SVM approach for gene selection and classification of microarray data. evoworkshops 2006, LNCS, vol. 3907, pages 34-44, 2006.
- ^ D. P. Muni, N. R. Pal et J. Das. Genetic programming for simultaneous feature selection and classifier design. IEEE Transactions on Systems, Man, and Cybernetics, Part B : Cybernetics, vol. 36, no. 1, pages 106-117, February 2006.
- ^ L. Jourdan, C. Dhaenens et E.-G. Talbi. Linkage disequilibrium study with a parallel adaptive GA. International Journal of Foundations of Computer Science, 2004.
- ^ Das el al, Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection
- ^ Liu et al, Submodular feature selection for high-dimensional acoustic score spaces
- ^ Zheng et al, Submodular Attribute Selection for Action Recognition in Video
扩展阅读
[编辑]- Feature Selection for Classification: A Review (Survey,2014)
- Feature Selection for Clustering: A Review (Survey,2013)
- Tutorial Outlining Feature Selection Algorithms, Arizona State University
- JMLR Special Issue on Variable and Feature Selection
- Feature Selection for Knowledge Discovery and Data Mining (Book)
- An Introduction to Variable and Feature Selection (Survey)
- Toward integrating feature selection algorithms for classification and clustering (Survey)
- feature subset selection and subset size optimization.pdf Efficient Feature Subset Selection and Subset Size Optimization (Survey, 2010)
- Searching for Interacting Features
- Feature Subset Selection Bias for Classification Learning
- Y. Sun, S. Todorovic, S. Goodison, Local Learning Based Feature Selection for High-dimensional Data Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 32, no. 9, pp. {0}8.{/0} {1}{/1}
外部链接
[编辑]- A comprehensive package for Mutual Information based feature selection in Matlab
- Infinite Feature Selection (Source Code) in Matlab
- Feature Selection Package, Arizona State University (Matlab Code)
- NIPS challenge 2003 (see also NIPS)
- Naive Bayes implementation with feature selection in Visual Basic (includes executable and source code)
- Minimum-redundancy-maximum-relevance (mRMR) feature selection program
- FEAST (Open source Feature Selection algorithms in C and MATLAB)
[[Category:{0}3{/0}{1} {/1}{0}模型选择{/0}]]