Curtin University Homepage
  • Library
  • Help
    • Admin

    espace - Curtin’s institutional repository

    JavaScript is disabled for your browser. Some features of this site may not work without it.
    View Item 
    • espace Home
    • espace
    • Curtin Research Publications
    • View Item
    • espace Home
    • espace
    • Curtin Research Publications
    • View Item

    Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression

    Access Status
    Fulltext not available
    Authors
    An, Senjian
    Liu, Wan-Quan
    Venkatesh, Svetha
    Date
    2007
    Type
    Journal Article
    
    Metadata
    Show full item record
    Citation
    An, Senjian and Liu, Wan-Quan and Venkatesh, Svetha. 2007. Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression. Pattern Recognition. 40 (8): pp. 2154-2162.
    Source Title
    Pattern Recognition
    DOI
    10.1016/j.patcog.2006.12.015
    ISSN
    00313203
    Faculty
    School of Electrical Engineering and Computing
    Department of Computing
    Faculty of Science and Engineering
    Remarks

    The link to the journal’s home page is: http://www.elsevier.com/wps/find/journaldescription.cws_home/328/description#description

    Copyright © 2007 Elsevier Ltd. All rights reserved

    URI
    http://hdl.handle.net/20.500.11937/43097
    Collection
    • Curtin Research Publications
    Abstract

    Given n training examples, the training of a least squares support vector machine (LS-SVM) or kernel ridge regression (KRR) corresponds to solving a linear system of dimension n. In cross-validating LS-SVM or KRR, the training examples are split into two distinct subsets for a number of times (l) wherein a subset of m examples are used for validation and the other subset of (n-m) examples are used for training the classifier. In this case l linear systems of dimension (n-m) need to be solved. We propose a novel method for cross-validation (CV) of LS-SVM or KRR in which instead of solving l linear systems of dimension (n-m), we compute the inverse of an n dimensional square matrix and solve l linear systems of dimension m, thereby reducing the complexity when l is large and/or m is small. Typical multi-fold, leave-one-out cross-validation (LOO-CV) and leave-many-out cross-validations are considered. For five-fold CV used in practice with five repetitions over randomly drawn slices, the proposed algorithm is approximately four times as efficient as the naive implementation. For large data sets, we propose to evaluate the CV approximately by applying the well-known incomplete Cholesky decomposition technique and the complexity of these approximate algorithms will scale linearly on the data size if the rank of the associated kernel matrix is much smaller than n. Simulations are provided to demonstrate the performance of LS-SVM and the efficiency of the proposed algorithm with comparisons to the naive and some existent implementations of multi-fold and LOO-CV.

    Related items

    Showing items related by title, author, creator and subject.

    • Unified formulation of linear discriminant analysis methods and optimal parameter selection
      An, Senjian; Liu, Wan-Quan; Venkatesh, Svetha; Hong, Y. (2011)
      In the last decade, many variants of classical linear discriminant analysis (LDA) have been developed to tackle the under-sampled problem in face recognition. However, choosing the variants is not easy since these methods ...
    • On-Line dynamic security assessment of wind farm connected power systems using a class of intelligent algorithms
      Tiako, Remy (2012)
      Recently large-scale wind farms are integrated quite commonly into power systems. The stochastic operation of wind plants due to intermittency and intra-interval effects of the wind is a problematic issue to determine the ...
    • Data-Driven Approach for Intelligent Classification of Tunnel Surrounding Rock Using Integrated Fractal and Machine Learning Methods
      Ma, Junjie; Li, Tianbin; Shirani Faradonbeh, Roohollah ; Sharifzadeh, Mostafa; Wang, Jianfeng; Huang, Yuyang; Materials, Chunchi; Peng, Feng; Zhang, Hang (2024)
      The degree of rock mass discontinuity is crucial for evaluating surrounding rock quality, yet its accurate and rapid measurement at construction sites remains challenging. This study utilizes fractal dimension to characterize ...
    Advanced search

    Browse

    Communities & CollectionsIssue DateAuthorTitleSubjectDocument TypeThis CollectionIssue DateAuthorTitleSubjectDocument Type

    My Account

    Admin

    Statistics

    Most Popular ItemsStatistics by CountryMost Popular Authors

    Follow Curtin

    • 
    • 
    • 
    • 
    • 

    CRICOS Provider Code: 00301JABN: 99 143 842 569TEQSA: PRV12158

    Copyright | Disclaimer | Privacy statement | Accessibility

    Curtin would like to pay respect to the Aboriginal and Torres Strait Islander members of our community by acknowledging the traditional owners of the land on which the Perth campus is located, the Whadjuk people of the Nyungar Nation; and on our Kalgoorlie campus, the Wongutha people of the North-Eastern Goldfields.