Curtin University Homepage
  • Library
  • Help
    • Admin

    espace - Curtin’s institutional repository

    JavaScript is disabled for your browser. Some features of this site may not work without it.
    View Item 
    • espace Home
    • espace
    • Curtin Research Publications
    • View Item
    • espace Home
    • espace
    • Curtin Research Publications
    • View Item

    Anomaly detection in large-scale data stream networks

    189224_189224.pdf (653.5Kb)
    Access Status
    Open access
    Authors
    Pham, DucSon
    Venkatesh, S.
    Lazarescu, Mihai
    Budhaditya, S.
    Date
    2014
    Type
    Journal Article
    
    Metadata
    Show full item record
    Citation
    Pham, Duc-Son and Venkatesh, Svetha and Lazarescu, Mihai and Budhaditya, Saha. 2014. Anomaly detection in large-scale data stream networks. Data Mining Knowledge & Discovery. 28 (1): pp. 145-189.
    Source Title
    Data Mining Knowledge & Discovery
    DOI
    10.1007/s10618-012-0297-3
    ISSN
    1573-756X
    Remarks

    The final publication is available at Springer via http://dx.doi.org/10.1007/s10618-012-0297-3

    URI
    http://hdl.handle.net/20.500.11937/43829
    Collection
    • Curtin Research Publications
    Abstract

    This paper addresses the anomaly detection problem in large-scale data mining applications using residual subspace analysis. We are specifically concerned with situations where the full data cannot be practically obtained due to physical limitations such as low bandwidth, limited memory, storage, or computing power. Motivated by the recent compressed sensing (CS) theory, we suggest a framework wherein random projection can be used to obtained compressed data, addressing the scalability challenge. Our theoretical contribution shows that the spectral property of the CS data is approximately preserved under a such a projection and thus the performance of spectral-based methods for anomaly detection is almost equivalent to the case in which the raw data is completely available. Our second contribution is the construction of the framework to use this result and detect anomalies in the compressed data directly, thus circumventing the problems of data acquisition in large sensor networks. We have conducted extensive experiments to detect anomalies in network and surveillance applications on large datasets, including the benchmark PETS 2007 and 83 GB of real footage from three public train stations. Our results show that our proposed method is scalable, and importantly, its performance is comparable to conventional methods for anomaly detection when the complete data is available.

    Related items

    Showing items related by title, author, creator and subject.

    • Effective Anomaly detection in Sensor Network Data Streams
      Pham, DucSon; Saha , Budhaditya; Lazarescu, Mihai; Venkatesh, Svetha (2009)
      This paper addresses a major challenge in datamining applications where the full information about the underlying processes, such as sensor networks or large online database, cannot be practically obtained due to physical ...
    • Detection of Cross-Channel Anomalies
      Pham, DucSon; Saha, Budhaditya; Phung, Dinh; Venkatesh, Svetha (2012)
      The data deluge has created a great challenge for data mining applications wherein the rare topics of interest are often buried in the flood of major headlines. We identify and formulate a novel problem: cross-channel ...
    • Detection of cross channel anomalies from multiple data channels
      Pham, DucSon; Saha, Budhaditya; Phung, Dinh; Venkatesh, Svetha (2011)
      We identify and formulate a novel problem: cross channel anomaly detection from multiple data channels. Cross channel anomalies are common amongst the individual channel anomalies, and are often portent of significant ...
    Advanced search

    Browse

    Communities & CollectionsIssue DateAuthorTitleSubjectDocument TypeThis CollectionIssue DateAuthorTitleSubjectDocument Type

    My Account

    Admin

    Statistics

    Most Popular ItemsStatistics by CountryMost Popular Authors

    Follow Curtin

    • 
    • 
    • 
    • 
    • 

    CRICOS Provider Code: 00301JABN: 99 143 842 569TEQSA: PRV12158

    Copyright | Disclaimer | Privacy statement | Accessibility

    Curtin would like to pay respect to the Aboriginal and Torres Strait Islander members of our community by acknowledging the traditional owners of the land on which the Perth campus is located, the Whadjuk people of the Nyungar Nation; and on our Kalgoorlie campus, the Wongutha people of the North-Eastern Goldfields.