Investigation of activation functions in deep belief network
dc.contributor.author | Lau, M. | |
dc.contributor.author | Lim, Hann | |
dc.date.accessioned | 2017-07-27T05:21:22Z | |
dc.date.available | 2017-07-27T05:21:22Z | |
dc.date.created | 2017-07-26T11:11:17Z | |
dc.date.issued | 2017 | |
dc.identifier.citation | Lau, M. and Lim, H. 2017. Investigation of activation functions in deep belief network, pp. 201-206. | |
dc.identifier.uri | http://hdl.handle.net/20.500.11937/54534 | |
dc.identifier.doi | 10.1109/ICCRE.2017.7935070 | |
dc.description.abstract |
© 2017 IEEE. Deep Belief Network (DBN) is made up of stacked Restricted Boltzmann Machine layers associated with global weight fine-tuning for pattern recognition. However, DBN suffers from vanishing gradient problem due to the saturation characteristic of activation function. Therefore, the selection of activation function in DBN is critical to reduce the network complexity and improve performance of pattern recognition. Unsaturated activation functions such as rectified linear unit and leaky rectified linear unit were recently proposed to avoid the effect of vanishing gradient for a deep learning neural network. In this paper, we investigated the network performance with both saturated and unsaturated activation functions. Besides that, the randomization of training samples would significantly improve the performance of DBN. The experimental results showed that hyperbolic tangent activation function achieved the lowest error rate which is 1.99% on MNIST handwritten digit dataset. | |
dc.title | Investigation of activation functions in deep belief network | |
dc.type | Conference Paper | |
dcterms.source.startPage | 201 | |
dcterms.source.endPage | 206 | |
dcterms.source.title | 2017 2nd International Conference on Control and Robotics Engineering, ICCRE 2017 | |
dcterms.source.series | 2017 2nd International Conference on Control and Robotics Engineering, ICCRE 2017 | |
dcterms.source.isbn | 9781509037735 | |
curtin.department | Curtin Sarawak | |
curtin.accessStatus | Fulltext not available |
Files in this item
Files | Size | Format | View |
---|---|---|---|
There are no files associated with this item. |