Machine Learning: Origins, Developments and Implications
First published:
ISBN: 978-1-78272-445-2
If you are a Risk.net subscriber you are entitled to 20% off your Risk books purchases. Please email enquiries@riskbooks.com for more information.
As part of your Risk.net subscription you are entitled to 20% off all of your Risk Books purchases. If you would like to place an order please email enquiries@riskbooks.com
Regularisation
Regularisation
Preface
Introduction: human-machine entanglement
Machine learning: origins
Useful tools
Decision trees
Introduction to neural networks
Back-propagation
Regularisation
Optimisation
Building neural networks
Early applications of machine learning
Interpreting neural network decisions
Predicting corporate bond returns
Deep learning networks
Applications of deep learning networks
Machine intelligence
Consciousness
The future and its challenges
Artificial intelligence and the military
Final thoughts
Appendix
Epilogue
Acknowledgements
Regularisation refers to techniques used to calibrate machine learning models in order to minimise the error function while preventing overfitting or underfitting. Overfitting is said to occur when a model learns both the detail and noise in the training data, negatively impacting the performance of the model on new data. Regularisation techniques adjust model learning algorithms during training such that the model generalises better. Regularisation is “any modification we make to a learning algorithm that is intended to reduce its generalization error, but not its training error.” (Goodfellow et al 2016)
7.1 REGULARISATION, OPTIMISATION AND DEEP LEARNING
Consider a neural network that is overfitting on the training data, as shown in Figure 7.1. This model fits the training data well, but will not generalise to cases not used for training. Neural networks are notoriously tricky to optimise. We cannot compute a global optimum for weight parameters nor guarantee convergence to a global optimum. We must seek acceptable solutions while trying to minimise overfitting to the data.
Popular regularisation methods include: L1 and L2 regulation; dropout; data augmentation; generative
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net
