We propose a model for conduct risk losses, in which conduct risk losses are characterized by having a small number of extremely large losses (perhaps only one) with more numerous smaller losses. It is assumed that the largest loss is actually a provision from which payments to customers are made periodically as required. We use the pseudo-marginal (PM) Markov chain Monte Carlo method to decompose the largest loss into smaller partitions in order to estimate 99.9% value-at-risk. The par-titioning is done in a way that makes no assumption about the size of the partitions. The advantages and problems of using this method are discussed. The PM procedures were run on several representative data sets. The results indicate that, in cases where using approaches such as calculating a Monte Carlo-derived loss distribution yields a result that is not consistent with the risk profile expressed by the data, using the PM method yields results that have the required consistency.