In this study we introduce a new approach to designing credit risk rating models for corporate entities. This approach allows a bank to make accurate and cost-effective rating decisions by maximizing risk-adjusted returns under model uncertainty. We propose a meta-algorithm, which exploits the ordinal information embedded in the expert-assigned credit ratings in order to accurately rank customers. Then, a costsensitive rating-assignment method is used to reduce the impact of model uncertainty on the bank’s risk-adjusted return. We provide detailed discussions on specifying the cost matrix under the Basel regulatory framework. Empirical results for North American large and medium-sized companies show strong performance from the proposed approach.