Credit ratings play an important role in the bond market, as the regulatory framework of this sector is based on ratings. A critical assumption is that the ratings of all the rating agencies are equivalent, ie, they exhibit very close agreement. This paper, borrowing concepts from measurement, test and psychometric theories, explores this issue in the Mexican corporate bond market. The ratings of all three credit rating agencies (CRAs) show a high degree of inter-rater reliability (similar ordinal relationships implied by the ratings); they only exhibit a mild level of inter-rater agreement (the degree to which two CRAs give the same rating), and they reveal significant discrepancy in ranking in paired observations. In summary, the three CRAs give ratings that are not equivalent, as their respective distributions are dissimilar. Therefore, our results challenge the suitability of ratings as a useful metric for regulatory purposes, as they create the possibility of arbitrage (rating shopping).