Despite his credibility in financial circles, the Federal Reserve chairman has come under fire from lawmakers.