Understanding companies requires time and application. At about twenty companies in one or two industries, you start hitting the ceiling of what is possible for one analyst to cover. Hedge funds often don’t have the assets to be able to afford that many analysts.
But I think the real reason is that it is too damned difficult to beat the index just analyzing companies based upon publicly available information. Everyone is seeking an informational edge over the market. Some of this edge is through channel checks and such legal but proprietary sources. Much of it is through rumors – legal but quasi-public. And some of it is through insider information.
Informational edge is a slippery slope at the bottom of which lies insider information – the most alpha-producing informational edge. If you are a high achiever like most hedge fund managers are, and you have profited from proprietary information in the past, it is almost irresistible to cross the line. It doesn’t help that the difference between the difference between a rumor and insider information is only in how the information was procured. A rumor very well could have started its life as insider information. On large caps, placing a bet that is significant for the fund but small enough to escape being noticed is not too difficult. My belief is that insider trading is far more common than what one major bust every few years will make it seem like.
Jargon buster: Alpha refers to market (i.e., index) beating returns.