A shortage of skilled labor is perhaps the biggest problem facing construction contractors. Lousy pay and even lousier prestige conspire with unfavorable demographics to repel talented young people from the trades. This makes it devilishly hard to hire top-notch workers.
If you don't feel that's the biggest problem, it probably comes in second to the one that bugs you the most, which would then be the proliferation of bottom-feeding competition everywhere you look. Too many contractors are superb with tools and shop drawings in their hands but clueless with a balance sheet. Their lack of business sense makes it impossible for them to prosper and harder than it ought to be for those of you who do know how to run a business.