Before I opened the studio I worked for many companies, large and small [...] And I have worked for studios that were ten, 20, 30 people. And I feel that when I look at the work emerging from any of these entities, the really good work was done by three people. When 20 people were involved, it mostly sucked. So for me, it never seems that size works in favor of the client.
– Stefan Sagmeister (from How to think like a great graphic designer)
An obvious reason has nothing to do with the design discipline or creativity: it's just that communication overhead increases when new people join the project team.
Another reason is that the success of a design is much related to the consistency and clarity of the concept. It must speak to you, it must have a personality that you get easily. And all that begins to turn into a mediocre compromise when too many people influence the design. The best way to cook a delicious cake is certainly not to ask 10 guys to throw their favorite ingredient in the mixture!
I don't think that involving extra people in a creative process is a bad thing per se (often an external contributor adds a key element that was missing), but it inevitably triggers more discussions, often about emotional aspects that can be debated endlessly anyway.
The problem is that it disturbs the refinement process, which is critical. When we enjoy the beauty of a nicely designed thing, we don't see the refinement steps that led to it. But if we don't allow that distillation to run in peace, and stay focused, we might never see that nice product come out at all.
So while creating, it's great to have a few external eyes, but not too many–and above all, not continuously! And you need a decision-maker.
See also in this blog:
Conceptual Integrity