First of all, let's clear up one cliché: people that are too good in their role never get promoted. And it is the other ones who spend more time doing politics, that get these promotions. This happens (especially in fast-growing and/or immature companies) and it is almost understandable why - it is not easy to identify these people (and fire them), the senior management is clueless and actually doing the same (like a ponzi scheme), etc ...
There is a different approach possible though.
As people progress in their career, they learn about other functions e.g. about project management for an engineering team lead. They work with many different managers who all have different approaches to doing their job. They make their own opinion about what could be done for the organization to be more efficient.
To be clear, there is no one-size-fits-all for defining the most efficient way-of-working across an organization. The context differs because of the nature of the industry it operates in and its customers and also depending whether it is shareholder-driven or a family business. Therefore there isn't a unique recipe for how to organize a company for maximum efficiency.
I often have to deal with the question of who is best to promote as team manager. I think the best opportunity to promote someone as manager is when that person realizes that he could do more if he had people with him, under his direct orders. Basically he wants to scale: by giving that person a team, and if it works well (i.e. he, as a manager, successfully delegates to the team members), he suddenly multiplies his through-put. In short, to become a manager, the candidate has to have a vision and the patience to implement that vision through team-work.
An example of someone who have been promoted that way to the top is Sundar Pichai. He has grown up in the management layers of Google starting with a technical focus and expanding it to a product management & business one. I think he was trusted as someone with a vision who could to scale up the Google CEO position.
That said, there are also engineers with a vision who are unable or uninterested to manage people. They would like to scale but stay focused 100% on technical contributions. These people then typically should get promoted on the technical track (as opposed to the managerial track) but the challenge is to have them share their vision with a manager that can implement it through team-work. Not always easy but possible.
One more thing about the team: it is the role of the manager to create a common spirit inside the team. Recently, I asked one of my manager why the hell he did not have weekly team meeting. He answered that each team member was working on way-too-different work packages which made the team meeting too boring. For me, this is wrong, a team must be united and even if it is somewhat artificially done, work and responsibilities must be shared between team members. If not, that team must be reorganized.
These days, we discuss a lot about the concepts of "feature team" versus "component team". Currently, we are very much a component-driven engineering organization (which is a great improvement compared to a firefighting-driven engineering organization). That said, it is not incompatible with a feature team approach, in the sense that:
- it is the role of the team manager to spread knowledge and responsibilities between team members.
- it is so much more powerful to split work in a way that it can be fully implemented by a team even that could mean doing changes in components owned by other teams.
