Or have the numbers filled in with the team colour, and then around it, have the negative of it for an outline. That'd work fine.
What about people who have custom HUDs. What would happen then?
Also, this ties in closely with the feature of auto-team-balance. The main reason why people didn't want that was because it would be too confusing. With the HUD, you can tell much easier which team you're on.