John and I began zen practice together in 1989, and in a deep sense I have always had the feeling he and I are in it together, though he does not sit formally with a sangha these days. He's grounded in Buddhism and the shamanic traditions of Central America, and is also skilled at baking the small French baguettes known as Batards. His post was a comment below but I like it so much I wanted to mark it up before replying.
If we're calling charisma a combination of eloquence and personal magnetism, I'm not sure that charisma by itself is the problem. The work of shifting our consciousness is often difficult, and boring, and painful, and if the teacher has some quality that makes them attractive, then that can keep us coming back to the work when instincts pull us away. Many a time I've been simmering in my own suffering during sesshin, and been very grateful for the lift provided by a rousing, inspirational exhortation.
I think your friend's implied point though is that there is the suspicion that a charismatic teacher might use that power to further a personal agenda, one which is not in the student's best interest. Sadly, discerning ethics is much harder than charisma. But, as you point out (and echo Jung), I think the truth is that every teacher has some level of personal agenda, some measure of shadow. So if part of the teacher's shadow involves their ability to influence others, the important question is how bad is it? Bright light, dark shadow...brilliant light, engulfing shadow. Does the fire warm, or burn?
Our shadow here is also key to this equation. We have to ask ourselves why a charismatic teacher gives us the willies. "I don't like feeling hypnotized by someone" or "it isn't healthy for someone to have that kind of sway over me" is another way of saying "I don't trust myself" (not to become a groupie, not to take bad advice and make poor decisions, etc).
I think it comes down to an honest assessment of our strengths and wounds, and how they match or don't match those of the teacher. Perhaps it's useful to look at how we fall in love. If we have a tendency to get swept away and hand over power to another, then that's probably how we'll approach our relationship with a sprititual teacher. If so, then best to stay away from the stronger magnets. But, if we're naturally resistant to others' influence (which has equal drawbacks), then blacksmithing next to the forge might turn out ok. :)