I think being crazy is acting outside the pattern of normal behavior. Breaking social behavior patterns is sometimes considered madness.

Crazy is not only the one who hears voices, but it also matters what he does with that new perception. Is going to give more attention to those voices than to the physical reality he experiences? If so, you may be considered crazy. If he can live with that without disturbing anyone, maybe nobody even finds out.

Labels are something that is placed from the outside after all. No one knows what goes on in the head of a crazy except the crazy himself.