According to science, when you're afraid, your body does produce a variety of chemicals: sweat, adrenaline, the stress hormone cortisol, sometimes even urine. Dogs have proven the ability to detect all sorts of biological cues — even the telltale smells of cancer. It seems likely that dogs could also smell the chemicals of fear, but no study has been done.
Still, dogs are mind-bogglingly intuitive. They're better at interpreting human cues than chimps are, they know when we're lying, and they can even "catch" our emotions. Studies also show that they can read our facial expressions to know if we're happy or angry. If they can pick up on a smile, it's likely that they can pick up on the tensed muscles, clenched jaw, or wide eyes of someone who's afraid of them.