A stock that has a low market cap and low price. It is usually traded outside major exchanges while carrying a price below $5.
They can be on such exchanges as NYSE but they usually trade via Over-The-Counter (OTC) transactions on the OTC Bulletin Board (OTCBB).
Since there can be a low float, some are very volatile. There also can be a large spread between the buy and ask. Liquidity is often a problem with penny stocks.