Yea I think the third book was the start of what the series to become more than book for children. First and second book introduce us to the magic system, sure, but Prisoner of Azkaban IMO is far superior in exploring the said concept about magic. The plot was so creative, and literally the plot device, haha, JK Rowling surely knows how to use it!