Why do elephants, whales, and other big animals, including extinct ones like dinosaurs, get so large? This is actually two different questions- one for land animals, and one for aquatic animals. Let's cover land animals first.
Argentinosaurus, a leading candidate for the largest land animal of all time. [Image source]
In order to discuss the true giants, we're actually going to go down in size quite a bit- down to insect scale, in fact. Meganeura, an inhabitant of the Carboniferous era (300 million years ago) was the largest dragonfly (well, technically a closely related genus) to ever live, with a wingspan of 27 inches. The largest dragonfly today has a wingspan of only 7.5 inches. So how'd Meganeura get so big?
Well, there are a few theories regarding that. The most popular one is that the much higher oxygen levels of the Carboniferous, nearly 20% higher than our own, allowed Meganeura to get much bigger than our insects. Since insects don't have lungs, and breathe through a series of tracheal tubes present throughout their body. There were quite a few other huge insects and other arthropods as well in the Carboniferous. Then, of course, there were no flying vertebrates to prey on Meganeura, and Meganeura seems to have been at the cap for maximum size for insects even then- Meganeura was the alpha predator.
Meganeura fossil. [Image source]
Around halfway through the Carboniferous the oxygen levels crashed and the giant insects, including Meganeura and cockroaches the size of house cats, vanished with them. The mighty forests of the Carboniferous began withdrawing into enclaves and splitting apart. This led to animal populations being isolated from one another, causing more extinctions and rapid changes in evolution- our old friend island biogeography rearing its head again. (Also, fun fact- most of our coal comes from Carboniferous deposits!)
We can draw some interesting lessons from Meganeura. For terrestrial organisms to achieve truly huge size, they need a couple of things. First off, they need ample resources- food, water, the works. It takes a lot of energy to fuel enormous bodies. This is why almost all of the giant animals on land have been herbivores- Meganeura is a rare exception to that rule. Second, they need a specific relationship with predators. Either they need to lack predators entirely, or they need to be involved in a size-based arms race against their predators. In the latter case, herbivores can usually always win- it's just much easier for ecosystems to support giant herbivores than predators. Predators take up a lot more energy, and have much larger territories they need to feed as a result. In the former case, we get things like island gigantism. Without predators, animals all of a sudden have much, much different population pressures. Territorial or mating battles within a species suddenly tend to drive population increases, for instance. The bigger you are, the more likely you are to win territory or mates. There are other pressures driving gigantism in these situations- but it should be noted that dwarfism is just as common if not more common of a response to lacking predators.
The blue whale is the single biggest animal to ever live, and it lives right now. At 100 feet long and nearly 200 tons in weight, it dwarfs everything in the fossil record. [Image source]
The story gets a little more complicated in the ocean. There aren't very many large herbivores in the ocean. All of the whales are carnivores. The blue whale, the largest animal to ever live, consumes nothing but krill- tiny animals almost too small to see. It's a free for all in the ocean. Unlike on land, you can't draw easy food webs- one species might prey on the young of a second species while being preyed on by the adults of the same species. Getting bigger is an excellent evolutionary tactic in the ocean. Also, the water supports the weight of large oceanic animals, so that helps them get way, way bigger.
Paraceratherium, once thought to be the largest land mammal. We now know there was one species that likely got even larger: Palaeoloxodon namadicus, an enormous ancient elephant. [Image source]
We should also mention large animal reproduction. There are two major reproductive strategies among animals. First, you can have an absolute ton of offspring and hope a few survive. Second, you can have a much smaller amount of offspring and invest huge amounts of energy in each one to ensure they survive. Most (though not all) giant animals follow the second strategy. It's an effective strategy for getting big. With a giant and terrifying parent hanging around, not much wants to mess with the kid.
There are, of course, limitations on the peak size of animals. Gravity prevents animals from getting too large- especially land animals. Food also is a limitation- the bigger you are, the more food you need. Most importantly, the square cube law prevents it. As the dimensions of the animal increases, the volume starts increasing much, much faster. Eventually, the mass gets so high that an absurd amount of the animal would have to be bones and muscle to support its weight- enough that it would no longer be viable. We're unlikely to see animals much larger than those that have already existed on Earth. My apologies to Godzilla and King Kong. On a lower gravity planet, of course, you could get much, much larger creatures.
Here's the inevitable point where one my post gets depressing as hell.
The elephant bird, Aepyornis maximus, the largest bird ever to exist. Driven extinct by humans. [Image source]
Unfortunately, being large isn't a great strategy in times of severe ecological upheaval. It takes a lot of energy to be big, and when there is rapid ecosystem change or collapse and the ecosystems can't provide that energy anymore, giant species are the first to go. They essentially never survive past mass extinctions. And, unfortunately, we're in a mass extinction now.
Humans have suddenly made being huge a really bad survival strategy. We're fragmenting and destroying ecosystems that giants depend on. We're hunting the giants- we killed the overwhelming majority of all terrestrial megafuana during prehistory, and we almost wiped out the whales during the few short centuries of whaling's heyday. We've fished out 90% of the large fish from the oceans. We've made it so that being small is a considerable environmental advantage. Our children won't get to see the giants our parents did. We're leaving them a much smaller world.
Bibliography:
https://en.wikipedia.org/wiki/Meganeura
http://www.eartharchives.org/articles/the-biggest-insect-ever-was-a-huge-dragonfly/
https://en.wikipedia.org/wiki/Carboniferous
https://en.wikipedia.org/wiki/Carboniferous_rainforest_collapse
https://en.wikipedia.org/wiki/Blue_whale
https://en.wikipedia.org/wiki/Largest_organisms
https://en.wikipedia.org/wiki/Paraceratherium
http://www.slate.com/articles/news_and_politics/explainer/2009/02/a_snake_the_size_of_a_plane.html
https://www.smithsonianmag.com/science-nature/why-king-kong-should-have-been-blue-whale-180962603/
https://en.wikipedia.org/wiki/Island_gigantism
