How you have explained it is similar to what I have believed since I was a kid. We build roads of information conduits, and the more we use a road, the more the brain builds it, so some are highways. I think the pruning happens naturally through disuse (things might become irrelevant as we age), or disease like Alzheimer's. I see the reason that Alzheimer's can be hard to detect very early, is that it is pruning and the smallest roads we don't use disappear and we don't notice. The ones we would notice are the highways, but they have a lot to lose before we are noticeably "affected" by the loss. At some point though, the loss of the smaller roads means that there isn't enough information feeding into the larger roads either, or it is joining the conscious flow in random spurts.

What I have noticed in myself however, is that it isn't always possible to build new roads once some are lost. The brain might be able to circumvent and "mimic" some of the lost function, but it doesn't necessarily replicate it.