No, not at all :) We actually don´t even drive and yet, we lived on islands too (Malta, Canaries, Azores) where getting around without a car is quite challenging as you surely know :) Madeira was just a short vacation for us as my wife was already pregnant. But the island made a huge impression on us and we can totally imagine living there too. Btw if you like off the beaten path locations with few to no people, you should check out the Azores. The mass tourism hasn´t really spread there yet, especially on the little islands and in the rural parts of the bigger ones.