Use passphrases.
I gave my passphrase to closer ones without letting them know what it really was. They can't use them without your device, neither with 'only' your 24 seed words.
'Keep this piece of paper secure just in case something very bad happens to me. Don't trust anyone. Never tell anyone about his existence.'
RE: Just because they never believed in you, it doesn't mean that they don't need to know...Our lonely ride.