Who we give heed to matters a lot. It is not everyone that we should listen to. The thing of question is that who should we obey more; man or God?
Acts 5:29
We ought to obey God rather than men
The yardstick we use to measure how we do things should always be according to the standards of God and not of man. Abraham had a history of idolatry from his family background, but that did not make him give in to the patterns of his fathers. Instead, he made a choice to walk with the one true God.
The worst thing that can happen to us is when our trust in God is displaced and can lead us away from the purpose of God. That was why God sent us His son in the person of Jesus to; to take us from the darkness and bring us to light. Because our ancestors were in darkness does not mean we also should continue in darkness.
Read next
From @cruciform