You really should not blame yourself and feel bad for spending time with your family in your free time. Family, friends and work are priority. And time for yourself too. :)