There are many problems and flaws with the EU as a supranational polity. But Brexit is not a libertarian or freedom project. It is a project mostly driven by right-wing nativism, collectivism and populism. Also, UK is one of the most centralized states in Europe. How many libertarians are in favor of villages, towns and cities leaving the UK?

https://www.cato.org/publications/commentary/libertarian-case-european-union?fbclid=IwAR3Yf_sBg_oAJj8W542JyKjb0ToQstUy6H08HASSGh_1GjejsnATTnvpoc0