For me, most of the stuff works if I go and remove the filters on CF... but the problem is that it breaks the certs (becomes a self-signed thing for the browsers).

And I am not sure if there is a way to change certs at the nginx/caddy level, depending on where the request is coming from. If there was something like that, accesses directly to my IP would use that one, while via CF, the CF cert.