Certainly look like lilies! A check on a gardening site will confirm it, such as Royal Horticultural Society, Wikipedia etc.

What a wonderful vibrant orange colour. In my garden, I have many tree saplings and plants appear that I have not planted. Must have come from bird droppings and seeds blown in by the wind.

Thanks for the photographs.