Maybe fish are bit clever this time. They stole your worm without getting caught 😬😁.
If your still fishing when the rainbow appears, maybe you could have seen the end of he rainbow.