Most topsoil is about half worm castings anyway. The other half is usually a combination of rocks, sand, sticks, and decomposing matter. I'm sure it'll work out great! Don't bring any bugs or eggs into the garden with the topsoil.