What's prevents me from rolling backwards on a hill when I release my clutch?

This is the first car I've owned that does this. When I release the clutch at standstill on a hill I don't immediately start rolling back. There's something holding me. Then after about 3 seconds it releases and I start to roll backwards. What's this called? Is it just my brakes?