Do'h! Homer'D!
I can't confirm this (I hope someone can) but I think the timer doesn't run during the loading/flying/traveling cutscenes. I don't know if it runs during the "cinematics" cutscenes though (but you can skip those on your 2nd playthrough).






