Geek check
Sarah can attest that I do something similar to this.
Except instead of factoring, I convert them into their lowest possible base. So, a time like 10:01 would be in base 2 (decimal: 9), a time like 1:22 would be in base 3 (decimal: 17), etc. It gets harder and harder to do, so base 7 is quite difficult (although base 9 is extremely easy by comparison).
I once told Josh a story about how I woke up at 8 (10:00), but didn't get up until 12 (11:00). I said that I needed to leave for my class around 4 (1:00), so that I could start taking my test at 6 (1:10), but that I definitely wouldn't be finished by 7 (1:11). He was confused, to say the least.