Why does it always feel so weird when you do the right thing?

In fourth grade, I got back an exam that was marked with a 100% at the top and a sticker. However, several of the answers were marked as incorrect (and, indeed, were incorrect). I took it back to the teacher, who thanked me and corrected the grade (it ended up being a 91% or something).

This month, we get our bill for rent for June, which states we have a credit of $140 and, besides that, our rent for June is about $150 lower than it should be. We call the “customer care center” who tells us that someone made a mistake when entering the new lease (since we had just renewed our lease for July 2009 through July 2010, at a lower rate) and changed the rent starting May 6 instead. They tell us not to worry about it, but I go into the rental office anyway and point it out. The lady there thanks me and corrects our June/July rent and goes back to fix May, so we end up with no credit and the full amount due for June.

In both of these cases, I feel like the guilt (or even paranoia, in the second case, since they have a signed lease with the correct amount) alone made them worth correcting.
But I can’t help but feel a bit of regret.

What are the odds that someone would have looked at one particular lease in a huge filling cabinet and noticed that the amounts between that sheet and the “official” amount in the computer were wrong? Probably more than the odds of the teacher asking for the quizzes back and noticing that mine had been graded incorrectly… but still remarkably small. But what it is that makes us decide that, regardless of how good the odds are that we could get away with it (and, really, we’ve done nothing wrong, so why are we worried?), we have to get the issue resolved?
I’d like to think that it’s because humans are fundamentally good and honest but, given how I’m feeling right now, I don’t really know if I believe that.