When there is a well established design convention, there is something worse than drifting away from it: you can conflict with it. Let's look at an example.
Web browsers popularized the rule that page navigation controls should be located at the top-left corner of the window. This has been adopted since then by file explorers, the iPhone UI, etc. If you support some kind of page navigation in your application, that's a good idea to follow this convention if possible, because this is where people will look for the controls, instinctively.
What iTunes does is particularly deceptive and naughty: at the place you're looking for the page navigation controls, you will find buttons that strongly resemble them, but the damn commands are for songs, not pages!
And although i DO know the problem, today I still frequently click Previous Track by mistake, when I just mean to go back to the All Artists view. The music stops, and I'm stuck on the same page... It happens often because I usually don't use my brain when I want to perform something as simple as navigating back. Today, it's something we do all the time, the same way (top-left corner).
Imagine that tomorrow, you suddenly had to turn anti-clockwise to screw. Even if you know the new rule, it's likely that you'll still be trying to screw clockwise for some months. Or imagine yourself cutting your hair in the neck, using a mirror: you know you have to move left to go to the right and vice-versa, but I'm sure you noticed how insufficient this knowledge is to make you feel confident. I guess the brain just doesn't get why left to right suddenly becomes right to left.
When looking at the screw and the screwdriver, or in the mirror, everything looks standard until you take action. Nothing suggests "Hey, be careful, it's not as usual". So if you don't follow a convention for some reason, at least make sure it doesn't seem that you do!