Actually, the idea you should exercise in the morning before you eat is a commonly held misconception. It is based on the theory that if you don't have food in your stomach then your glycogen supply will be depleted overnight, which enables the body to use fat faster and more efficiently during your workout.
In reality, you need to eat before exercise, especially if you exercise in the morning. The simplified explanation (i.e., the explanation without all of the medical jargon) is that the body needs some sugar (read = carbohydrate) to burn fat. Remember that when you wake up in the morning you have been "fasting" for several hours while you are asleep, so your body has not been replenishing any of the sugars (glycogen/glucose) that it needs for fuel. If sugar is too low, the body does not tap fat stores to fuel itself through the workout, it actually taps Protein stores (which, as you know, inhibits muscle development). The food you need to eat can be something small - orange juice, yogurt, banana, Protein shake, but it needs to have a little carb and if possible a little protein.
Research says that (generally) the afternoon is the best time to exercise. Studies show that strength and endurance are better in the afternoon and injuries are less likely in the afternoon. A lot of people suggest morning exercise because studies find that people are more consistent.
So, finally, to answer your question, you can exercise later in the day if that is a time that is better for you but you should also make sure that you pick a time that you can be consistent. Sorry, this was the very long answer to your question.