Hi Mark broma16,
I don't believe you are missing anything. This issue is most likely caused due to a known limitation of how the Milestone charts work. Currently the milestone charts include items both directly associated to the milestone and items that indirectly associated. However, the list of scheduled items for the milestone only includes the items directly associated which can easily cause confusion.
We have received both questions and requests to enhance the functionality of these charts to display this information differently. I would highly encourage you to visit this idea https://ideas.rallydev.com/ideas/D4230 in which another user explained the issue extremely well. Unfortunately changing this behavior is not currently planned so its important to add your vote and comments to it to help it gain attention so that it might be changed.