Observed with Adamants orbiting at or near 10 Mm, attacking a smaller fleet of Minotaurs orbiting below them near a planet's surface. The missiles take insane paths that loop ABOVE the orbital of the Adamants and rarely reach the Minotaurs. Adamants lose badly! Easily replicated.
Video here. Weird behavior seen after 1:20.
Check how missile paths are calculated, something really screwy is going on.
I believe the missile code is using a straight-line extrapolation to compute the future position of the target. Instead, it needs to use an orbital position.
I've re-worked the missile engine to fix these problems. In general, though missile paths are still a little odd (due to calculating paths in a gravity well), they always reach their targets.