One of the biggest problems in creating a SF story in any medium is the concessions that have to be made to the medium in which the story is told. This is especially prevalent in visual media such as movies, games, and comics/animation. The problem is acerbated for those how prefer SF that falls onto the 'harder' end of the spectrum, as they lose many inaccuracies usually used because they make the telling of the story easier. While someone unconcerned with how a space battle might actually look given the technology concerned it is a simple matter to add some glory lasers and explosions, copying what has worked in other franchises like Star Wars or Star Trek. For those of you that do worry over these details, read on..
I'm going to discuss some of the more common elements of space battles in visual SF, although by no means all of them. For each I'll try to look at why they are shown the way they are, why they are inaccurate, and how they can be changed for greater realism. A lost of this sort of thing comes down to the exact details of the setting so I will focus on fairly broad issues, things that are unlikely or impossible given our current understanding of physics.
While you might think that a discussion of physics or technology would be the most important part of this blogpost, there are other aspects to creating a space battle equally important. If the battle is part of a story then it has to show the story or it is ultimately pointless.
Say you are producing something along the lines of Star Wars or Battlestar Galactica with main characters who fly fighters, and with a story that focuses on the personal element that they bring into the tale, rather than the grand strategy of admirals and kings. In this situation frantic dogfights with wheeling fighters, explosions, and hair-raising manoeuvres will probably serve the story quite well.
If, on the other hand, your story is about a nation, a war, or a leader rather than a simple soldier it has different requirements. Tactics will let you explore the leader's abilities, strategy will convey the importance of certain actions or individuals in the grand scheme of things. In this case your might want to set your scene on the bridge of a starship, with huge tactical displays, and hasty conferences between the officers.
The answers to the above questions will help you to decide on the details of your setting that facilitate the decision; things like the technology used, the strategies and goals, etc. Only then can you begin to work on the visual aspects of the space battle and know that it will support and compliment the story you are trying to tell, rather than distracting the viewer from it.
The two examples I gave are by no means the only choices, and you can even switch between the two within the course of a single battle. There are also other similar considerations; the design or a ship can be used to reflect on the nature of those who built them. Dull colours and blocky shapes can suggest lower level technology when contrasted with organic curves and polished metal. The scenario can also be expanded visually; chaotic formations during an ambush, or elegant fleet manoeuvres during the siege of a planet.
Space is called Space for a reason, mostly because there is a lot of it. Yet in a almost any work of visual SF - Star Wars, Star Trek, Farscape, Babylon 5, etc. - ships huddle together as though running out of space; as they are, in a way. The situation is even more ludicrous when you delve into the lore of the various settings and find out that the weapons involved are apparently capable of engaging the enemy at beyond visual rages. In anything remotely 'realistic' this is an almost inexcusable error. The reason it is so widespread is quite simple; screens have only so much space. To show a fleet or a battle in as exciting a way as possible, or to convey the course of a battle, the artist needs to crowd the spacecraft together.
Fortunately the cure is almost as simple as the explanation. The first is a technique occasionally used in Babylon 5; it involves showing one ship firing, and then cutting to its target and showing the impact of its weapons. Having a planet or other large astronomical feature in the background can help to give a sense of the distance between the opposing spacecraft, and if visible energy weapons are being used colour can differentiate between enemy and friendly fire. The second technique, and the one that better conveys a fleet, is to frequently show the tactical displays being used by the officers directing the battle. Since such a display should be designed to provide information in a clear manner it should not be too hard to make it legible to the audience. It also has the advantage that the display can be used to show planned manoeuvres before they are carried out.
Whenever missiles are used in visual media they are almost always have a neat trail of smoke or vapour to mark their progress, something occasionally produced by manned craft as well. Likewise energy weapons will glow brightly in all the colours of the rainbow, and even kinetic rounds will glow brightly. Once again the reasoning behind this is simple, it makes it easy to see what is happening and it looks cool. Movies and games both use this tactic, games often going on to give fighters long exhaust trails that make their trajectory say to determine visually. The Homeworld strategy games are notable in this regard with most small ships producing visible trails.
Glowing kinetic rounds, while they may seem the most unlikely, are in fact the most realistic - to an extent. Gauss cannon or railguns could heat up the projectile they are firing; simple iron shots might be glowing red or white hot by the time they leave the barrel. But while they might emit some light it is likely to be so little that combined with the small size and high speed of the projectiles they would be invisible to the human eye.
Missiles with smoke trails also seem to be quite plausible at first. Aircraft can leave contrails behind them, and chemical fuelled rockets will most likely produce some smoke, although it is minimised in missiles to make them harder to spot with the Mk.1 Eyeball. In space however a vapour or smoke trail will never form; with no air to slow and support particulate matter from the rocket engine the trail will disperse far to fast for the human eye to catch.
Energy weapons glowing seems like a pretty rational thing. They are after all composed of energy and in the case of plasma weapons are utilising a state of matter that does usually glow. Lasers will not be visible without particles to scatter the beam, the only reason that we can see them on Earth. The component particles of a particle beam do not emit energy unless they are slowed or deflected, and when this does occur the radiation emitted is probably not going to be visible given the energies involved. Plasma weapons might be visible as glowing projectiles, if they ever prove to be feasible in the first place; but given the tennis nature of fusion plasmas, and the relatively small volume of a plasma bolt, they might be very hard to see even then. The weapons that will be very visible would be things like nuclear shaped charges, although they would also be very brief.
Aside from the fact that this means space battles will never look like how Hollywood thinks they will there are a few other result of this. You will almost never see incoming fire. In the case of particle beams and lasers even sensors will not, in the first case because there is no radiation emitted, in the latter because the beam is travelling at the speed of light. Missiles and projectiles might be seen, but not by the human eye due to their probable velocities. In a written work this is easy to accommodate and while a visual work might have more trouble many weapons can be given a muzzle flash or other effect to show that they have fired, and energy weapons can have radiators that begin to glow(although visible glowing radiators are in themselves unrealistic). The bigger problem is that without visible weapons fire it may be difficult to convey to the viewer what the individual spacecraft are shooting at.
One of the most dramatic moments in a space battle is when a spaceship with 'the good guys' aboard takes a hit. In most franchises this results in a big gout of orange fire and then, if we're lucky, a shot of the hull plating torn away from the support structure underneath, preferably glowing from the energy of the shot. This same ball of improbable fire makes its appearance when a spaceship, missile, or asteroid is blown up, and often on the detonation of nuclear weapons.
The problem with this is that high energy explosives use in warheads do not produce an orange fireball like an exploding car. Normal high explosives usually produce a nearly invisible blast since it is thermal energy that creates the visible fireball, and most explosive warheads are optimised to produce mechanical damage. A thermobaric warhead would create a nice gout of flame, but they are fundamentally unless in space. The explosion of fuel stores, the ignition of leaking atmosphere, and the release of superheated coolant can be used to explain these gouts of flame to some extent, but even then the effect will be far more rapid than inside an atmosphere, becoming more of a flash than a ball of fire. And even if something aboard the targeted vessel is ignited there will be no billowing smoke clouds, since these too will expand extremely rapidly in the vacuum of space.
High energy weapons such as lasers, particle beams, nuclear warheads, and possible plasma cannon, might leave glowing areas on the armour of a spaceship. I don't have the knowledge to judge on that. I do know that they will not produce big orange fireballs, though. High powered weapons strikes are going to be a bright flash of light, potentially coloured depending on the materials and weapons involved, with mechanical deformation affecting the surrounding area to some extent. The extent of the damage and its appearance is quite a complex question, and the best place to get answers is over on Atomic Rockets.
Related to this is the explosive end of every ship damaged in battle, usually with a nice fireball. Given the penetrative qualities of kinetic weapons, and the soft kill potential of radiation based weapons, it seems likely that many ships could be put out of action with very little exterior damage, at least in the case of larger ships. Given that real fusion and fission reactors are likely to explode the only explanation is that it is a result of high energy reactors of other kinds loosing containment. Certainly in Star Trek with their antimatter warp cores even an small but penetrating hit could cause a massive explosion. From a historical perspective it was easier to add a fireball back when CGI was in its infancy that it was to do complex battle damage, so that is the reason why it is so common in older works; newer works just continuing the trend for the most part.
The final inaccuracy commonly found alongside dramatic explosions is that nearby spacecraft will shudder as though hit by a blast wave; yet this is impossible without an atmosphere. The total mass left behind by an exploding missile, or even a ship, is insignificant compared to the volume of space, and unless the other spacecraft is ridiculously close, or shrapnel is involved, this is not going to happen.
None of the above are significant problems. They are mostly matters of convenience, of limitations in the medium used to depict that battle, or of convention. There is no particular trick to incorporating the 'correct' version of space battle visuals, but it does seem to be somewhat uncommon. I suspect that it is largely a result of the saturation of Star Wars and Star Trek; people are inspired by these franchises to create their own SF but often end up with a variation, rather than something that makes more sense on the world-building side. This is kind of like the ridiculous size of many spacecraft in SF, everyone else is doing it, so there is a temptation to use the exist metric rather than try to make the viewer use your universes internal scale or logic.
