Speculation is an approach whereby the compiler or processor guesses the outcome of an instruction to remove it as a dependence in executing other instructions. Speculation may be done in the compiler or by the hardware. With speculation, we fetch, issue, and execute instructions, as if our branch predictions were always correct.
Wrong?
The difficulty with speculation is that it may be wrong. So, any speculation mechanism must include both a method to check if the guess was right and a method to unroll or back out the effects of the instructions that were executed. In the case of speculation in software, the compiler usually inserts additional instructions that check the accuracy of the speculation and provide a fix-up routine to use when the speculation is incorrect. In hardware speculation, the processor usually buffers the speculative results until it knows they are no longer speculative.
Exceptions
Speculating on certain instructions may introduce exceptions that were formerly not present. In compiler-based speculation, such problems are avoided by adding special speculation support that allows such exceptions to be ignored until it is clear that they really should occur. In hardware-based speculation, exceptions are simply buffered until it is clear that the instruction causing them is no longer speculative and is ready to complete; at that point the exception is raised, and normal exception handling proceeds.
Speculation takes time and energy, and the recovery of incorrect speculation further reduces performance. In addition, to support the higher instruction execution rate needed to benefit from speculation, the processor must have additional resources, which take silicon area and power. Finally, if speculation causes an exceptional event to occur, such as a cache or translation lookaside buffer (TLB) miss, the potential for significant performance loss increases, if that event would not have occurred without speculation.
Most pipelines with speculation will allow only low-cost exceptional events (such as a first-level cache miss) to be handled in speculative mode. If an expensive exceptional event occurs, such as a second-level cache miss or a TLB miss, the processor will wait until the instruction causing the event is no longer speculative before handling the event.
Speculation through multiple branches complicates speculation recovery.
Hardware vs Software
Hardware-based speculation works better when control flow is unpredictable and when hardware-based branch prediction is superior to software-based branch prediction done at compile time.
Hardware-based speculation maintains a completely precise exception model even for speculated instructions.
Hardware-based speculation does not require compensation or bookkeeping code, which is needed by ambitious software speculation mechanisms.
Compiler-based approaches may benefit from the ability to see further in the code sequence, resulting in better code scheduling than a purely hardware- driven approach.
Hardware-based speculation with dynamic scheduling does not require different code sequences to achieve good performance for different implementations of an architecture.