Except that the MS Catapult results seem to suggest that you can get a substantial timing bump without much power overhead by using a second network of FPGAS running concurrently with the CPUS.
And in the case of Catapult, they'll be refactoring the algorithm to represent changes to their search feature matching, which is the majority of what they offloaded to FPGAs.
Yes, the Bing example seems to give real-world validation of all this.
In general it seems that hardware vs. software fit is orthogonal to concerns such as frequent reconfiguration. Some algorithms simply match hardware well (high concurrency, low-complexity control flow, able to stream through data without complex state). These are often algorithms that do not fit general-purpose CPUs well (cache hierarchy wasted on streaming; lots of control overhead; low core counts relative to FPGA-level parallelism). Some of these algorithms may be for specialized and/or frequently-changing applications such that they should not be burned into an ASIC that will live in a datacenter for 3-5 years.
And in the case of Catapult, they'll be refactoring the algorithm to represent changes to their search feature matching, which is the majority of what they offloaded to FPGAs.