The new chips are substantially faster than older products, for several reasons. The Ivy Bridge family doubled the number of GFLOPS/cycle that the CPU was capable of performing, Ivy Bridge hits modestly higher clock speeds than the older Westmere parts, the chips themselves have 50% more cores, and Intel has improved performance incrementally between Westmere and SNB, and then between SNB and IVB.
Add all those things together, and the end result is a substantially faster chip. One of the goals for the new E7 v2 family, however, was to create cores that could scale to hit higher performance targets but would also be more power efficient than previous designs.
In many ways, these new cores are a response to the rapidly changing market conditions of the current PC industry. These new enterprise servers are designed for the kind of Big Iron deployments where performance trumps most other considerations, but they address power efficiency and efficient scaling both per-socket and across an entire server node. Intel doesn't have many competitors left in the Big Iron space, but it clearly views these products as essential to its long-term business health.
The significant improvements to the new Xeons will see their first serious challenge when IBM debuts its next-generation POWER8 architecture -- we'll be able to tell then if Intel made the right bets for this market space.