When doing viewport rendering the color management happens on the CPU. This has overhead in downloading a float texture from the gpu to cpu and doing color management on the CPU.
OpenColorIO color management currently is of higher quality, but when doing viewport rendering it is ok to do it with less quality. Performance is much important than quality. Final rendering still uses colormanagement on the CPU (Quality over performance).
Baseline (HD render of wanderer.blend workbench engine no samples)
After first changes: 9.412880s