In general, the Linaro images main aim for the greatest possible hardware enablement. The Reference Platform (Build) images have a very different focus. These are reference builds to show, by example, how to integrate the very latest open source software. The kernel in RPBs is very recent and fairly closely related to mainline; if certain drivers for a board are not compatible with recent kernels they will not be included. Additionally a single RPB will support a variety of different boards, some of which are not 96Boards products, and may not be configured optimally for a specific board (bug 402 is an example of this).
That means a rough approximation is, if you want to use all the board hardware features then try the Linaro images first, if you want to develop new software features then try the RPBs first.
Note also that, when there is an active development team working to upstream driver support, then the hardware support in the RPBs is likely to fill out at bit. In an ideal world the RPB might even eventually catch up with the Linaro build for a board. Ignoring KVM, I believe the Hikey RPB is getting close to this point.