Photon-based strongly-correlated lattice models like the Jaynes-Cummings and Rabi lattices differ from their more conventional relatives like the Bose-Hubbard model by the presence of an additional tunable parameter: the frequency detuning between the pseudo-spin degree of freedom and the harmonic mode frequency on each site. Whenever this detuning is large compared to relevant coupling strengths, the system is said to be in the dispersive regime. The physics of this regime is well-understood at the level of a single Jaynes-Cummings or Rabi site. Here, we extend the theoretical description of the dispersive regime to lattices with many sites, for both strong and ultra-strong coupling. We discuss the nature and spatial range of the resulting qubit-qubit and photon-photon coupling, demonstrate the emergence of photonpairing and squeezing, and illustrate our results by exact diagonalization of the Rabi dimer.