In charge-coupled circuit QED systems, transition amplitudes and dispersive shifts are governed by the matrix elements of the charge operator. For the fluxonium circuit, these matrix elements are not limited to nearest-neighbor energy levels and are conveniently tunable by magnetic flux. Previously, their values were largely obtained numerically. Here, we present analytical expressions for the fluxonium charge matrix elements. We show that new selection rules emerge in the asymptotic limit of large Josephson energy and small inductive energy. We illustrate the usefulness of our expressions for the qualitative understanding of charge matrix elements in the parameter regime probed by previous experiments.