Behind the blur caused by the high-dimensional nonlinear dynamics and the intricate organization of complex systems, hide essential mechanisms that explain the emergence of macroscopic phenomena. To uncover those mechanisms, it has been common practice for researchers to model complex systems using dynamics that depend upon low-rank matrices describing the networks of interactions---what we call the low-rank hypothesis. We present three indicators of the low-rank hypothesis and evidence of its ubiquity among random network models used in various fields of study, ranging from network science and machine learning to neuroscience. We then verify the hypothesis for real networks of various origins and use our observations to examine the repercussions of the low-rank hypothesis on nonlinear dynamics. In particular, we show that having networks described by low (effective) rank matrices enables the dimension reduction of the nonlinear dynamics they support. As a surprise, we find that higher-order interactions emerge naturally from an optimal dimension reduction, which demonstrates the profound interplay between the description dimension of a complex system and the possibility of having higher-order interactions.