To characterize gradient field nonuniformity and its effect on velocity encoding in phase contrast (PC) MRI, a generalized model that describes this phenomenon and enables the accurate reconstruction of velocities is presented. In addition to considerable geometric distortions, inhomogeneous gradient fields can introduce deviations from the nominal gradient strength and orientation, and therefore spatially-dependent first gradient moments. Resulting errors in the measured phase shifts used for velocity encoding can therefore cause significant deviations in velocity quantification. The true magnitude and direction of the underlying velocities can be recovered from the phase difference images by a generalized PC velocity reconstruction, which requires the acquisition of full three-directional velocity information. The generalized reconstruction of velocities is applied using a matrix formalism that includes relative gradient field deviations derived from a theoretical model of local gradient field nonuniformity. In addition, an approximate solution for the correction of one-directional velocity encoding is given. Depending on the spatial location of the velocity measurements, errors in velocity magnitude can be as high as 60%, while errors in the velocity encoding direction can be up to 45 degrees. Results of phantom measurements demonstrate that effects of gradient field nonuniformity on PC-MRI can be corrected with the proposed method.