In this paper we prove a regularity and rigidity result for displacements in GSBD^p , for every p > 1 and any dimension n ≥ 2. We show that a displacement in GSBD^p with a small jump set coincides with a W^{1,p} function, up to a small set whose perimeter and volume are controlled by the size of the jump. This generalises to higher dimension a result of Conti, Focardi and Iurlano. A consequence of this is that such displacements satisfy, up to a small set, Poincaré-Korn and Korn inequalities. As an application, we deduce an approximation result which implies the existence of the approximate gradient for displacements in GSBD^p.