Blur your maps

Very bright and dark parts of the displacement map will be shifted more than midtones. So to make the map more defined, you might want to increase contrast with Levels or Curves.

It can sometimes help to add a slight blur to the displacement map file, as this can lead to a smoother result after you apply it – especially if the surface of the map is very textured. It’s worth experimenting with.