Downhill wheels are generally quite large compared to a skateboard so they usually go over stones OK, especially at speed, but it can definitely mess you up.
Same as with motorcycles, you stay away from edges of the road if possible, unless you know it's been swept by yourself or another rider...