I'm right handed so for pencil work I usually start at the top left of the drawing and work toward the lower right side.  This way I (try to) avoid smudging the drawing.