kinematics - Why does work equal force times distance?

My book says:

Energy is the capacity to do work and work is the product of net force and the 1-dimensional distance it made a body travel while constantly affecting it.

This seems quite unmotivated to me.

Why does work equal $F \cdot d$ ?

Where does the distance part come from?

I always thought of time as the one thing we can only measure (not affect) so it justifies why we may measure other things in relation to time. But we have a much greater control over distance (since it's just a term for a physical dimension we can more or less influence as opposed to time).

Edit: That^ doesn't make much sense, but it's been graciously addressed in some of the answers.

Level: High-school Physics.