Lifetime of a virtual particle?

It is common to read that the lifetime of a virtual particle is given by the uncertainty relation:

$$\tau \sim \frac{\hbar}{E}$$ on the premise that the virtual particle 'borrows energy'. This statement is infact wrong (at least I think it is) since energy is conserved in Feynmann diagrams and thus no energy needs to be borrowed. Given this, how do we actually determine the lifetime of a virtual particle, and why is it not just the same as the real particle?