electrostatics - Field between the plates of a parallel plate capacitor using Gauss's Law

Consider the following parallel plate capacitor made of two plates with equal area $A$ and equal surface charge density $\sigma$:

The electric field due to the positive plate is

$$\frac{\sigma}{\epsilon_0}$$

And the magnitude of the electric field due to the negative plate is the same. These fields will add in between the capacitor giving a net field of:

$$2\frac{\sigma}{\epsilon_0}$$

If we try getting the resultant field using Gauss's Law, enclosing the plate in a Gaussian surface as shown, there is flux only through the face parallel to the positive plate and outside it (since the other face is in the conductor and the electric field skims all other faces).

$$\Phi = \oint \vec{E}\cdot\vec{dA} = EA$$

where $E$ is the electric field between the capacitor plates. From Gauss's Law this is equal to the charge $Q$ on the plates divided by $\epsilon_0$

$$\frac{Q}{\epsilon_0}\implies E = \frac{Q}{A\epsilon_0} = \frac{\sigma}{\epsilon_0}$$

I know there is something fundamentally incorrect in my assumptions or understanding, because I frequently get conflicting results when calculating electric fields using Gauss's Law. I am, however, unsuccessful in identifying this.

Edit: Also, another problem I noticed was that even if we remove the negative plate from the capacitor and then apply Gauss's Law in the same manner, the field still comes out to be $\sigma/\epsilon_0$ which is clearly wrong since the negative plate contributes to the field. So, maybe the problem is in the application of Gauss's Law.