quantum gravity - Is the consideration of spacetime as a smooth manifold only an assumption?

General relativity (and already the two postulates of special relativity) seems to refer to timelike and spacelike worldlines of particles and fields. It seems to be impossible to apply special and general relativity to vacuum points of spacetime. One reason is the fact that the velocity-dependant Lorentz-factor cannot be assigned to points for which no velocity is defined. Vacuum seems to be a matter rather of quantum physics than of relativity.

Accordingly, e.g. John Lee writes in "Introduction to smooth manifolds", chapter 1:

"In the application of manifold theory to general relativity, spacetime is thought of as a 4-dimensional smooth manifold that carries a certain geometric structure, called a Lorentz metric, whose curvature results in gravitational phenomena."

Does this mean that the smoothness of spacetime is an assumption which is independent of (and additional to) special and general relativity? Please note that I am not referring to any smoothness of any 3D-space, but exclusively to smoothness of 4D-spacetime of special and general relativity. In particular, I am asking about smoothness in spacelike direction, which seems to be a prerequisite for certain theories of quantum gravity.

Also, it is hard for me to find literature on this question. The mentioned citation of John Lee is the first indication I found. Where can I find more formal information on the question if or if not the consideration of spacetime as a smooth manifold is only an assumption?

Edit: One example for a textbook not addressing the problem seems to be Wald: General Relativity, chapter 2.1 Manifolds. At the beginning of the chapter, Wald states:

"However, in general relativity we will be solving for the spacetime geometry, and we do not wish to prejudice in advance any aspects of the global nature of spacetime structure."

Shortly later follows a general definition of what a manifold is, without reference to spacetime. But there is no qualification of spacetime in this chapter, and also in the later chapters it seems to me that no such assumption of spacetime being a smooth/ continuous/ differentiable manifold is included.