I think what you're saying is right but the explanation is backwards. I think of "science fiction" as being fiction in which the science is held constant--the universe works more or less the same way, but with humanity and their technology exploring a different landscape of the possibility space. So the science is real, not fictional. The plot is fiction.
Of course this assumes we're talking about hard sci-fi, which doesn't include stuff like Star Wars.
Most hard sci-fi breaks the rules too unless it's strictly near future on earth with no space stuff.
Ramscoops don't actually work, you lose velocity collecting the mass. Everything else is subject to the tyranny of the rocket equation unless we're inventing new physics.
Citation for ramscoops not working? Losing velocity is sometimes the point (it works as a sort of magnetic sail / parachute), but I believe the kinetic energy loss is less than the potential energy represented by using the hydrogen as fusion fuel.
I found a paper when I deepdived after reading "A deepness in the sky" but I couldn't find it just now, sorry.
IIRC, it had to do with taking a velocity hit at like 0.9c and then the best you can do is send that atom out at 0.999c out the back. And also the scoop size was gigantic to get mass in sparse interstellar space.
Of course this assumes we're talking about hard sci-fi, which doesn't include stuff like Star Wars.