My suggestion, or, "The way I do it".......
I usually DRILL the inside circle to get a perfectly round hole. I use a forester drill bit in the drill press to get a nice, smooth hole with as little tear-out as possible. If I don't have one of the EXACT size, I drill using the next size smaller and file or sand the inside to the final dimension.
As for the outer circle, I use the scroll saw to cut just a bit OUTSIDE the line and then sand down to the final dimension.
Of course, it depends on what you are cutting the circle for and if it is a complete ring or maybe has something protruding to the inside or outside of it.
As for using JUST a scroll saw and, depending on the diameter of the circle, I cut just off of the line so that it can be sanded or otherwise worked down to final dimension and/or appear to be perfectly round. Just go slow, take your time, breathe, realize that NOTHING IS PERFICT, it just LOOKS that way!
![Grin ;D](https://www.stevedgood.com/community/Smileys/default/grin.gif)
At any rate, accept the fact that is going to take some sanding and hand work and it may not be exact but, it will LOOK GOOD to everyone else.
![Wink ;)](https://www.stevedgood.com/community/Smileys/default/wink.gif)
Rog