If I understand, you don't glue the insert in the pattern?
So the hole needs to be really tight with the insert, so the insert clock won't fall?
Yes, you do not glue the insert into the wood. It is always best to drill the hole with a forstner bit. A forstner bit drills a flat bottom hole. They are designed so you can drill a hole without drilling all the way through. They come in all the sizes you need for any clock insert. For best results, it is best to use a drill press.
You can cut the hole with the scroll saw if you can cut the hole accurately. If you cut it to large, the insert won't stay in. If you cut to small, you will have to sand or file the hole to fit.
The inserts come with a means to grab the inside of the hole. Some have rubber gaskets for a friction fit, others have built in clips that clip inside the hole.