Shrink fit holders and extensions often come with a big through hole.
Its primary use is to allow the shank be knoked out from he back should the tool ever snap off. It is also used to supply coolant for CTS machines.
Unfortunately said hole affects rigidity of the holder making it more likely to chatter leaving bad surface finish and badly affecting tool life.
There is however an old trick to prevent or minimize the chatter.
All you have to do is pack that hole with some thick grease.
Don't forget to cap off the oppening so that grease does not escape when the tool is spinning.
Here are several photos of surface finish before and after grease application. All cutting parameters were exactly the same in both cases.