Hi! My son started prep this year at 6 (January baby) so if held back your son would be a bit older when starting. I held my son back as we decided he needed more time in kinder to mature and socialise in a playbased environment...he also still struggles regulating his emotions and is very sensitive! I think its so important to start school when more emotionally/socially prepared (although it can still be a BIG adjustment no matter the age or maturity level!). The academics will come easier if the child is coping with other aspects of school and feeling positive about being there.
I would be inclined to listen to the kinder teachers recommendations and also your gut feeling. Do you trust their judgement? You could also talk to the school about it...they should have advice about supporting students as they adjust to prep and school life. My son did receive extra support as he adjusted in term 1...and whenever he gets overtired and emotional, we give him a rest day at home which usually resets his equilibrium!
Remember they are still so little and have lots to learn about coping away from Mum and Dad...baby steps and lots of love and patience!