Not from the Gov.UK website, but this University of South Wales website summarises the situation and presumably applies to babies born in Wales.
Babies born in the UK wrote:Birth in the UK does not automatically make a baby a British citizen. The baby needs to have a parent with British citizenship or settled status in the UK in order to be born British.
If your baby is born in the UK but is not a British citizen, it is quite lawful for him or her to remain in the UK without making an immigration application. However, the baby will need immigration permission to re-enter the UK after any travel abroad, and for babies born to those who hold Student or Tier 4 visas, there are limited instances when the baby can apply for immigration permission as your dependant.
Immigration Health Surcharge
From April 2015 babies born in the UK are not entitled to free healthcare from the age of 3 months old, unless it is emergency treatment. In order to access healthcare parents of babies born in the UK will need to make an immigration application within the first 3 months of the child’s life and pay the Immigration Health Surcharge.
You can of course choose to take out private healthcare insurance for the child if you do not plan for the child to travel outside the UK and so do not plan to apply for a visa.
But either way, the healthcare will be need to be paid for, either through the IHS or private healthcare insurance.
I am not a lawyer or immigration advisor. My statements/comments do not constitute legal advice. E&OE. Please do not PM me for advice.