Looking on the internet deeply has found these results:

Sorry for my late reply is the most popular phrase on the web.

sorry for my late response 142,100 results on the web	Sorry for my late reply 1118,000 results on the web More popular!
Some examples and use cases from the internet:	Some examples and use cases from the internet:
Do forgive me, Mrs Priestley, everyone, for my late arrival. First of all I would like to apologise to Members for my late arrival. Mr President, I apologise profoundly to Mr Le Rachinel for my late arrival in the Chamber. Please pray for my late husband. Mr President, I apologise for my late arrival and I would like to thank you for rescheduling my speech to a later time. I'm sorry for the late response. Mr President, I apologise for my late arrival. Mr President, can I first of all apologise to you, your predecessor and the House for my late arrival.	First of all I would like to apologise to Members for my late arrival. Please pray for my late husband. Mr President, I apologise profoundly to Mr Le Rachinel for my late arrival in the Chamber. Do forgive me, Mrs Priestley, everyone, for my late arrival. Mr President, I apologise for my late arrival. Mr President, I apologise for my late arrival. Mr President, can I first of all apologise to you, your predecessor and the House for my late arrival.

sorry for my late response

142,100 results on the web

Sorry for my late reply

1118,000 results on the web

More popular!

Do forgive me, Mrs Priestley, everyone, for my late arrival.
First of all I would like to apologise to Members for my late arrival.
Mr President, I apologise profoundly to Mr Le Rachinel for my late arrival in the Chamber.
Please pray for my late husband.
Mr President, I apologise for my late arrival and I would like to thank you for rescheduling my speech to a later time.
I'm sorry for the late response.
Mr President, I apologise for my late arrival.
Mr President, can I first of all apologise to you, your predecessor and the House for my late arrival.

First of all I would like to apologise to Members for my late arrival.
Please pray for my late husband.
Mr President, I apologise profoundly to Mr Le Rachinel for my late arrival in the Chamber.
Do forgive me, Mrs Priestley, everyone, for my late arrival.
Mr President, I apologise for my late arrival.
Mr President, I apologise for my late arrival.
Mr President, can I first of all apologise to you, your predecessor and the House for my late arrival.