Otherwise the vampires would just send Renfield in and have him invite them inside

This literally happens. In Seward’s diary entry of 3 October (chapter 21), Seward quotes Renfield (spoilers, for anyone participating in our chronological Dracula bookclub!):

and before I knew what I was doing, I found myself opening the sash and saying to Him: ‘Come in, Lord and Master!’ The rats were all gone, but He slid into the room through the sash, though it was only open an inch wide—just as the Moon herself has often come in through the tiniest crack and has stood before me in all her size and splendour.

And of course, then the important detail: once a vampire has been invited inside, they implicitly have an invitation to return:

All day I waited to hear from him, but he did not send me anything, not even a blow-fly, and when the moon got up I was pretty angry with him. When he slid in through the window, though it was shut, and did not even knock, I got mad with him. He sneered at me, and his white face looked out of the mist with his red eyes gleaming, and he went on as though he owned the whole place, and I was no one.