I have had more moments of speaking to crickets than I personally enjoy. I have learned to enjoy the process and the 'practice' I get from speaking my truth even to an empty room.
Don't get to upset about a lack of initial traction in getting people listening to your podcast. There are a lot of podcasts to choose from, and not enough capacity to for many people to get past the 8-second mark.
It is usually some side issue that scuttles your progress in podcasting before the real podcast problems can begin to creep into the process.