There were years of our ministry that I foolishly made the church members who showed up feel beat up on behalf of those who didn't. Then I realized that was really stupid and served no purpose. I would ramble about people who didn't show up to the faithful ones who sat in front of me. How much sense did that make? None. Totally counter-productive. I'm sorry I ever did that and made changes to remedy that.
One thing the Lord told me years ago was, "move with the movers." My goal is to pour into those who have made themselves available and not get in the throes of depression over those who haven't.
Some days I do better with that goal than others. :P