A pastor, and any Christian for that matter, should recognize that 'coming to church' to worship is a facet of worship that honors God. It is not the only worship and it is biblically encouraged.
Also, unfortunately, Worship has come to mean just the music and singing type praise to many...