Police were called to the church on New South Head Road in Rose Bay about 6.30pm yesterday.
Officers spoke to a 36-year-old man and found him to be in possession of capsicum spray and a multi-tool with a knife.
He was arrested and taken to Waverley Police Station and charged with possessing a prohibited weapon and a knife.
The Bangor man was granted conditional bail to appear at the Downing Centre Local Court on July 26.
