This exclamation is not as pious as it sounds. In fact, conservative religious types would probably find it tasteless (not to mention that it breaks the fourth commandment!) and would likely substitute with “Oh, my goodness!” Denizens of the internet probably recognize the version of this phrase that’s become enshrined in meme-dom as “ermahgerd.”