Nothing to do with morals? How about: It's acceptable to punish the innocent if it also punishes the guilty (Noah, and putting aside the fact that death seems to be the only "justice" in the old testament most of the time). Or that your own beliefs are always superior and that you are inheritablly right, others are just deceived idiots that you don't need to listen to (Creation, that's why light came *before* the sun, to claim that sun-worshippers of the time were not only not worshipping the source of light [since it pre-dated the sun], but that it was also created as a mere object inferior to god). Or that it's perfectly acceptable to punish people for crimes others committed and over which they had no power (Eve as source of sin, cause of pain in childbirth). Or that there is only a single way to do things, to live, to behave, and all who deviate are bad and must be persecuted (gays)?
The Bible is full of hateful stuff. I'm not saying that there are no good things in it, the saying of Jesus are, I believe, good, despite being twisted by the ages for various political reasons.
While Church can, on occasion, be entertaining (I enjoy deciphering the pagan roots of the myths and practices), it isn't something I'd do frequently. Not to mention I find priests to be rather unpleasant people, and the ratio of pedophiles among priests is significantly higher than among non-priests...