It's the law of big numbers. Each day I spend maybe 1-2 minutes dealing with my spam. Multiply that by millions of people each day and you get massive business (and personal) time losses which equates to lost productivity/money.
Also, spam encourages or requires other illegal activities -- money laundering, fraud, violation of health standards, piracy, identity theft. And the profits are likely to be fed into other questionable businesses.