Well, I mean, the justice system isn't just there to enact justice (I know). One of the most powerful aspects of an effective judicial system is the deterrent effect. You threaten a certain punishment for a crime with the idea that the risk of such a punishment outweighs the reward gained by exploitative behaviour. The threat is only effective as a deterrent if it's credible, and that means carrying through with the punishment on those who choose to break the law anyways, even if it seems overboard.
Of course, deterrence isn't the only priority. It's much cheaper and a much more effective deterrent to say, assign the death sentence to just about any violent crime. We don't do this anymore because it could lead to innocent deaths if someone is wrongly convicted, and of course out of simple moral consideration to the criminals themselves. Such a violent judicial system was more common in the past because enforcement was less effective and they needed to balance the reduced risk of getting caught with a greater punishment. Nowadays, we use the greater efficacy of our judicial system to enact less harsh and more expensive punishments.
I do agree with the moral sentiment of your idea though. I find it absurd when drug possession charges lead to jail time, for example. It makes sense that since non violent offenders don't really pose much more of a threat to the physical wellbeing of others that they needn't be separated from society. In that sense, we needn't keep violent offenders in tiny cells when we could simply just put them on an island and have them grow their own food and build their own living quarters (Australia anyone?). At least if the idea of jail was simply to keep them from harming other citizens.
But like I've said earlier, the idea is that it is an actual punishment. You have to take away the things people value, such as their freedom, in order for a punishment to have any sort of effect. In that vein, maybe we ought to deny visitation rights to prisoners. It's the actual punishment that acts as the deterrent. Jailing Madoff for 100 years isn't meant to change anything that Madoff did, it's to warn any potential Madoffs that if they try to do the same thing, they risk his fate.