YouNotSneaky asks us to make explicit our moral trade-off between natives' and foreigners' well-being:
Let's put on our annoying-economist hat and consider the question; if you consider a foreign national to be only 1/2 a human being (alright, alright, only 1/2 as "important") as a native citizen, are you justified in opposing immigration? After all, it takes a real jerk to argue that foreign people's welfare should not count at all. Suppose the foreigners are only 1/10th as important? Surely, if natives' welfare counts for ten times as much as that of foreigners, we would be justified in banning immigration since it may adversely affect the wages of the unskilled in US? Well, let's see...
The bottom line is that under plausible assumptions about the rate at which marginal utility is diminishing, one would have to consider the welfare of the native worker to count 20 times or more as much as that of the potential migrant. The main driver of this result is that the gains to the migrant are huge (in view of the wage differential here and abroad) relative to the estimated wage loss for natives.
It is possible to quibble with some of the background assumptions behind this calculation, but I believe the basic message is correct: given how restrictive our current regime is, some relaxation in these restrictions provides huge benefits to migrants relative to distributional effects at home.
UPDATE: Borjas elaborates further on his thoughts on guest workers.