On Friday, two federal judges refused to step in as lawyers for the men launched a desperate legal campaign to prevent their deportation, even as one judge said the case raised legitimate concerns.