Using artificial intelligence for 'extreme vetting'? Not on our watch, 50 scientists say

Leading researchers castigated a federal plan that would use artificial intelligence (AI) methods to scrutinise immigrants and visa applicants, saying it is unworkable as written and likely to be "inaccurate and biased" if deployed.

The experts, a group of more than 50 computer and data scientists, mathematicians and other specialists in automated decision-making, urged the Department of Homeland Security (DHS) to abandon the project, dubbed the "Extreme Vetting Initiative".

That plan has its roots in President Donald Trump's repeated pledge during the 2016 campaign to subject immigrants seeking admission to the United States to more intense ideological scrutiny — or, as he put it, "extreme vetting".

Over the summer, DHS published a "statement of objectives" for a system that would use computer algorithms to scan social media and other material in order to flag undesirable entrants — and to continuously scan the activities of those allowed into the US.

The goal, that document stated, was to let computers help determine whether an immigrant "intends to commit criminal or terrorist acts," as well as their likelihood of becoming a "positively contributing member of society".

In a joint letter to DHS on Thursday (16 November), the dissenting researchers called that approach "neither appropriate nor feasible".

In the document, the office of US Immigration and Customs Enforcement said it was seeking a contractor to automate background checks of foreigners seeking temporary or permanent entry to the US.

It outlined plans for mining Facebook, Twitter, LinkedIn, academic websites and other publicly-available internet data, then using AI techniques to analyse it and to keep monitoring those sources.

This contractor would be expected to generate at least 10,000 investigative leads a year that would be forwarded to federal agents, according to the document.

ICE's public presentations to interested vendors haven't been updated since August, raising the possibility that the agency might have scrapped or significantly altered the plan since then.

ICE played down the significance of its summer outreach to contractors in a statement Thursday, saying it was "an opportunity to gather information from industry professionals and other government agencies on current technological capabilities".

But there's been pressure from the White House to implement strict measures quickly.

Hours after the deadly 31 October truck attack in New York City, Trump said on Twitter that he "ordered Homeland Security to step up our already Extreme Vetting Program".

Trump also called for Congress to repeal the diversity visa lottery program that the suspected attacker, an immigrant from Uzbekistan named Sayfullo Saipov, used to enter the country in 2010.

Trump has made repeated attempts to restrict entry into the US through several iterations of a travel ban, all of which have received a tough reception in the courts.

The technology experts, who hail from both academia and big tech firms such as Google and Microsoft, warned that current AI methods aren't capable of evaluating the traits that the government seeks to measure.

"Neither the federal government nor anyone else has defined, much less attempted to quantify, these characteristics," the technologists wrote. "Algorithms designed to predict these undefined qualities could be used to arbitrarily flag groups of immigrants under a veneer of objectivity."

Among the problems is that unlike algorithms that mine the traditional criminal justice system, where there's no shortage of cases, the pool of data that could help predict terrorist leanings is tiny.

"In the national security context it's much harder to investigate the problem because you have a couple cases a year," said UC Berkeley postdoctoral researcher Joshua Kroll, who signed the letter.

One contracting-firm employee who attended an informational ICE meeting in July said the proposed system would simply expand data-management work that private vendors already undertake to help the government vet visa applicants.

It wouldn't give contractors any legal authority to approve or deny immigration benefits, said Bill Carney, who manages federal programs for consulting firm McManis & Monsalve Associates.

But Jacob Shapiro, a Princeton University politics professor who attended one of the meetings as president of investigative analytics firm Giant Oak, said the effectiveness of the vetting system would depend on how it's crafted.

"There are many ways one could meet that statement of objectives, some of which would make the system fairer, more equitable and faster for those seeking to immigrate," Shapiro said. "But some of which could be biased and unfair, as any algorithm can be."

Advocates who coordinated both the technologists' protest letter and a second letter from civic society groups have also privately lobbied potential vendors such as IBM not to participate in the initiative. IBM said this week that its attendance at the government's summer meeting was routine and not a sign that it's planning to make a bid.

"As there is no active project being proposed, it is premature to say whether there is even an opportunity that IBM would consider pursuing," said IBM spokesman Ian Colley.

David Robinson, a Georgetown University law professor and co-founder of Upturn, a newly-formed group that looks at the civil rights implications of new technology, said the extreme vetting project doesn't make sense and doesn't comport with any reasonable set of values.

"This isn't something that anyone should be willing to build," said Robinson, who signed the technologists' letter. "Whatever you think about what the immigration rules ought to be, this is just nuts. And it's nuts dressed up as science."