The remarks were widely reported by US media.
The comment, made during talks on an immigration deal, was apparently in reference to people from Haiti, El Salvador and African countries.
The White House made no attempt to deny the comment.
In recent weeks, the Trump Administration has been trying to limit the number of family members of immigrants who can enter the US, and has moved to end the protected status of thousands of immigrants already in the US.
Mr. Trump’s reported remark came as lawmakers from both parties visited him on Thursday to propose a bipartisan immigration deal.