Fox News pundit Juan Williams on Sunday suggested that conservatives calling for President Obama's impeachment are racist given that they mostly come from white people.

During a "Fox News Sunday" panel, Heritage Foundation CEO Michael Needham said that while conservatives were concerned that Obama is "lawless," Democrats are pushing the impeachment rhetoric.

Williams then tore into Needham, explaining that conservative pundits and some elected officials have been pushing for impeachment.

“You listen to Michael, and you understand why there are lots of Republicans who think, ‘This man’s a demon, this guy’s awful, we got to get this guy out of here any way we can, he’s breaking the law,’” he said.

“And then you come on and say, ‘Oh, no. We’re not talking about impeachment, that’s the Democrats.’ All the Democrats are doing is taking advantage of the fact that you guys have demonized President Obama to this extent," Williams continued.

He then said that some criticize Obama because of his race.

“Lot’s of people see it, especially in the minority community, as an attack on the first black president, think it’s unfair, so it’s going spur their turnout in midterms which is going to be critical in several races," Williams said.

Fox host Chris Wallace then jumped in to ask if Williams really meant to accuse conservatives of racism.

“Well, all I can do is look at the numbers,” Williams responded. “If you look at the core constituency of people, let’s say, who are in tea party opposition support of impeachment, there’s no diversity. It’s a white, older group of people.”

Needham then jumped back in to accuse Williams of "demonizing" conservatives.

“And he might have well just said, ‘And they’re all racists,’” Needham said.