NEW YORK (Fortune) -- What did Larry Summers really think of the Winklevoss twins?

"Rarely, have I encountered such swagger, and I tried to respond in kind," the former president of Harvard said in an interview at Fortune's Brainstorm Tech conference.

Cameron and Tyler Winklevoss were at Harvard at the same time that Mark Zuckerberg launched Facebook, and they had come to Summers for help in their fight for a piece of the action.

Summers dismissed them, a scene dramatized in the movie the "Social Network."

Summers didn't try to dispel the portrayal.

"One of the things you learn as a college president is that if an undergraduate is wearing a tie and jacket on Thursday afternoon at three o'clock, there are two possibilities. One is that they're looking for a job and have an interview; the other is that they are an a**hole. This was the latter case."
6 hot dormroom startups

The Winklevoss twins have taken dozens of runs at Facebook in court since 2004. The brothers and a third Harvard classmate, Divya Narendra, have alleged that Zuckerberg had stolen the idea for Facebook from them.

Facebook promptly countersued the trio and their own social networking site, ConnectU, alleging that they had hacked into Facebook to steal data and spam users. The legal volley of claims and counterclaims went back and forth until 2008, when the Winklevosses and Narendra signed a settlement valued at $65 million, part in cash and part in Facebook stock.

But the trio later tried to pull out of the deal, alleging that Facebook had misrepresented the value of its stock. A judge ruled against them and forced the settlement to go through, but the Winklevosses and Narendra took the case to an appeals court.