There was an article today in the NYTimes about click fraud. The piece is pretty balanced: it's happening but the search engines are dealing with it. I think that's the reality in the marketplace; it's there but it's not terribly widespread and there are pretty strenuous, though not widely understood, efforts to combat it.
So the real danger around this issue is PR.
The attendence at SES confirmed anecdotally that the vast majority of marketers have not tried or just now starting to get into SEM. Large companies with sophisticated ad agencies can deal rationally with the perceived problem and it's not going to deter them in the same way it might frighten an SME.
In other words, I believe the perception of a problem with "click fraud" or any suggestion that clicks aren't real leads may create a PR problem in driving deeper into the SME market ("How do I know that's not my competitor trying to run up my bill or just some 'random insomniac'?). Pay per call doesn't face quite the same type of concern, although there may be other issues in that space.
However, I think the numerous intermediaries and adoption channels emerging will mitigate that potential "chilling effect" of click fraud–so will consumer behavior. SMEs recognize the growing importance of online marketing vis-a-vis consumer usage of the Internet.
But the PR problem should not to be dismissed and the engines/networks need to get out in front of it so the perception of click fraud doesn't become far worse than the reality.