One way to do it:
There’s two main changes you have to make.
In vars.php, find this line:
$wp_smiliessearch[] = $smiley;
and replace it with this one:
$wp_smiliessearch[] = '/'.$smiley.'/';
Then, in functions-formatting.php, find this:
$content = str_replace($wp_smiliessearch, $wp_smiliesreplace, $content);
and replace it with this:
$content = preg_replace($wp_smiliessearch, $wp_smiliesreplace, $content,5);
And there you go. Note that this change is kinda dirty, and it will apply to your own posts as well, limiting the number of replacements. This also doesn’t prevent them from putting all the smilies in, it just prevents them from becoming images later.
I could come up with a better solution, but it’s a bit late right now. ??