Spammers flagging emails and showing in to-dos

Is there a way to prevent people (spammers) emailing me, from setting a flag on the email?  It seems weird that not only can someone else flag an incoming email, but that it's also flagged when in the Junk folder.  This then adds their spam/phishing email to my to-do app as well.