How Effective Is Phishing Advice? We Put It to the Test!
Phishing attacks are everywhere, and let’s be real—users are often the last line of defense. But here’s the problem: most advice on spotting phishing emails is vague and not very useful in real-life situations. The emails that slip past spam filters tend to be highly sophisticated, taking advantage of the differences between how humans and computers process information.
So, what kind of advice actually helps users make better decisions? That’s what we set out to explore in our study. We tested three types of guidance:
- Generic advice (basic tips you often see online)
- Perfect advice (accurate, tailored tips based on email features)
- Realistic advice (similar to perfect advice but with occasional inaccuracies, like what AI systems might generate)
We surveyed 489 participants on Prolific to see how well they could detect phishing emails with and without guidance.
The results?
Advice tailored to the email significantly improved accuracy compared to generic tips. However, when the advice contained errors, it sometimes misled users, making them less accurate in spotting phishing attempts.
The takeaway?
Personalized, accurate advice is a game-changer—but if it’s wrong, it can do more harm than good. As phishing attacks evolve, we need to ensure the guidance we provide is both relevant and reliable.
For more details, visit Rephrain Project or read our CHI 2025 paper.