In fiction, as in life, the unreliable narrator is a narrator who can't be trusted. Either from ignorance or self-interest, this narrator speaks with a bias, makes mistakes, or even lies. Part of the pleasure and challenge of these first-person stories is working out the truth, and understanding why the narrator is not straightforward. It's also one tool an author uses to create an aura of authenticity in his or her work.
The term originates from Wayne C. Booth’s 1961 Rhetoric of Fiction, and though it was a key component of modernism, we find unreliable narratives in classics like Wuthering Heights (in both Lockwood and Nelly Dean) and Jonathan Swift's Gulliver. For an example of how this can work, see an explication of the narrative voice in Chang-rae Lee's contemporary novel A Gesture Life.