Coincidentally enough, before reading your post but after seeing the subject line, my first thought was The First Time Ever I Saw Your Face.
If I had to choose a second song, it would probably be another performance from a female vocalist: Dusty Springfield's You Don't Have To Say You Love Me.
I'd mention Hey Jude, but that shouldn't be considered IMO because it shouldn't have been released.
BTW Eddy Arnold wrote You Don't Know Me, though I believe Charles' version is the one most people know.
We seem to have a lot of 'least favorite' threads, don't we?