It means that one's looks doesn't conform to the society's definition of pretty or attractive and even average-looking.
"people in school always told me I was ugly"