hardcore few

The small number of people left at the end of a party or a night out.

At midnight, after all the boring people had made their excuses and gone home, the hardcore few partied 'til dawn.