Yup. In theory they could say, "You smell bad, no cake." Or "We don't serve people with freckles." In most states, protected classes are rigidly defined. You probably wouldn't want to try it in California, where the state holds a looser interpretation, but in Virginia? Sure, you can discriminate against anything you can think of that isn't a protected class. In Virginia the protected classes are: race, color, religion, national origin, sex, disability, and familial status.
Within US law, public accommodations are generally defined as entities, both public and private, that are used by the public. Examples include retail stores, rental establishments and service establishments, as well as educational institutions, recreation facilities and service centers. Essentially, if you sell things to people, and anyone can be a customer (you're not a private club) and you aren't an actual religious institution, then you're a public accommodation. What's your business and state?