There are to many "good" units to name them all, but I think that most players will agree that you need a lot of vanilla infantry. (2-4 points line troops, for normal builds. Then you'll want 2 or 3 of all the artillery, machine guns and commanders. (Except for a few like the Chinese or Polish ones, most you'll use pretty often!)
For infantry, the "nice" ones are Hero's, Paratrooers and Elite's (5/5 defense line troops like the Ranger, SNLF Fanatic, and SS Panzergrenadier to name a few). For MG's most nations have 1, and they all are pretty good, even the Chinese one!
Commanders vary, but SS Haupsturmfuhrer (Hamster), Vigilant Lieutenant, Inspiring Lieut., Red Devil, Eagle Eye NCO, and Imperial Sgt. are among the best, the rest are often only of limited use. Artillery are good, but being slow and weakly armored, they die easily! Good ones include 7.5cm LelG, 88mm Flak, Quad 50, Bofers, Pak 40, 17 pounder, Pack Howitzer, and 20mm Flak 38, to name a few.
For tanks, most like one of each at least, but that's often hard to do with the "rare" rares (Panthers, Special Shermans, Soviet Heavies). You'll want a few staple tanks like the Panzer IV and III's for the Germans, T34 and BT tanks for the Soviets, and Stuarts, Shermans and Matilda's for UK/US. Then you'll want some Mobile Artillery like the Brumbar, SiG 33, Sherman 105, Sherman Caliope, ISU-122, Katyusha, M12 MGC, Hummel and Wespe. These are nice for taking out dug in infantry. No doubt you'll use a lot of tank destroyers, since they're really nice, so the best (for their points) are the Elite Stug III, Sherman Vc 17 pounder, SS Jagdpanther, StuG III, Semovente 90mm, M16 GMC, Hellcat. There are also heavier, but more expensive Jagdtigers and SU series for the German and Soviets. Medium-Heavy tanks like the Panthers, Comets, KV-1's, and Sherman E8's are sometimes hard to find, but are most often worth the points. Then comes the hardest to find... the heavy tanks. Tigers of all varieties, Pershings, Centurions, IS series (IS-3 and IS-2 Vet) and such like them. Often Very hard to find, they can be nice to have in you army for the scare factor, but most players like multiple smaller tanks instead!
Lastly, all SS, Veteran, Elite, SNLF, ect.. units are very nice to have, either tanks of vehicles. And units from the first 5 sets (out of print) are often hard to get as well, so they are "rare" in more ways then one!
The Reserves set is the hardest to get, so almost any units from it are nice, but not especially needed.
All in all, getting a well rounded team for how much you can afford is your best bet, don't try for everything, but get what's needed to play a solid army for your favorite nations.
If you're wondering where to get all these units, checking online stores or your LGS might work, if not check ebay to find the latest Axis and allies miniatures deals. Hope that helped!