Theres no mechanic provided in the game, but a simple pad and paper can track how many units come from each territory. In the event of a retreat, you cannot retreat more units than came from that territory when declaring an attack.
Of course, thats just another layer of complexity and time required before conducting combat. But that seems the most logical rule if you don’t care for the gamey version written in the rules where you can send 1 Infantry in from one side and send 10 infantry from another and effectively move the 10 infantry two spaces.