As long as you have a stable pack, backpack or rucksack on your character, then every 20 lbs. + your Strength Attribute, will cost you +1 Vitality per Physical Action that you commit.
It is typical of an individual about to engage in a fight, to put down their pack. No warrior would go into a Melee fight with extra pounds tied to their back.
If at any point you carry something in your hands or arms, the Vitality would increase.