You correctly state that your goals are in conflict. By design, retention rules will win in a conflict with deletions. They wouldn't be much use otherwise. You need to decide which goal matters more to your organization and it may be different on a per user or group of users basis. Typically retention goals trump deletion goals because retention is driven by legal/compliance/regulation. At some lifecycle stage, the reverse becomes true and items that are X years old are deleted because legal/compliance/regulation retention requirements have been met and the continued retention actually becomes a potential risk.
One compromise between your goals would be to adjust lifecycle rules so items are moved to an archive mailbox where they are retained, so that space is freed up in the user's primary mailbox. The default rules for EOL are here. https://docs.microsoft.com/en-us/exchange/security-and-compliance/messaging-records-management/defau...
You could reduce the default 730 day move to Archive to be a shorter period.