Recommendation systems connect users to items and create value chains in the internet economy. Recommendation systems learn from past user-item interaction histories. As such, items that have short interaction histories, either because they are new or not popular, are disproportionately under-recommended. This long-tail item problem can exacerbate model bias, and reinforce poor recommendation of tail items. In this paper, we propose a novel training algorithm, \textit{biased user history synthesis}, to not only address this problem but also achieve better personalization in recommendation systems. As a result, we concurrently improve tail and head item recommendation performance. Our approach is built on a tail item biased User Interaction History (UIH) sampling strategy and a synthesis model that produces an augmented user representation from the sampled user history. We provide a theoretical justification for our approach using information theory and demonstrate through extensive experimentation, that our model outperforms state-of-the-art baselines on tail, head, and overall recommendation. the source code is available at https://github.com/lkp411/BiasedUserHistorySynthesis.