Abstract : The Reinforcement Learning (RL) field has been successfully applied to a wide range of problems in the past years, demonstrating both its versatility and efficiency. The robotic domain in particular benefited from the tremendous progresses made in RL. Despite numerous improvement regarding its sample-efficiency, learning a policy from scratch still requires millions (if not dozens of millions) of interactions with the environment to converge to a high-reward policy. This is usually because the agent does not have any prior information about the task or its own physical embodiment. In general, the canonical way to address and mitigate data-hungriness is to use Transfer Learning (TL), a method leveraging prior experience or knowledge to speed up learning of a distinct but related task. This is currently extensively used in deep learning based Computer Vision or Natural Language Processing. However, finding a practical and general approach for pre-training and transfer learning in the context of RL policies, is still an open research problem. In this work, we explore TL in the context of RL with the specific purpose of transferring policies from one agent to another, even in the presence of morphology discrepancies or different state-action spaces. In other words, we wish to reuse knowledge acquired by an agent (source) on a task to speed up (or even avoid) the learning process of a dissimilar agent (target) on the same task. We highlight the particular challenges raised by the application of TL in the context of cross-robot skill transfer and, inspired by prior works, propose an efficient method to address these matters.