In one-class recommendation systems, the goal is to learn a model from a small set of interacted users and items and then identify the positively-related (i.e., similar) user-item pairs among a large number of pairs with unknown interactions. Most loss functions in the literature rely on dissimilar pairs of users and items, which are selected from the ones with unknown interactions, to obtain better prediction performance. The main issue of this strategy is that it needs a large number of dissimilar pairs, which increases the training time significantly. In this paper, the goal is to only use the similar set to train the models and discard the dissimilar set. We highlight three trivial solutions that the models converge to when they are trained only on similar pairs: collapsed, dimensional collapsed, and shrinking solutions. We propose a hinge pairwise loss and an orthogonality term that can be added to the objective functions in the literature to avoid these trivial solutions. We conduct experiments on various tasks on public and real-world datasets, which show that our approach using only similar pairs can be trained several times faster than the state-of-the-art methods while achieving competitive results.