Self-supervised learning deals with problems that have little or no available labeled data. Recent work has shown impressive results when underlying classes have significant semantic differences. We will discuss strategies to tackle to enable learning from unlabeled data even when samples from different classes are not prominently diverse. We approach the problem by leveraging novel ensemble-based clustering strategies where clusters derived from different configurations are combined to generate a better grouping for the data samples in a fully-unsupervised way. We will see results for Person Re-Identification and Text Authorship Verification but the techniques are useful in other applications as well. Moreover, we also detail recent efforts on Causal Analysis to refine AI methods.