On triangle inequalities of correlation-based distances for gene expression profiles

Background: Distance functions are fundamental for evaluating the differences between gene expression profiles. Such a function would output a low value if the profiles are strongly correlated—either negatively or positively—and vice versa. One popular distance function is the absolute correlation distance, d_a= 1 - |ρ| , where ρ is similarity measure, such as Pearson or Spearman correlation. However, the absolute correlation distance fails to fulfill the triangle inequality, which would have guaranteed better performance at vector quantization, allowed fast data localization, as well as accelerated data clustering. Results: In this work, we propose d_r = √1 - |ρ| as an alternative. We prove that d_r satisfies the triangle inequality when ρ represents Pearson correlation, Spearman correlation, or Cosine similarity. We show d_r to be better than d_s = √1 - ρ², another variant of d_a that satisfies the triangle inequality, both analytically as well as experimentally. We empirically compared d_r with d_a in gene clustering and sample clustering experiment by real-world biological data. The two distances performed similarly in both gene clustering and sample clustering in hierarchical clustering and PAM (partitioning around medoids) clustering. However, d_r demonstrated more robust clustering. According to the bootstrap experiment, d_r generated more robust sample pair partition more frequently (P-value < 0.05). The statistics on the time a class “dissolved” also support the advantage of d_r in robustness. Conclusion: d_r, as a variant of absolute correlation distance, satisfies the triangle inequality and is capable for more robust clustering. © 2023, The Author(s).