Differences in concordance index results via lifelines and scikit-survival

[leandervaneekelen]

Hi there! I am taking a look at using either lifelines and sksurv for some survival analysis, but I just discovered that I get different results for the same data on a concordance index.

import numpy as np
from lifelines.utils import concordance_index
import lifelines
from sksurv.metrics import concordance_index_censored

# Generate some sample data
np.random.seed(42)
y_true = np.random.randint(1, 100, 100)
y_pred = np.random.randint(1, 100, 100)
event_observed = np.random.randint(2, size=100).astype(bool)

c_index_lifelines = concordance_index(y_true, y_pred, event_observed)
c_index_sksurv, _, _, _, _ = concordance_index_censored(event_indicator=event_observed, event_time=y_true, estimate=y_pred)

print(f"c-index lifelines: {c_index_lifelines}")
print(f"c-index sksurv: {c_index_sksurv}")
print(f"1 - c-index sksurv: {1 - c_index_sksurv}")

print(f"\nLifelines version: {lifelines.__version__}\n")
print("sksurv version information:")
print(sksurv.show_versions())

I get the following results:

c-index lifelines: 0.533683800623053
c-index sksurv: 0.466316199376947
1 - c-index sksurv: 0.533683800623053

Lifelines version: 0.30.0

sksurv version information:
SYSTEM
------
Platform          : Linux-5.15.0-124-generic-x86_64-with-glibc2.35
Python version    : CPython <module 'platform' from '/usr/local/lib/python3.11/platform.py'>
Python interpreter: /usr/local/bin/python3.11

DEPENDENCIES
------------
scikit-survival   : 0.22.2
scikit-learn      : 1.3.2
numpy             : 1.23.5
scipy             : 1.10.1
pandas            : 2.2.3
numexpr           : 2.10.0
ecos              : 2.0.13
osqp              : 0.6.5
joblib            : 1.2.0
matplotlib        : 3.7.1
pytest            : 7.3.1
sphinx            : None
Cython            : 0.29.34
pip               : 23.1.2
setuptools        : 67.8.0
None

Am I misunderstanding something about the implementation of the concordance index in scikit-survival? In the extreme case, where I set y_pred to be y_true, lifelines gives me a concordance index of 1.0, while sksurv gives me a concordance index of 0.0. Why this inverse relationship?

[sebp]

The semantics are different. For lifelines y_pred is a score for survival, i.e. a higher score indicates longer survival, for sksurv y_pred should be a risk score, i.e. higher values indicate shorter survival. As you wrote, you can also simply use 1 - c_index to make the results equivalent.

[leandervaneekelen]

Thanks, clear answer.