Computational phenotyping is a process to detect patient cohorts from electronic health records (EHRs) and serves as a foundation for clinical research, population health assessment, and decision support. The rapid proliferation of phenotyping algorithms for the same conditions has created major challenges for researchers to choose or reuse existing definitions. Many algorithms labelled under the same disease name target different clinical definitions, diagnostic criteria, or patient subgroups. This can lead to inconsistent cohort identification and decreased reproducibility in studies. The PhenoFit framework was developed to help researchers determine whether a phenotyping algorithm is appropriate for specific settings and whether it can be successfully reused in a new setting.
The aim of the PhenoFit framework is to determine whether an algorithm is fit for purpose, which means it reliably identifies the intended population with performance characteristics suitable for the application of the investigators. It also evaluates fitness for reuse, indicating whether the algorithm can be implemented at new sites while maintaining expected performance and generalizability. Â
The framework helps investigators to navigate uncertainty in existing phenotype definitions and promotes efficient and reproducible cohort construction. The PhenoFit method comprises sequential evaluation of three criteria: intended population, appropriate performance, and fitness for reuse, which includes implementability and generalizability.
Intended population assessment involves examining how phenotype was originally defined and validated, and recognizing that multiple algorithms with the same disease label may target distinct clinical definitions or subtypes. Proper performance assessment requires reviewing the reported validation metrics like positive predictive value (PPV), specificity, or sensitivity, and determining whether these metrics align with the intended application of the users, while accounting for biases from the original validation context. The investigator must assess the suitability of the algorithm for reuse only after it is determined to be proper for the intended application. This involves determining whether the method is adequate, whether necessary data and technology are locally available, and whether the algorithm can generalize to new patient populations and care situations.
Real-world application of PhenoFit has revealed several challenges. Many algorithms fail because their target population is unclear or mismatched with the investigator’s cohort. Hidden criteria, like requiring an ejection fraction below 40% for a heart failure algorithm, may limit detection to a specific subgroup. Underspecified algorithms often fail the implementability test due to missing thresholds, insufficient model specification, nonstandard local data formats, or the unavailability of natural language processing (NLP) components.
Generalizability can be compromised by differences in population structure, care processes, or data filtering between the development and the reuse sites. Algorithms developed in a specialized clinical environment, an enriched research cohort, or a biobank restricted populations often perform poorly when applied to structurally different or broader EHR systems. PhenoFit highlights the need for basic local validation and more extensive assessment when original performance characteristics are unavailable, as generalizability cannot be reliably predicted a priori.
The PhenoFit framework provides a structured method to evaluate the reliability of phenotyping algorithms in new research and clinical settings. Differentiating between fitness for purpose and fitness for reuse helps researchers detect mismatched populations and potential implementation challenges. While PhenoFit cannot eliminate all uncertainty, it offers a systematic approach to managing it. Combined with enhanced reporting guidelines and standardized metadata in phenotype libraries, PhenoFit can improve reproducibility and enable more accurate identification of patient cohorts in the healthcare system.
Reference: Wiley LK, Rasmussen LV, Levinson RT, et al. PhenoFit: a framework for determining computable phenotyping algorithm fitness for purpose and reuse. J Am Med Inform Assoc. 2025;ocaf195. doi:10.1093/jamia/ocaf195
