---
title: "6. The Sertotype Database API: Virtual Crossmatch Simulation"
author: "William Lane MD, PhD, A(ACHI)"
format: 
  html:
    code-fold: false
    toc: true
    theme: cosmo
execute:
  warning: false
  message: false
---

# Virtual Crossmatch Simulation

In this document, we'll demonstrate how to perform a virtual crossmatch simulation using the Serotype Database API. We'll query common HLA alleles based on CIWD3 status and compare their serotypes against simulated recipient antibody profiles.

::: {.callout-warning}
## Clinical Disclaimer

This analysis is for educational purposes only and should not be used for clinical decision making. The virtual crossmatch predictions shown here are illustrative; always follow your institution's protocols and use FDA-approved methods for clinical HLA testing and crossmatching.
:::

## 6.1 Setup Packages

```{r}
#| label: setup-packages

# Clear everything
rm(list = ls())

# Install required packages if not already installed
options(repos = c(CRAN = "https://cloud.r-project.org"))
if (!requireNamespace("httr", quietly = TRUE)) install.packages("httr")
if (!requireNamespace("jsonlite", quietly = TRUE)) install.packages("jsonlite")
if (!requireNamespace("conflicted", quietly = TRUE)) install.packages("conflicted")
if (!requireNamespace("dplyr", quietly = TRUE)) install.packages("dplyr")
if (!requireNamespace("tidyr", quietly = TRUE)) install.packages("tidyr")
if (!requireNamespace("knitr", quietly = TRUE)) install.packages("ggplot2")
if (!requireNamespace("knitr", quietly = TRUE)) install.packages("knitr")
if (!requireNamespace("purrr", quietly = TRUE)) install.packages("purrr")
if (!requireNamespace("dotenv", quietly = TRUE)) install.packages("dotenv")

# Load packages
library(httr)
library(jsonlite)
library(conflicted)
library(dplyr)
library(tidyr)
library(ggplot2)
library(knitr)
library(kableExtra)
library(purrr)
library(dotenv)

# Load environment variables
load_dot_env()

# Resolve conflicts
conflict_prefer("filter", "dplyr")
conflict_prefer("map", "purrr")
```

## 6.2 Set API Key

To query the Serotype Database API, you will need an API Key, which are available for free by signing up for an account at <https://www.serotype.org/user>.

If you are familiar with how to set R environment variables, you can save the key to your .env file as `SEROTYPE_API_KEY=YOUR_API_KEY`, replacing `YOUR_API_KEY` with your actual API key. This is considered best practice because it keeps your private API key separate from your code, enhancing security and making your code easier to share or collaborate on without exposing sensitive information.

Alternatively, you can set the value directly in the code block below by assigning your API key to `apiKeyOverride`. However, be cautious: if you choose to embed your API key in the code, ensure you remove it before sharing the file, as each user must use their own unique API key for security and proper functionality.

```{r}
# Check for the API key in environment variables
apiKey <- Sys.getenv("SEROTYPE_API_KEY", unset = NA)

# Allow manual override of the API key by user here
apiKeyOverride <- ""  # Set this to your API manually if not using environment variables

# Use the override if provided, otherwise use the environment variable value
if (!is.null(apiKeyOverride) && nzchar(apiKeyOverride)) {
  apiKey <- apiKeyOverride
}
```


## 6.3 Fetch All HLA Alleles and Select Common Ones

We need every locus at two-field resolution — that's ~24,000 rows,
well above the API's 5,000-row cap. So instead of calling the API,
we pull the pre-built bulk file from object storage. Lesson 3 covers
this pattern in depth; here's the condensed version.

```{r}
#| label: fetch-all-alleles

# GraphQL URL for later sections of this notebook (queries that stay
# under the 5,000-row API cap).
graphql_url <- "https://www.serotype.org/api/graphql"

# Auto-discover the current version and where the export files live.
meta <- jsonlite::fromJSON("https://www.serotype.org/api/latest-version")
manifest <- jsonlite::fromJSON(meta$manifestUrl)

# Pick the bulk two-field JSON entry from the manifest. `base::url()`
# below opens the URL as a connection; we wrap in gzcon() so jsonlite
# decompresses the .gz stream transparently.
bulk_two <- subset(manifest$files,
                   scope == "bulk" & resolution == "two_field" & format == "json")
stopifnot(nrow(bulk_two) == 1)
df_alleles <- jsonlite::fromJSON(
  gzcon(base::url(file.path(meta$downloadBaseUrl, bulk_two$path)))
)
cat("Fetched", nrow(df_alleles), "rows for HATS version", meta$version, "\n")

# Select two common alleles (CIWD3 == "C") for each locus
selected_alleles <- df_alleles %>%
  filter(ciwd3 == "C") %>%
  group_by(locus) %>%
  slice_head(n = 2) %>%
  ungroup()

# Display selected alleles
kable(selected_alleles, "html", caption = "Selected Common HLA Alleles (CIWD3 = C)") %>%
  kable_styling() %>%
  scroll_box(height = "300px")

# Store unique serotypes from selected alleles
donor_serotypes <- unique(selected_alleles$serotype)
cat("\nNumber of unique serotypes in donor typing:", length(donor_serotypes), "\n")
```

## 6.4 Get All Possible Serotypes

Now let's get all possible serotypes to create our recipient antibody profiles:

```{r}
#| label: query-serotypes
serotype_query <- '{
  getDataElementLists(
    fieldName: serotypes
    returnFormat: flat
  )
}'

resp_serotypes <- POST(
  graphql_url,
  body = list(query = serotype_query),
  encode = "json",
  add_headers(`x-api-key` = apiKey)
)
all_serotypes <- fromJSON(content(resp_serotypes, "text"), flatten = TRUE)$data$getDataElementLists

# Create simulated recipient profiles
set.seed(42) # for reproducibility

# Function to generate random antibodies
generate_antibodies <- function(n_antibodies, available_serotypes) {
  if(n_antibodies > 0) {
    sample(available_serotypes, n_antibodies)
  } else {
    character(0)
  }
}

# Generate 20 recipients with 0-10 antibodies
n_recipients <- 20
recipients <- tibble(
  recipient_id = paste0("R", sprintf("%02d", 1:n_recipients)),
  n_antibodies = sample(0:10, n_recipients, replace = TRUE),
  antibodies = map(n_antibodies, generate_antibodies, all_serotypes)
)

# Display recipient profiles
recipients_display <- recipients %>%
  mutate(antibodies = map_chr(antibodies, ~ paste(.x, collapse = ", ")))

kable(recipients_display, "html", 
      col.names = c("Recipient ID", "Number of Antibodies", "Anti-HLA Antibodies")) %>%
  kable_styling() %>%
  scroll_box(height = "300px")
```

## 6.5 Perform Virtual Crossmatch

Let's perform virtual crossmatches between our donor alleles and recipient antibodies:

```{r}
#| label: virtual-crossmatch

# Display donor typing with serotypes
cat("\nDonor HLA Typing and Serotypes:\n")
selected_alleles %>%
  group_by(locus) %>%
  summarise(
    alleles = paste(allele, collapse = ", "),
    serotypes = paste(serotype, collapse = ", ")
  ) %>%
  kable("html", col.names = c("Locus", "Alleles", "Serotypes")) %>%
  kable_styling()

# Function to check if any donor serotypes match recipient antibodies
check_crossmatch <- function(donor_serotypes, recipient_antibodies) {
  any(donor_serotypes %in% recipient_antibodies)
}

# Perform crossmatch for each recipient
crossmatch_results <- recipients %>%
  mutate(
    positive_crossmatch = map_lgl(antibodies, ~ check_crossmatch(donor_serotypes, .x)),
    prediction = if_else(positive_crossmatch, "Positive", "Negative"),
    antibodies = map_chr(antibodies, ~ paste(.x, collapse = ", "))
  )

# Function to find crossreactive antibodies
find_crossreactive_antibodies <- function(recipient_antibodies, donor_serotypes) {
  intersect(recipient_antibodies, donor_serotypes)
}

# Add crossreactive antibodies analysis
crossmatch_details <- crossmatch_results %>%
  mutate(
    recipient_antibodies = map(antibodies, ~ strsplit(., ", ")[[1]]),
    crossreactive_abs = map(recipient_antibodies, 
                           ~ find_crossreactive_antibodies(., donor_serotypes)),
    crossreactive_summary = map_chr(crossreactive_abs, 
                                  ~ if(length(.) > 0) paste(., collapse = ", ") else "None")
  )

# Display crossreactive antibodies for each recipient
cat("\nCrossreactive Antibodies by Recipient:\n")
crossmatch_details %>%
  select(recipient_id, n_antibodies, prediction, crossreactive_summary) %>%
  kable("html", 
        col.names = c("Recipient ID", "Number of Antibodies", 
                     "Crossmatch Prediction", "Crossreactive Antibodies")) %>%
  kable_styling(bootstrap_options = c("striped", "hover")) %>%
  row_spec(which(crossmatch_details$positive_crossmatch), background = "#ffcccc") %>%
  row_spec(which(!crossmatch_details$positive_crossmatch), background = "#ccffcc")
```

## 6.6 Analyze Results

Let's visualize the relationship between number of antibodies and crossmatch results:

```{r}
#| label: analyze-results
#| fig.height: 8
#| fig.width: 12

# Create summary plot
ggplot(crossmatch_results, 
       aes(x = n_antibodies, fill = prediction)) +
  geom_histogram(binwidth = 1, position = "dodge") +
  labs(title = "Distribution of Virtual Crossmatch Results by Number of Antibodies",
       x = "Number of Antibodies",
       y = "Count",
       fill = "Crossmatch Prediction") +
  theme_minimal() +
  scale_fill_manual(values = c("Negative" = "#ccffcc", "Positive" = "#ffcccc"))

# Calculate and display summary statistics
summary_stats <- crossmatch_results %>%
  group_by(prediction) %>%
  summarise(
    count = n(),
    mean_antibodies = mean(n_antibodies),
    median_antibodies = median(n_antibodies)
  )

kable(summary_stats, "html", 
      caption = "Summary Statistics of Virtual Crossmatch Results") %>%
  kable_styling()
```

## 6.7 Clinical Considerations

Several important points to consider regarding this simulation:

1. **Educational Purpose Only**: This simulation is designed for educational purposes to demonstrate the concepts of virtual crossmatching and HLA compatibility assessment.

2. **Limitations**:
   - Does not account for antibody strength/MFI values
   - Simplified serotype matching without considering cross-reactivity
   - No consideration of complement-fixing vs non-complement-fixing antibodies
   - Random assignment of antibodies may not reflect real-world patterns
   - Uses only CIWD3 common alleles, which may not represent all populations

3. **Clinical Practice**:
   - Always use FDA-approved methods for clinical testing
   - Follow institutional protocols for crossmatching
   - Consider multiple factors beyond simple presence/absence of antibodies
   - Consult with HLA laboratory directors and transplant teams for clinical decisions

4. **Additional Considerations**:
   - Historical crossmatch results
   - Donor-specific antibody monitoring
   - Patient sensitization history
   - Previous transplant history
   - Current clinical status