ReMARK Evaluation

There are three main objectives of the protocol. First, we will evaluate the added clinical and cost effectiveness of parent-targeted motivational aids (reminder/recall and phone-based MI) alone and when combined with community-targeted healthcare access assistance beyond the effects of clinician-targeted training. Second, we will estimate the differential effectiveness of the implementation strategies by patient-level factors (age, race/ethnicity, sex, distance from home to clinic, social vulnerability). Third, we will measure moderation of implementation strategy effectiveness by clinic-level factors (HPV vaccination priority, resources, clinic visit types, scheduling practices, and implementation success).

Within 11 rural North Central Florida counties, we will evaluate the layering of evidence-based implementation strategies that progressively addressing clinician, parent, and healthcare access barriers faced by rural communities on HPV vaccination rates among 9* to 12-year-olds. To best address our main question of whether layering complementary strategies continues to increase effects on HPV vaccination, we will test our hypotheses with a three-arm cluster randomized study design of nested strategies. The proposed nested study design optimizes evaluation, causal inference, and scientific rigor by putting the maximum number of clinics towards addressing the layering of strategies. Randomization will occur at the clinic level. All clinics will receive implementation

Strategy A: clinician-targeted recommendation training. A random 20 of 30 of clinics will also receive facilitation of parent-targeted motivational aids (B) for an implementation strategy package of A+B. Finally, a random half of the clinics who receive A+B will also receive community-targeted healthcare access (C) for a total implementation strategy package of A+B+C. This equates to a three-arm cluster randomized trial in which 10 clinics receive clinician* targeted recommendation training alone (A), 10 clinics receive clinician-targeted recommendation training and parent-targeted motivational aids (A+B), and 10 clinics receive clinician-targeted recommendation training, parent-targeted motivational aids, and community-targeted healthcare access (A+B+C).

There are three main objectives of the protocol. First, we will evaluate the added clinical* and cost-effectiveness of parent-targeted motivational aids (reminder/recall and phone-based MI) alone and when combined with community-targeted healthcare access assistance beyond the effects of clinician-targeted training. Second, we will estimate the differential effectiveness of the implementation strategies by patient-level factors (age, race/ethnicity, sex, distance from home to clinic, social vulnerability). Third, we will measure moderation of implementation strategy effectiveness by clinic-level factors (HPV vaccination priority, resources, clinic visit types, scheduling practices, and implementation success).

We propose a multilevel approach, incorporating evidence-based strategies that address clinicians' recommendations, parents' hesitations, and access to healthcare is needed to reduce the HPV-related cancer disparities in rural areas. Within 11 rural North Central Florida counties, we will adapt, implement, and evaluate the layering of evidence-based implementation strategies by progressively addressing clinician, parent, and healthcare access barriers faced by rural communities on HPV vaccination rates among 9* to 12-year-olds. To increase reproducibility, we propose a conceptual framework, specified implementation strategies, randomization, adaptive and sequential implementation strategies, process and outcome measures, appropriate statistical analysis, and conservative power calculations.

To best address our main question of whether layering complementary strategies continues to increase effects on HPV vaccination, we will test our hypotheses with a three-arm cluster randomized study design of nested strategies. The proposed nested study design optimizes evaluation, causal inference, and scientific rigor by putting the maximum number of clinics towards addressing the layering of strategies. Randomization will occur at the clinic level. All clinics will receive implementation strategy A: clinician-targeted recommendation training. A random 20 of 30 of clinics will also receive facilitation of parent-targeted motivational aids (B) for an implementation strategy package of A+B. Finally, a random half of the clinics who receive A+B will also receive community-targeted healthcare access (C) for a total implementation strategy package of A+B+C. This equates to a three-arm cluster randomized trial in which 10 clinics receive clinician-targeted recommendation training alone (A), 10 clinics receive clinician-targeted recommendation training and parent-targeted motivational aids (A+B), and 10 clinics receive clinician-targeted recommendation training, parent-targeted motivational aids, and community-targeted healthcare access (A+B+C).

Random Assignment:

Clinics will be randomized into three equal arms. Ten clinics will receive clinician-targeted recommendation training alone (A). Ten clinics will receive clinician-targeted recommendation training and parent-targeted motivational aids (A+B). Ten clinics will receive clinician-targeted recommendation training, parent-targeted motivational aids, and community-targeted healthcare access (A+B+C).

Delivery of interventions:

As delivery of interventions is conducted at the clinic level and all interventions are evidence-based, these interventions will be conducted as quality improvement.

Clinician-targeted Recommendation Training. All participating clinicians and nurses at all 30 participating clinics will be offered trainings via a one-hour Zoom video conference. Structured after and intending to enhance and expand the NCI Evidence-based Program by focusing on empathy as central to the conversation and including 9* to 10-year-olds who are not eligible for Tetanus, Diphtheria, and Pertussis or Meningitis ACWY, Drs. Staras (PI), Thompson, and Bylund developed and have conducted preliminary testing of the C-LEAR (Counsel, Listen, Empathize, Answer, Recommend) training as part of our R37 (Staras PI) (Section C.2.2). C-LEAR incorporates concepts from Motivational Interviewing, facework theories, and patient-centered communication.

Parent-targeted Motivational Aids. Twenty of the 30 clinics will be offered the parent-targeted motivational aids implementation strategy which includes reminder/recall and parent-targeted phone* based MI.

Reminder/Recall. We will facilitate clinic staff sending these customized reminder/recall messages via one of three methods (postcards, text messages, or phone calls) to increase clinic adaptability to existing services and workflow. We will assist clinics in identifying 9* to 12-year-olds who have not received the HPV vaccine though the Florida state immunization registry and their EHR systems. Reminder/recall for initiation and follow-up doses will be sent at three meaningful milestones:

(1) two weeks before an adolescent's birthday, (2) two weeks prior to fall school start, and (3) within the first weeks of January.

Parent-targeted Phone-based MI. Following our previously feasible and acceptable process for phone-based MI, parents who have been sent at least two reminder/recall messages for initiation or follow-up without response (contact or clinic visit) after one month will be eligible for phone-based MI. Based on our pilot study, we anticipate MI sessions will last approximately ten minutes.

Community-targeted Healthcare Access. We will address social determinants faced by

Families living in rural areas in three ways: (1) linking patients to transportation assistance, (2) providing a mobile vaccination clinic, and (3) linking patients to health insurance navigators. The activities will be managed by the UF Health Cancer Center's Community Outreach and Engagement Community Navigation office as these services are consistent with their current offerings. UF Pediatrics and UF CTSI will offer mobile vaccine clinics as part of standard of care 40 times during the intervention period (4 times for each of the 10 assigned clinics). The mobile vaccination clinics will offer all adolescent vaccines (HPV, Tetanus, Diphtheria, and Pertussis, Meningitis ACWY, Influenza

• in season, and COVID-19). Adolescents will receive the vaccines free of charge: billed to insurance or the Federal Vaccines for Children Program which covers vaccines for all children 18 years and under who are eligible for Medicaid or are uninsured, underinsured, or Native American or Alaskan Native. All administered vaccines will be reported to the Florida immunization registry, and, thus discoverable by clinics statewide including participating clinics.

Statistical Analysis and Design:

We will randomize 30 participating clinics to three arms (10 clinics per study arm). We will use constrained randomization to reduce imbalance among a priori determined confounders (i.e., baseline vaccination rate). The statistical team will be blinded to study arm assignments.

Clinical Effectiveness Analysis. The proposed group randomized trial consists of clustered data with patients nested within clinics. Any statistical analysis that ignores lack of independence due to nesting will be subject to inflated type-1 errors. To account for the clinic level clustering effect, intervention effects on HPV vaccine initiation will be estimated with generalized linear mixed models. We describe briefly our general strategy for specifying both the covariance and mean models for proposed analyses in Aims 1 through 3.

In this trial, cluster sampling generates exchangeable observations, and therefore compound symmetric covariance for patients nested within clinics is appropriate. Implementing the covariance model can be done for our binary outcome (HPV vaccine initiation) in PROC GLIMMIX in SAS by specifying a random intercept for clinic. In addition to efficiently accounting for non-independence, a generalized linear mixed model approach provides the flexibility to specify different link and variance functions based. Our primary specification will be a log() link and binomial variance to directly estimate the risk ratio. Should model convergence issues arise, we will estimate a logit model on the odds scale.

Inclusion Criteria:

• Adolescents Age 8* to 12-years-old Visited participating clinic in past year or during study No records of receiving both HPV vaccine doses

• Providers Provide 9* to 12-year-old patients primary care Practice at a participating clinic

Exclusion Criteria:

• Adolescents Out of age range

• Providers Unwilling to consent