Chapter 18 Appendix A: Responsible AI Checklist

As we conclude this book, it is crucial to remember that technical skills are only half of the equation. Data science has real-world consequences. Before deploying any model, analysis, or reliable pipeline to production, use this checklist to ensure your work is robust, fair, and transparent.

This checklist is designed to be actionable for R users, pointing to specific packages and practices where applicable.

18.1 Data Quality & Lineage

• “Garbage in, garbage out” applies to ethics as well as accuracy.*

• Provenance: Do I know exactly where this data came from? Is the source trustworthy?

• Consent & Privacy: Was the data collected with consent? Does it contain Personally Identifiable Information (PII)?

  • Tip: Use packages like introdat or custom scripts to scan for patterns resembling PII (emails, SSNs) before data leaves your secure environment.

• Representation: Does the training data match the real-world population it will be applied to?

  • Action: Check distribution of key demographics in your train vs. production sets.

• Validation: Have I validated the data schema and constraints?

  • Tool: Use the pointblank or validator packages to define and enforce data quality rules (e.g., col_vals_between(age, 0, 120)).

18.2 Fairness & Bias

Algorithms can reinforce existing inequalities.

• Protected Classes: Have I checked performance across different groups (Gender, Age, Ethnicity)?
• Bias Detection: Have I quantified the bias in my model?
  • Tool: Use fairness, fairmodels, or dalex to calculate metrics like Disparate Impact or Equal Opportunity difference.
  • Example Code: fairness_check(explainer, protected = data$gender, privileged = "Male")
• Proxy Variables: Are there variables (like Zip Code) acting as proxies for protected classes?
• Impact: Who could be harmed if this model makes a mistake? (e.g., Denying a loan vs. Recommending a bad movie).

18.3 Transparency & Explainability

Black boxes should not make high-stakes decisions.

• Documentation: Is the model card created? (Inputs, Outputs, Limitations, Intended Use).
• Explainability: Can I explain to a non-technical stakeholder why the model made a specific prediction?
  • Tool: Use dalex, lime, or iml to create feature contribution plots or breakdown plots.
• Feedback Loop: Is there a mechanism for users to report errors or contest decisions?

18.4 Reproducibility & Integrity

Science must be reproducible.

• Environment Sealing: Is the R environment reproducible?
  • Tool: Use renv to capture package versions in a renv.lock file.
• Randomness Control: Are random seeds set (set.seed()) for key steps like splitting data or initializing weights?
• Code Versioning: Is the code committed to version control (Git) with clear messages?

18.5 GenAI Specifics

If using Large Language Models (LLMs).

• Fact-Checking: Have I verified AI-generated code/facts against reliable sources? Hallucinations are common.
• Security: Have I ensured no sensitive data is being sent to public APIs?
• Attribution: Am I transparent about which parts of the work were AI-generated?

“With great power comes great responsibility.” — Stan Lee (and every Data Scientist)