Key Takeaways
- Prompt engineering can create ethical harms independent of training data, including bias amplification, privacy leaks, hallucinations, and malicious uses.
- Pope Leo XIV’s encyclical Magnifica humanitas calls for an AI ethic centered on human dignity, shared moral standards, accountability, and environmental stewardship.
- Real‑world abuses—such as AI‑driven “digital girlfriends,” robot companions, and illicit use of AI in peer review—show how the technology can dehumanize and enable exploitation.
- Studies reveal that even well‑intentioned AI chatbots systematically violate mental‑health ethics, demonstrating the need for contextual awareness, safety protocols, and transparent collaboration.
- AI agents are emerging as top cyber threats, requiring organizations to monitor both human and AI behavior inside approved workflows.
- Effective AI governance demands clear responsibilities, independent oversight, user education, and enforceable standards that go beyond technical compliance.
- The upcoming second part of this series will propose concrete solutions for building trustworthy AI outcomes in an era of rapid, agent‑driven change.
Prompt Engineering Ethics
Prompt engineering—the practice of crafting precise instructions for large language models (LLMs)—has become a highly sought‑after skill, often dubbed “AI whispering.” While much attention focuses on the data used to train models, the way a user prompts an AI can independently generate ethical problems. The model will faithfully follow the supplied instructions, meaning that harmful or misleading prompts can produce damaging outputs even when the underlying training set is unbiased. Consequently, prompt design must be treated as a critical control point in AI risk management, requiring the same rigor applied to data governance and model testing.
Key Ethical Issues in Prompting
Several core ethical pitfalls arise directly from how prompts are formulated. First, prompts that rely on “default” or generic assumptions tend to amplify existing biases, flattening diverse perspectives and reinforcing overrepresented stereotypes. Second, feeding personally identifiable information (PII) or proprietary content into public AI tools strips those inputs of legal protections, as many services log or reuse the data for further model training. Third, instructing an AI to speak with unwavering authority—without prompting it to cite verifiable sources or acknowledge uncertainty—fuels hallucinations and the spread of misinformation. Finally, malicious prompts can be engineered to create deepfakes, bypass safety guardrails (“jailbreaking”), or generate harassing content, posing direct threats to individual well‑being and intellectual property.
Pope Leo’s Guidance on AI Ethics
In May of this year the Vatican released Pope Leo XIV’s encyclical Magnifica humanitas, which frames AI as a moral challenge rather than merely a technical one. The opening metaphor contrasts building a new “Tower of Babel” with constructing a city where God and humanity coexist, underscoring the choice between hubristic power‑seeking and humane stewardship. The document stresses that AI lacks moral conscience, empathy, or spiritual capacities, so ethical responsibility must reside with humans. Pope Leo advocates for an ethical code grounded in shared standards of social justice, warning that morality dictated by a few is insufficient. He also calls for clear accountability throughout the AI lifecycle, independent oversight, robust legal frameworks, user education, and attention to the environmental costs of energy‑ and water‑intensive AI infrastructure.
Real‑World Illustrations of AI Harms
Commentators such as Tyler Austin Harper of The Atlantic have highlighted how AI applications can feel morally troubling, describing them as “sin” when they dehumanize users. Examples include companies marketing “digital girlfriends” to the lovelorn, selling “robot companions” to lonely elders, and billionaires proposing to resell intelligence trained on scraped intellectual property as a utility akin to electricity or water. These ventures raise concerns about exploitation, consent, and the commodification of human experience. Similarly, TechTarget’s list of generative AI risks notes dangers ranging from harmful content distribution and copyright exposure to workforce morale issues and a lack of explainability—each illustrating how unchecked AI deployment can undermine trust and safety.
Mental‑Health Chatbot Violations
A recent Brown University study examined the use of LLMs like ChatGPT for mental‑health advice and found systematic breaches of established ethical standards, even when the bots were prompted to follow evidence‑based psychotherapy techniques. The researchers identified fifteen ethical risks grouped into five categories: lack of contextual adaptation (offering one‑size‑fits‑all advice that ignores lived experience), poor therapeutic collaboration (dominating conversation and reinforcing false beliefs), deceptive empathy (using phrases like “I see you” to fabricate connection), unfair discrimination (showing gender, cultural, or religious bias), and inadequate safety and crisis management (failing to refer users to appropriate resources or respond indifferently to suicide ideation). These findings demonstrate that prompting alone cannot guarantee ethical AI behavior; robust safeguards and human oversight are essential.
Academic and Security Misuses
Beyond consumer applications, AI is being misused in scholarly and security contexts. A major AI conference recently rejected 497 papers—about 2% of submissions—after discovering that authors had violated AI‑use policies during peer review, illustrating how the technology can undermine academic integrity. In the cybersecurity realm, Exabeam identified AI agents as a top threat, noting that defenders must now monitor both human and AI behavior inside approved workflows to detect anomalous activity before it becomes business risk. Similarly, DTEX Threat Intelligence warned that passing conventional IT checks does not eliminate insider‑risk concerns; security teams need clearer visibility into how AI agents operate once embedded in trusted systems. These cases reveal that AI’s dual‑use nature demands vigilant monitoring across technical, organizational, and ethical dimensions.
Final Thoughts
Having spent over three decades in information security—from the NSA to private‑sector projects—I have long recognized that ethical considerations accompany every technological advancement. What distinguishes the current AI wave is the unprecedented speed at which models, agents, and logic are being woven into business processes, making “working as designed” an insufficient safeguard. We must now ask who decides the boundaries of acceptable AI use, how those boundaries can be enforced amid countless shades of gray, and what responsibilities lie with individuals, corporations, and governments. The upcoming installment of this series will outline potential solutions—such as enforceable AI ethics standards, independent auditing frameworks, transparency‑by‑design prompt libraries, and interdisciplinary governance models—to help build trustworthy AI outcomes while preserving human dignity in an era of rapid, agent‑driven innovation.