Lapse of Sanity AI Safety & Security Research by Marnick Vandecauter

Resources

What I reach for in AI-security work, and what I send people when they ask how to get into technical AI safety. Programmes change every cycle and links rot, so treat the information below as a starting point.

Education

For Beginners

New to the field, with an interest in technical AI safety:

  • The Future of AI (BlueDot): two hours, free, self-paced, no technical background assumed. The shortest orientation that does not oversell.
  • Technical AI Safety (BlueDot): free and facilitated in cohorts of about eight, either six weeks at roughly five hours a week or six days intensive. Alignment and RLHF, mechanistic interpretability, evaluations and red-teaming, AI control, scalable oversight.
  • AI risk problem profile (80,000 Hours) and the matching technical research career review: why the field treats this as urgent, and what the work actually consists of.
  • aisafety.info: short answers to the questions everyone has in the first week.
  • The ecosystem map (aisafety.com): who exists, what they do, and where the entry points are.
  • The Safety Apprentice: David Williams-King’s community resource for people new to the field, pairing explainers on misuse, misalignment and systemic risk with a running list of fellowships, grants and jobs. Free to read, with optional paid one-to-one career calls (paused at the time of writing).
  • Rob Miles on YouTube: the explainers most people in the field were handed first.
  • AI Safety Quest: free calls and study partners for when self-study stalls.

Already technical (machine learning or security background):

Fellowships and programs

Technical tracks only. Every entry is free to attend and most pay a stipend. Checked October 2026, and deliberately listed without deadlines, because all of them move each cycle: follow the link for the live dates.

  • SPAR: remote, part time, 5 to 40 hours a week across three months, unpaid, open to students and professionals with no prior research experience required. The lowest-friction way to get a mentor.
  • Apart Research: weekend research sprints, online and at local sites, open to anyone without an invitation. Doing well in one is the route into their Studio and fellowship tracks.
  • AI Safety Camp: remote and unpaid, teams formed around proposed projects, with project leads expected to commit at least 10 hours a week.
  • MARS (Cambridge AI Safety Hub): hybrid, a fortnight in Cambridge then 8 to 10 weeks remote at 8 to 15 hours a week. No stipend, but travel, accommodation, meals and a compute budget are covered.
  • AI Security Bootcamp: seven days in person, free and fully funded, in cohorts of 16 to 20. Built for people who already work in security (offensive security, incident response, threat intelligence, infrastructure, application security) and covers adversarial ML, LLM security, and infrastructure and governance. Cohorts have run in San Francisco, London, Singapore and Las Vegas. Fiscally sponsored by BlueDot Impact.
  • ARENA: four to five weeks in person in London, with travel, visa costs, accommodation and meals covered. Expects strong Python and the maths of modern machine learning.
  • Iliad: two programmes in mathematical and theoretical alignment, both held in London or the Bay Area, both with no tuition. The Intensive is four weeks of classes, five days a week, with a USD 5,000 travel-and-housing allowance, office space and two meals a day. The Fellowship is three months of full-time individually mentored research with a USD 6,000 a month travel-and-housing allowance, aimed at PhD students, postdocs and researchers with equivalent experience. Both expect a strong mathematical background (typically a graduate degree in maths, physics or theoretical computer science), though the Intensive does not require a PhD.
  • Algoverse AI Safety Fellowship: free, funded by Open Philanthropy, 12 weeks at 25 or more hours a week in teams of three. Sixty are admitted to a one-week trial and thirty continue as fellows.
  • LASR Labs: GBP 15,000 plus food, office and travel, London, in person and full time, 13 weeks, three cohorts a year, teams of three to four taking a project from proposal to preprint.
  • ERA Fellowship: stipend plus meals, transport and visa costs, Cambridge, 10 weeks, with separate technical, technical-governance and governance tracks.
  • Pivotal Research Fellowship: GBP 6,000 to 8,000, London, in person, 10 to 15 weeks, with travel and housing support. Technical safety sits alongside governance and biodefence tracks.
  • MATS: USD 1,250 a week plus about USD 2,000 a week of compute, Berkeley or London, 12 weeks, with a funded 6 to 12 month extension to apply for. Housing provided in Berkeley, a housing stipend in London.
  • Astra Fellowship (Constellation): USD 8,400 a month for empirical fellows plus a substantial compute budget, Berkeley, in person, five months. Mentors come from Redwood, Anthropic, OpenAI and others.

Courses and curricula

Communities and events

National hubs

National field-building organisations are often the shortest path from reading about this field to working on it with other people. Two in continental Europe:

  • Safe AI Germany (SAIGE): a research and capacity-building initiative running a three-month part-time incubator (10 to 20 hours a week, teams of one to five on a project proposed by an expert mentor, weekly mentor meetings, optionally extended by another three months), a Maths for AI Safety workshop in Bonn, and support for more than a dozen local hubs across German cities. Technical safety, governance, technical governance and field-building tracks, with no prior AI-safety experience required, and events open to people outside Germany.
  • Safe AI Netherlands: a volunteer-run foundation offering free course tracks (fundamentals, technical alignment, governance and policy), hackathons and talks, chapters in Utrecht, Groningen and Amsterdam, and a research hub that matches supervisors, provides compute and helps get the work published. Remote participation is welcome on research projects.

Events and conferences

  • Gray Swan: red-teaming arenas and competitions against production-grade models, with prizes and public leaderboards.
  • AI Security Forum: forums and workshops that put AI, cybersecurity and national-security people in one room around securing frontier models, run alongside events in Tel Aviv, Washington DC, Las Vegas and Paris. Attendance is by application per event, and their Substack carries the announcements.
  • EA Global and EAGx: the conference circuit where a lot of this field meets in person. EA Global runs a handful of large events a year (New York, San Francisco, London, Washington DC) while the regional EAGx conferences are where most people start, and both are by application. AI risk takes up a large share of the programme.

Advocacy and research organisations

  • MIRI: the Machine Intelligence Research Institute, which originated much of the early alignment research agenda and now argues for a globally coordinated and enforced moratorium on developing artificial superintelligence. Their book If Anyone Builds It and a shorter introduction called The Briefing are the current entry points.
  • PauseAI, and its German chapter PauseAI Deutschland: a volunteer advocacy movement calling for a prohibition on developing superintelligence until there is broad scientific consensus that it can be done safely and controllably. Demonstrations, petitions, mail tools for contacting representatives and university outreach, organised through local groups. The German site is in German, the international one in English.

Tools and benchmarks

Harnesses and scanners

  • Inspect (UK AISI): the evaluation framework worth learning first. Source and a large library of ported evals.
  • Petri: an auditing agent that runs scripted investigations against a target model and reports what it found, built on Inspect.
  • garak (NVIDIA): a scanner with a broad catalogue of probes, good for a first sweep over a new deployment.
  • PyRIT (Microsoft): a red-teaming automation framework built around orchestrators and scorers.
  • promptfoo red teaming: the pragmatic option when the thing under test is an application rather than a model.

Benchmarks and datasets

  • HarmBench: the standardised red-teaming evaluation, including its classifier. Source and data.
  • StrongREJECT: a grader built specifically because weaker judges score non-answers as successful jailbreaks.
  • ClearHarm: 179 prompts across biological, chemical, nuclear, conventional and cyber weapons, chosen so that no legitimate rephrasing answers the same request. FAR.AI built it because earlier sets mix in dual-use questions, where an attack can look successful while only the acceptable version was answered. Dataset.
  • AgentHarm: harmfulness measured over agent trajectories and tool calls rather than single replies.
  • AgentDojo: a dynamic environment for prompt-injection attacks and defences against tool-using agents.
  • AdvBench: the behaviours and strings set that most of the optimisation-attack literature reports against.

Other lists and standards