Coefficient Giving’s Global Catastrophic Risks (GCR) group focuses on solving problems that could cause severe or even irreversible harm to humans on a global scale. Our work currently focuses on Transformative AI and Biosecurity.
We’re highly scope sensitive: we focus on the most severe harms that affect the most people. We fund directions that seem tractable, and have an appetite for high-uncertainty, high-upside bets. And we prioritize projects that are likely to be neglected by other sources of funding, including the market.
Within these areas, some important problems still need ownership. We think they’d benefit from general managers: people who hold themselves accountable for solving them.
Navigators is our leadership incubator — a selective program for people who are excited to own tractable and neglected GCR problems. Participants will have between six months and two years to research a problem and develop a pitch. Ideally, you’ll pitch CG on an ambitious grant, put together an excellent founding team, and launch with a clear path to scaling up your project substantially, such as $10 million or more.
The first Navigators cohort will focus on Securing Transformative AI. Each cohort will call for different profiles and backgrounds, but we expect every fellow to be exceptional.
How it works
- Apply as an individual or a team (up to three people).
- We’ll aim to match your current salary up to a generous cap (or use a reasonable benchmark if you’re in transition between roles), for between six months and two years, and cover research expenses like coding tools and compute.
- Part time options are possible for exceptional candidates.
- We’ll support you in relocating to a hub for your work, such as San Francisco, London, or D.C.
- You can also do the program remotely with occasional travel (~1 week/quarter).
- We’ll support access to coworking space (e.g. Constellation if in the San Francisco Bay Area), a research and conference budget, and events to help you build your network.
- You’ll receive guidance from advisors in your domain, with input and planning support from CG grantmakers and structured feedback from your cohort.
What comes after the program?
We’ll support your next steps with advice, warm introductions, and potential funding. Some outcomes we’d be excited about:
- You launch an ambitious new non-profit to solve a GCR problem, funded by us or a similar funder.
- You start a new team within an existing organization.
- You find a role at an organization where you can make maximum impact.
- Having identified a market mechanism to address your problem, you start a for-profit startup. We wouldn’t take an investment stake, but we could connect you with relevant venture funders.
If you’re an ambitious, high-ownership person who can work in uncharted territory, we’re excited for you to apply.
FAQ
How long is it exactly?
We expect most participants to take a year to develop their plans, though we’d be thrilled if you scaled faster. We offer up to two years of funding to provide greater income stability.
Can you sponsor my visa?
We won’t sponsor visas for this program, but we can support remote work (you can do it from your home country), relocation to a hub in your country, or travel expenses for in-person weeks.
We can also support short-term relocations (e.g. a few months’ stay) that are permitted by an independently-obtained visa.
Does getting into the program mean that my project will get funded by Coefficient?
Joining the incubator does not guarantee funding for your project. However, the goal of the program is to help you develop plans that we feel excited to scale up. We will work with you to develop a grant proposal that will be reviewed under our normal grant evaluation process, or help you pursue outside funding. If there’s not a mutually satisfactory project to scale up, we will support you in finding your next role.
Securing Transformative AI
D.C. / San Francisco / London / Tel Aviv / Remote
The Challenge
The more powerful AI gets, the more critical it is to have strong authorization systems around it.
We need to ensure that malicious actors can’t take unauthorized actions, like:
- Deploying CBRN capabilities
- Secretly modifying or exfiltrating models
- Stealing access to compute for dangerous training or inference
However, most people working on catastrophic AI risk don’t have the necessary infosecurity experience to address these problems.
We expect that transformative AI will need to be secured from threat actors like nation-states, terrorists, insider threats, and even misaligned AIs themselves. These are threat actors who may have enormous resources and capabilities, intentions to harm large numbers of people, or significant legitimate access. These may be the hardest, highest-stakes security problems you ever work on, and you’ll be one of a very small pool of people qualified to do so. The challenge — and your potential for impact — is huge.
We expect this cohort to focus primarily on securing AI infrastructure in frontier labs and data centers, or what we call the AI Infrastructure Security Shortlist. Specific problems we’re looking for owners on:
- Securing AI from exfiltration: Protecting model weights and algorithms from theft.
- Securing AI integrity: Protecting models from tampering via data poisoning or other attacks.
- Disconnected model safety: Preventing models that are not being served by AI labs from being used for catastrophic purposes
- Secure compute verification: Verifying that large-scale compute workloads are compliant with law or international agreements.
- Protocols for using compromised AI: Also called AI control – how to use untrusted AI to do useful work.
- Detecting and responding to rogue deployments: What to do if we fail to contain transformative AI and it gets deployed (or deploys itself) without oversight.
You can read a series of one-pagers about these problems here.
Who we’re looking for
- You have one of the following backgrounds, or similar:
- Experience working on cybersecurity in a national security context (especially building SL5 facilities, intelligence, and offensive security).
- Experience working on ML security research (tamper resistance, backdoors, password locking).
- Experience working in a relevant industry context — e.g. at a frontier AI lab, or security at a hyperscaler, hardware company, or government contractor.
- Exceptional capability in a relevant area — e.g. academic security research, security entrepreneurship, developing security standards, or AI hardware or firmware.
- You break down ambiguous problems into clear next steps, and don’t get overwhelmed when you lack guidance.
- You can clearly articulate your plans, verbally and in writing, to coordinate and motivate others.
Nice to haves
- You’ve thought deeply about existential risk and made contributions to reducing it.
- You’ve led or started an organization, or managed people.
- You have already worked on related AI security problems, and have ideas you’re ready to pursue.