Ai Safety & Entrepreneurship V1.0

Why did I create both a post and a wiki article?

For the most up-to-date version, see the Wiki page.

Articles:

There should be more AI safety organisations

Why does the AI Safety Community need help founding projects?

AI Assurance Tech Report

AI Safety as a YC Startup

Alignment can be the ‘clean energy’ of AI

AI Tools for Existential Security

Incubation Programs:

Def/acc at Entrepreneur First - this is a new program, which focuses on "defensive" tech, which will produce a wide variety of startups, some of which may help with reducing x-risk, but others may include AI applications like autonomous weapons or diagnostic medical software, which, while they generally won't increase x-risk, may not reduce it either.

Catalyze - AI Safety Incubation

Seldon Accelerator: “We take on AGI risk with optimism”

AE Studio has expressed interest in helping seed neglected approaches, incl. ideas that take the form of business ideas

AIS Friendly General Program:

Founding to Give: General EA incubator

Entrepreneur First - They run def/acc, so this could be a decent alternative

Halycon Futures: “We’re an entrepreneurial nonprofit and VC fund dedicated to making AI safe, secure and good for humanity.” They do new project incubation, grants, investments, ect.

Communities:

Entrepreneurship Channel EA Anyway

AI Safety Founders: Website, Discord, LinkedIn

VC:

AI Safety Seed Funding Network

Lionheart Ventures - “We invest in the wise development of transformative technologies.”

Juniper Ventures - Invests in companies “working to make AI safe, secure and beneficial for humanity”. Published the AI Assurance Tech Report

Polaris Ventures - “We support projects and people aiming to build a future guided by wisdom and compassion for all”

Mythos Ventures - “an early-stage venture capital firm investing in prosocial technologies for the transformative AI era.”

Metaplanet - “We support novel and evidence-based innovation that could produce an outsized return for the benefit of humankind.”

Anthology Fund - Backed by Anthropic. One of the five key areas is “trust and safety tooling”

Menlo Ventures - Lead the GoodFire round (article)

Safe AI Fund

Organisational Support:

Ashgro - fiscal sponsorship

Rethink Priorities Special Projects - provides fiscal sponsorship or incubation

Other:

Hackathon for AI Safety Startups - Ran once by Apart Research, may run again

Constellation Residency - Year-long position

Nonlinear - Free coaching for people who are running an AI safety startup or who are considering starting one

Dustbin - Things that previously existed

Longtermist Entrepreneurship Project - Retrospective