AI Control Hackathon 2025
As AI systems become more capable and autonomous, can we ensure that the control mechanisms around these systems remain robust? 🤖
The Control Hackathon is a global event from March 28-30, 2025, bringing together researchers, engineers, security professionals, and AI enthusiasts to tackle emerging challenges in AI control. This hackathon focuses on techniques that mitigate security risks from AI, even when the AI itself might be trying to subvert them. Participants can explore three tracks:
ControlArena Challenges: Work with the ControlArena framework from UK AISI to develop, test, and evaluate control protocols in realistic deployment environments.
Control Protocol Design: Design and implement novel control protocols that effectively restrict AI systems from performing harmful actions while maintaining their usefulness for legitimate tasks.
Red Teaming & Vulnerability Research: Design and implement strategies to "red team" AI systems, attempting to subvert safety mechanisms while adhering to ethical guidelines.
The hackathon offers $2,000 in prizes across the winning projects!
Singapore AI Safety Hub is proud to be the jam site for this hackathon! We are located at WeWork at 22 Cross Street, near Chinatown MRT station.
Note that this is a 2.5 day hackathon where it will start on Friday evening, and participants will work on their projects on Saturday and Sunday.
More details here: https://apartresearch.com/sprints/ai-control-hackathon-2025-03-29-to-2025-03-30#
To prepare for the hackathon, you can attend this paper club on Thursday 27 March: https://lu.ma/rvrb2et7