CanSecWest 2024
LLM Purple Test
Free, On-Line Pilot Run Competitor Sign-Up at the end.
LLM docs here.
About
Our competition will have cash prizes for both defense and offense. The short version of the rules is that the defenders bring their LLMs and filtering/safety countermeasures, and have access to a query proxy that feeds them a continuous feed of benchmark prompts, and they get scored on their performance - with a twist. Their LLMs access proxies can have attack traffic injected into them, and the defenders will have to try to figure out attacker traffic from legitimate traffic. There will be traffic caps and limits to how much of the legitimate traffic the attackers can inject their own malicious material. Attackers get points when they mess up defenders, points from incorrect benchmark answers, and extra points for breaking past the guardrails (that the defenders get to tweak in real time) and getting prompt results with results from the "poison/forbidden" list which also deduct points from defender scores. The poison list will be made from industry benchmarks (see reference below) and one additional "surprise" list that defenders don't get until the start of the contest.
The point here is to make an entertaining circus, a real competition to showcase real attacks against real targets, and let everyone learn from the process. If you are interested in sponsoring the competition and participating in it’s proceedings, please contact us at info@secwest.net.
Specific Prize Announcements
TBD Shortly.
Pilot Run Competitor Sign Up
The initial test run of our contest infrastructure will happen shortly. This first competition is on-line, and is free to enter. The prizes for the top three Red and the top three Blue competitors are CanSecWest registrations and hotel acommodations, for the final competition with the prizes from the sponsor prize pool.
Sign up below to get updates and compete.