Shahar is a senior researcher at the Center for the Study of Existential Risk in Cambridge. In his past life, he was a Google Engineer, though right now he spends most of your time thinking about how to prevent the risks that occur if companies like Google end up deploying powerful AI systems by leading AI Governance role-playing workshops ([ Ссылка ]).
Transcript & Audio: [ Ссылка ]
Host: [ Ссылка ]
Shahar: [ Ссылка ]
OUTLINE
00:00 Highlights
01:20 Intelligence Rising
06:07 Measuring Transformative AI By The Scale Of Its Impact
08:09 Comprehensive AI Services
11:38 Automating CEOs Through AI Services
14:21 Towards A "Tech Company Singularity"
15:58 Predicting AI Is Like Predicting The Industrial Revolution
19:57 50% Chance Of Human-brain Performance By 2038
22:25 AI Alignment Is About Steering Powerful Systems Towards Valuable Worlds
23:51 You Should Still Worry About Less Agential Systems
28:07 AI Strategy Needs To Be Tested In The Real World To Not Become Theoretical Physics
31:37 Playing War Games For Real-time Partial-information Advesarial Thinking
34:50 Towards World Leaders Playing The Game Because It’s Useful
39:31 Open Game, Cybersecurity, Government Spending, Hard And Soft Power
45:21 How Cybersecurity, Hard-power Or Soft-power Could Lead To A Strategic Advantage
48:58 Cybersecurity In A World Of Advanced AI Systems
52:50 Allocating AI Talent For Positive R&D ROI
57:25 Players Learn To Cooperate And Defect
01:00:10 Can You Actually Tax Tech Companies?
01:02:10 The Emergence Of Bilateral Agreements And Technology Bans
01:03:22 AI Labs Might Not Be Showing All Of Their Cards
01:06:34 Why Publish AI Research
01:09:21 Should You Expect Actors To Build Safety Features Before Crunch Time
01:12:39 Why Tech Companies And Governments Will Be The Decisive Players
01:14:29 Regulations Need To Happen Before The Explosion, Not After
01:16:55 Early Regulation Could Become Locked In
01:20:00 What Incentives Do Companies Have To Regulate?
01:23:06 Why Shahar Is Terrified Of AI DAOs
01:27:33 Concrete Mechanisms To Tell Apart Who We Should Trust With Building Advanced AI Systems
01:31:19 Increasing Privacy To Build Trust
01:33:37 Sensibilizing To Privacy Through Federated Learning
01:35:23 How To Motivate AI Regulations
01:37:44 How Governments Could Start Caring About AI risk
01:39:12 Attempts To Regulate Autonomous Weapons Have Not Resulted In A Ban
01:40:58 We Should Start By Convincing The Department Of Defense
01:42:08 Medical Device Regulations Might Be A Good Model Audits
01:46:09 Alignment Red Tape And Misalignment Fines
01:46:53 Red Teaming AI systems
01:49:12 Red Teaming May Not Extend To Advanced AI Systems
01:51:26 What Climate change Teaches Us About AI Strategy
01:55:16 Can We Actually Regulate Compute
01:57:01 How Feasible Are Shutdown Switches
01:58:20 Data Is Much Harder To Regulate Than Compute
02:01:48 How Will Humans End Disputes In Governance?
02:03:54 Final Thoughts
