I’ve been enjoying reading on the beaches of Dubrovnik for the past few days. I’m generally a big fan of long-form magazines like Asterisk and Works In Progress, but I really really liked issue 3 of Asterisk in particular.
Here are some notes!
AI Isn’t Coming for Tech Jobs - Yet by Jonathan Mann
- This is a nice BOTEC of how LLMs will affect tech employment by 2025
- It decomposes the total change into the counterfactual without LLMs + the job growth in existing industries from LLMs + the job growth in new industries from LLM - the job displacement due to LLMs
- Starts with some estimates of LLM adoption rates, rate of integration of LLM into businesses and productivity gains from LLMs
- Gets reasonable numbers throughout: 4% (based on secular trends), 1% (based on the price elasticity of SWEs), 3.5% (based on LLMs as a fraction of the tech market), and 3.5% (based on substitution effects) respectively, netting to 5.5% job growth
Crash Testing GPT-4 by Beth Barnes
- I’ve spent a bit of time following ARC Eval’s work on model evaluations, so this wasn’t that new
- Cool to hear from Beth about some of the future plans relating to evals and audits
What We Get Wrong About AI & China by Jeff Ding
- Useful to know that people are estimating China’s models to be a year or two behind the frontier ones from big US labs
- It made a good point that a lot of the gains from GPT-3 to ChatGPT was InstructGPT, and RLHF might pose a challenge given Chinese censorship requirements
- I liked the comments on how we overrate “leading in innovation” relative to the “diffusion of innovation”
A Field Guide to AI Safety by Kelsey Piper
- This is a classic Future Perfect style piece from Kelsey
- I thought the framing of the three camps of Yudkowsky, Christiano/Karnofsky, and LeCun was an interesting choice
- Again, much as with Beth’s, not much was new to me
Through A Glass Darkly by Scott Alexander
- Correctly notes the cognitive biases associated with forecasting and Platt’s Law that forecasts of transformative AI are always 30 years in the future
- The focus on three types of forecasts i.e. surveys (especially Katja Grace’s), prediction markets (especially Metaculus) and models (especially Ajeya Cotra’s) was reasonable, but it felt a bit rushed throughout
How Long Until Armageddon? by Michael Gordin
- A cool history of forecasting when the Soviets would get nuclear weapons
- Didn’t get a great sense of how this should update our ways of forecasting TAI, except that it is difficult
Are We Smart Enough to Know How Smart AIs Are? by Robert Long
- I really enjoyed the history of our understanding of animal cognition and how some of the clear pitfalls along the way might apply to us trying to understand AI intelligence/consciousness
- I liked how he warned against dangers on both sides of the spectrum of anthropomorphism i.e. imposing our understanding of “theory of mind” onto these vastly different cognitive structures versus reductively retreating into dismissing models as “token predictors”
How We Can Regulate AI by Avital Balwit
- This focused heavily on compute governance at the expense of other regulatory proposals
- Nice overview of the infrastucture required: a chip registry, pre-registering training runs, privacy-preserving on-chip verification mechanisms, KYC-esque regulations, US-led international enforcement, evaluations and standards
- Interesting BOTEC that training GPT-3 would require 10,000 2022 MacBook Pros i.e. $24 million, which is more than the $4 to $12 million it actually cost at the time, but not absurdly more, making distributed training a real possibility in the future
The Transistor Cliff by Sarah Constantin
- A fantastic deep dive into what compute and semis really are, and probably the piece from which I learnt the most
- Good overview of scaling laws in AI with respect to data, parameters and compute, where 10x compute lowers loss by 11%
- Some facts about the GPTs: GPT-2 took 300m tokens and 1.5b parameters, GPT-3 was 350b and 175b (so 150k times the compute of GPT-2) and GPT-4 probably was 8t and 700b (so 100 times the compute GPT-3)
- TIL: from 2012 to 2023, the compute for SOTA models increased by 1e8 i.e. 8 OOMs
- Moore’s Law says that the number of transistors per chip can double every two years for the same price
- It has not held when considering cost ever since the 28nm node in 2011, with costs of modern 3nm nodes back at 2005 levels
- The compute budget of an AI model is training times \(\times\) number of cores \(\times\) peak flops \(\times\) utilisation rate
- GPUs spend around half the time idling to wait for memory calls and cross-processor communications, with this increasing in the number of cores being run in parallel
- Solutions include increasing the memory bus width (which is constrained by heat) and algorithmic improvements like FlashAttention
- GPU flops have doubled every two years due to smaller transistors and more transistors per GPU
- There are limits to transistor size: the energy required to flip the gates for silicon is getting close to random background heat and the light used in UV lithography cannot etch patterns smaller than half the frequency of the wave, and we expect both of these to bind by 2030
- Various solutions present: redesigning chips to stack transistors vertically, use special purpose ASICs/FPGAs like TPUs, replacing transistors with other switches
The Puzzle of Non-Proliferation by Carl Robichaud
- More nuclear analogies for AI
- Interesting that India’s nuclear test might have been the trigger for US-Soviet cooperation
- Preventing “horizontal proliferation” has been possible via security guarantees, treaties, sanctions, military action, and norms, but this has not stopped “vertical proliferation” of a country which already had nukes improving them
The Great Inflection by Matt Clancy and Tamay Besiroglu
- Matt and Tamay are two of my favourite economists who think about growth and AI
- Both acknowledge that there is a possibility of transformative AI causing explosive economic growth i.e. >20% a year
- The intuition is that since human-level AI can be thought of as adding to the labour force, and since semi-endogenous growth theory tells us that larger population begets more ideas and ideas are non-rivalrous inputs, this can produce increasing returns to scale
- The standard objections about Baumol bottleneck effects, o-ring production functions and the various other messy bits of reality (e.g. regulations, Moravec, the need for authentic human labour, the fact that some R&D just require physical experimentation) etc. all apply
Emotional Intelligence Amplification by Jamie Wahls
- I’m not a huge fiction person, but this was fun
- Paints an interesting picture of what human relationships might look like in a post-AGI world