Skip to content

Tracking AI

This is a 30-minute presentation used to help organization understand how tracking AI is part of their AI strategy and their overall organizational strategy.

Slide 1: Introduction

Does your organization have a formal AI strategy?

How do you predict the growth rate of AI?

In this presentation we will review the shared beliefs we have about the rates that AI is growing and how this will impact our organizations AI strategy. We will review how we measure AI capabilities using the time humans take to complete tasks.

Slide 2: Growth Rates

There are four possible growth rates of AI:

  1. Growth and Decline - AI is just a passing fad
  2. Steady State - no change from current capabilities
  3. Linear Growth - slow rise over time - arithmatic growth rates
  4. Exponential Growth - geometric growth - like Moore's Law - doubling every X months

What does the evidence show about how AI is growing in capabilities?

Slide 3: Visualizing Four Future Growth Rates

Slide 4: METR: Measuring Task Complexity

Measuring AI Capabilities With Task Complexity

How do we measure the overall capability of an AI system?

One way is to estimate how long a human takes to complete a task. We can assume that longer tasks are more complex and require more intelligence. We can then measure what the probability of an AI completing a task correctly. Example: Given a task to count the words in a paragraph. How long would this task take an average human? We can then create a database of tasks and measure how long humans to take to complete a task. This task/time database can form the basis of objective measures of AI capabilities.

Slide 4: METR

The METR.org studies show that AI task completion has been doubling every 7 months since 2019.

Slide 5: Projecting AI

We can now project this trend line out until 2030. The results are sometimes difficult to believe.