The Yuan requests your support! Our content will now be available free of charge for all registered subscribers, consistent with our mission to make AI a human commons accessible to all. We are therefore requesting donations from our readers so we may continue bringing you insightful reportage of this awesome technology that is sweeping the world. Donate now
AI in the Long Tail
By Kirk Borne  |  Oct 08, 2021
AI in the Long Tail
Image courtesy of and under license from Shutterstock.com
This cogent article by Kirk Borne employs mathematical reasoning to explore how smaller companies’ collective investments - the ‘long tail’ - drive the advance of artificial intelligence even more than outlays by bigger players, particularly in healthcare.

COLUMBIA, MARYLAND - Long tail distributions are very useful in many statistical applications. One example well known to statistics fans is Zipf's Law, which applies in many domains. It shows an inverse relation between rank and frequency, e.g., the N'th most common word used in a language occurs 1/N times as frequently as the No.1 ranked word.

Zipf’s Law has been applied to the national size distribution of cities, to the distribution in lengths of rivers, to the distribution in sizes of businesses and more. It is no fundamental law, like the law of gravity, but it is an empirically observed relationship that seems to hold up in many situations.

The value of the 1/N function increases systematically (i.e., asymptotically) to its peak value as values of N get ever smaller, nearing its peak value with the top-ranked item in the list at N=1. Conversely, the value of the 1/N function decreases slowly and systematically as values of N (the rank) grow ever larger, decreasing ever more slowly with each new value of N at very large N -- that's called the long tail of the distribution. It can be very long indeed.

One application of the long tail in e-commerce online sales is that the total (aggregate sum) of sales to all small buyers (high values of N, thus not in the category of highest-ranked buyers) is comparable to the total combined sales of the highest ranked buyers.

Mathematically, this last statement is strictly true in decade ranges, e.g., the aggregate sum of sales in the range ‘ranks N=1 through N=10’ is roughly equivalent to the aggregate sum of sales in the range ’ranks N=10 through N=100,’ which is roughly equivalent to the aggregate sum of sales in the range ’ranks N=100 through N=1000.’ As long as items are on the ranked list, and the total N keeps increasing, the total volume of sales keeps growing. These approximations are based on an empirically observed relationship, not a natural law of math

The content herein is subject to copyright by The Yuan. All rights reserved. The content of the services is owned or licensed to The Yuan. Such content from The Yuan may be shared and reprinted but must clearly identify The Yuan as its original source. Content from a third-party copyright holder identified in the copyright notice contained in such third party’s content appearing in The Yuan must likewise be clearly labeled as such.
Continue reading
Sign up now to read this story for free.
- or -
Continue with Linkedin Continue with Google
Comments
Share your thoughts.
The Yuan wants to hear your voice. We welcome your on-topic commentary, critique, and expertise. All comments are moderated for civility.