The AI Basketball Referee: The Days of Yelling at Human Officials Could Soon Be Over

Welcome to this week’s Deep-fried dive with Fry Guy! In these long-form articles, Fry Guy conducts an in-depth analysis of a cutting-edge AI development. Today, our dive is about the AI Basketball Referee. We hope you enjoy! 🙂

Yelling at referees is one of basketball’s most beloved pastimes … however, those days could soon be over. A young man named Ayush Pai has created an “AI Basketball Referee” which could eventually replace the traditional human official.

Ayush Pai is a sophomore Computer Science student at the Georgia Institute of Technology, where he specializes in Artificial Intelligence devices. Last basketball season, like most fans, he began to grow tired of missed calls which had a high impact on the outcome of games. He reflects, “One particularly memorable instance was when Jayson Tatum clearly fouled LeBron James with just one second remaining in a tied game, but no call was made.” This call was all over social media, and one of the players on James’ team, Patrick Beverley, pulled out a camera to show the referee the call he missed, a call which cost the Lakers an important win in their season.

Maybe Beverley was onto something … Maybe a camera could do a better job at calling fouls than a human official after all. As Pai states, “This incident and other similar situations served as a call to action for me to develop a technology that could minimize such missed calls and enhance fairness in basketball games.”

HOW DOES IT WORK?

The AI Basketball Referee is a multi-step and multi-faceted, open-source development. It begins with the detection of various elements involved in the game, such as players and a basketball, and then tracks their movements accordingly to monitor for rule violations.

Utilizing over 3,000 labeled images, Ayush Pai has trained a machine learning model called YOLO (You Only Look Once) to detect basketballs of all shades in various levels of lighting. This model also detects players in the image and differentiates them from fans, disregarding boxes with low confidence scores (such as player detections 0.05 and 0.15 below).

Pai also uses a computer technique called “pose estimation” which has the ability to track body part coordinates and movements in real time, which allows for player movement tracking across 18 key body points, or “keypoints.” Pai explains, “The system utilizes computer vision algorithms to analyze video frames in real-time, identifying basketballs and tracking player keypoints. By analyzing the position, movement, and relationships between player keypoints, the AI Basketball Referee [utilizing advanced AI algorithms] can accurately detect rule violations such as traveling [when a player moves both feet without dribbling the ball] and double dribbling (when a player who was previously dribbling picks up the ball and then proceeds to dribble again). The system also incorporates customizable parameters, allowing users to fine-tune the detection threshold and sensitivity to adapt to specific requirements.”

Pai has recently finished version 2.0 of the AI Basketball Referee which can identify player movements in relationship to the ball and associated rule violations, such as traveling and double dribbling. However, as he continues to work on version 3.0, he will be facing some challenges involving the interpreting of player movements and intentions. He explains, “While travels and double dribbles can be challenging due to the complexity of tracking player keypoints and analyzing their motion, other calls like shooting fouls involve more subjective elements such as player intent or contact assessment. These subjective calls may require further advancements in technology and additional data collection to ensure accurate and reliable decision-making.”

Pai believes expanding the capabilities of the AI Basketball Referee and refining its performance are the next steps. In the near future, he plans to incorporate the ability for more rule violations to be identified by the bot, “such as shooting fouls, goaltending, and reach-in fouls, as well as enhancing the system's accuracy and real-time feedback capabilities.” Pai also plans to “collaborate with basketball organizations, coaches, and referees to gather feedback and further refine the technology based on their expertise and requirements.”

WOULD THIS TAKE THE HUMAN TOUCH OUT OF THE GAME?

Many people have pushed back on the idea of automated calls in sports such as football and baseball in fear that it might remove the human touch from the game. The introduction of AI in sports officiating has the potential to alter the level of human subjectivity involved, which is a well-established characteristic of sports. Not to mention, fans of the game would have a lot less to take their anger out on!

Pai is aware of this concern and says that “while replacing human referees might not be entirely practical or even desired, designing an AI system to monitor games could offer a valuable solution to missed calls.” Pai imagines this AI system acting as a standard by which one might monitor the accuracy of human referee calls throughout a season. “With such a system implemented, the NBA could reward higher accuracy referees and punish those who don’t make the cut.”

So even for those who don’t want to take the human element out of refereeing entirely, Pai believes this technology could nonetheless prove useful in helping to reduce the number of bad calls being made.

WHAT DOES THE FUTURE HOLD FOR THE AI BASKETBALL REFEREE?

Pai has spoken about potentially creating an app for people to use for recreational play, but he does not view the AI Basketball Referee as just a playground toy. He says, “I envision expanding this project to be used in actual NBA games … By continually refining the AI Basketball Referee, I aim to make it a comprehensive tool for assisting referees in making accurate decisions and ensuring fair gameplay.”

One pressing question does surface with this revolutionary innovation … who will we swear at when our favorite teams lose a heartbreaking game?

Learn more about the AI Basketball Referee in this video made by the developer, Ayush Pai: https://www.youtube.com/watch?v=VZgXUBi_wkM