Comet Releases MLOps Industry Report | 2023 Machine Learning Practitioner Survey

NEW YORK, Feb. 23, 2023 — Comet, provider of the leading MLOps platform for machine learning (ML) teams from startup to enterprise, today announced the results of its second annual survey on the state of ML. The all new State of MLOps Industry Report | 2023 Machine Learning Practitioner Survey, which includes responses from 503 ML practitioners, sheds light on several important issues affecting the adoption of ML as well as project and initiative success.

“Our latest survey comes as ML practitioners are facing a new reality, with its own unique set of challenges ahead,” said Gideon Mendels, CEO and co-founder of Comet. “Organizations may tighten their budgets during these unstable economic times, but because leaders recognize ML’s potential to unlock incredible business value, there may be a push for ML practitioners to tackle more complex problems quickly. This may include addressing bias or adhering to a new AI Bill of Rights, making it imperative for ML practitioners and the success of an organization’s ML projects, to have the right tools in place.”

Among the key topics that emerged in this year’s report are: the AI Bill of Rights, bias, and operational challenges amid tightening budgets.

AI Bill of Rights

The US White House Office of Science and Technology Policy (WHOSTP) published a recent Blueprint for an “AI Bill of Rights,” setting out five principles which should guide the design, use and deployment of automated systems. The document provides a framework for how government, technology companies, and citizens can work together to ensure more accountable AI. In terms of the reaction within the ML community:

• Almost three quarters (73%) of ML practitioners agree that the AI Bill of Rights should be mandatory by law vs. opt-in.
• Around 2 in 5 (39%) think the AI Bill of Rights will affect their approach to ML deployment and development by slowing the process down.
• Similarly, 37% think the AI Bill of Rights will make the ML deployment and development process more difficult.
• Over a third (35%) believe it will make the process more expensive.
• On a positive note, almost 2 in 5 (38%) believe the AI Bill of Rights will make the ML deployment and development process safer, and a similar percentage (37%) think it will reduce the possibility of privacy violations.
• Over a third (35%) think the AI Bill of Rights will reduce the frequency of unsafe or ineffective ML systems.

Bias in AI Products

In fact, one of the reasons the WHOSTP created the initial Blueprint for the “AI Bill of Rights” was due to the prominence of bias in AI products. Bias has been one of the leading topics related to AI in recent years. Some view bias as overhyped, with ML practitioners capable of implementing best practices to mitigate it, while others think it is a problem that will continue to plague AI systems. The latest survey reveals how ML practitioners are approaching bias.

• Over a quarter (27%) of ML practitioners surveyed believe that bias will never truly be removed from AI-enabled products.
• Of organizations surveyed, almost 2 in 5 (38%) have a designated point of contact or support team that is looking out for bias when planning the design and/or launch of an AI-enabled product.
• A third (33%) of respondents think reducing the risk of bias occurring is one of the main benefits of Explainable A.I, which might indicate this could be a solution (though not without its own challenges).

Additional ML Challenges

The state of the economy clearly weighed on respondents’ minds as they considered its impact on their business and how that could trickle down to affect investments in ML. They also identified other areas of stress or challenges they anticipate facing in 2023:

• All (100%) machine learning practitioners surveyed said the economic situation will impact their business. The most common way, according to respondents, will be redundancies in the tech team (40%), followed by an impact on budgets (37%) and a hiring freeze (36%).
• Almost a third (32%) say innovation will slow as a result.
• Over the next year, ML practitioners surveyed anticipate the biggest to be sustainability (41%), followed by retention (39%), hiring staff with correct institutional knowledge (36%) and Explainable A.I (36%).

The complete survey can be found here.

State of MLOps Industry Report | 2023 Machine Learning Practitioner Survey Methodology

The research was conducted by Censuswide with 503 US 18+ (Machine Learning Practitioners) between December 20 and December 29, 2022. Censuswide abide by and employ members of the Market Research Society which is based on the ESOMAR principles and are members of The British Polling Council.

About Comet

Comet provides an MLOps platform that data scientists and machine learning teams use to manage, optimize, and accelerate the development process across the entire ML lifecycle, from training runs to monitoring models in production. Comet’s platform is trusted by over 150 enterprise customers including Affirm, Cepsa, Etsy, Uber and Zappos. Individuals and academic teams use Comet’s platform to advance research in their fields of study. Founded in 2017, Comet is headquartered in New York, NY with a remote workforce in nine countries on four continents. Comet is free to individuals and academic teams. Startup, team, and enterprise licensing is also available. To learn more, visit www.comet.com or join our community at heartbeat.comet.ml.

Source: Comet