Estimated total cost of ownership for different archetypes
Example use case
Off-the-shelf coding assistant for software developers
General-purpose customer service chatbot with prompt engineering only and text chat only
~$0.5 million, recurring annually
~$0.5 million to $2.0 million, one-time
Estimated total cost of ownership
McKinsey & Company
Maker
Taker
Plug-in-layer maintenance: up to ~$0.2 million annually, assuming 10% of development cost.
Model inference:
General-purpose customer service chatbot: ~$2.0 million for building plug-in layer on top of 3rd-party model API. Costs include a team of 8 working for 9 months.
Off-the-shelf coding assistant: ~$0.5 million for integration. Costs include a team of 6 working for 3 to 4 months.
General-purpose customer service chatbot: ~$0.2 million annually, assuming 1,000 customer chats per day and 10,000 tokens per chat
Off-the-shelf coding assistant: ~$0.2 million annually per 1,000 daily users
Shaper
Taker
~$2.0 million to $10.0 million, one-time unless model is fine-tuned further
Model maintenance: ~$0.5 million. Assume $100,000 to $250,000 annually for MLOps platform⁴ and 1 machine learning engineer spending 50% to 100% of their time monitoring model performance.
Model inference: up to ~$0.5 million recurring annually. Assume 1,000 chats daily with both audio and texts.
~$0.5 million to $1.0 million, recurring annually
Plug-in-layer building: ~$1.0 million to $3.0 million. Costs include a team of 6 to 8 working for 6 to 12 months.
Data and model pipeline building: ~$0.5 million. Costs include 5 to 6 machine learning engineers and data engineers working for 16 to 20 weeks to collect and label data and perform data ETL.¹
Estimated total cost of ownership
Customer service chatbot fine-tuned with sector-specific knowledge and chat history
Example use case
Shaper
¹
Extract, transform, and load.
²
Model is fine-tuned on data set consisting of ~100,000 pages of sector-specific documents and 5 years of chat history from ~1,000 customer representatives, which is ~48 billion tokens. Lower end cost consists of 1% parameters retrained on open-source models (eg, LLaMA) and upper end on closed-source models. Chatbot can be accessed via both text and audio.
³
Model is optimized after each training run based on use of hyperparameters, data set, and model architecture. Model may be refreshed periodically when needed (eg, with fresh data).
⁴
Gilad Shaham, “Build or buy your MLOps platform: Main considerations,” LinkedIn, November 3, 2021.
Model fine-tuning²: ~$0.1 million to $6.0 million per training run³
Lower end: costs include compute and 2 data scientists working for 2 months
Upper end: compute based on public closed-source model fine-tuning cost
Plug-in-layer maintenance: up to ~$0.3 million recurring annually, assuming 10% of development cost.
~$5.0 million to $200.0 million, one-time unless model is fine-tuned or retrained
Model maintenance: ~$1.0 million to $4.0 million recurring annually. Assume $250,000 annually for MLOps platform⁴ and 3 to 5 machine learning engineers to monitor model performance.
Plug-in-layer maintenance: up to ~$0.3 million recurring annually, assuming 10% of development cost.
Model inference: ~$0.1 million to $1.0 million annually per 1,000 users. Assume each physician sees 20 to 25 patients per day and patient speaks for 6 to 25 minutes per visit.
~$1.0 million to $5.0 million, recurring annually
Model development: ~$0.5 million. Costs include 4 data scientists spending 3 to 4 months on model design, development, and evaluation leveraging existing research.
Data and model pipeline: ~$0.5 million to $1.0 million. Costs include 6 to 8 machine learning engineers and data engineers working for ~12 weeks to collect data and perform data ETL.¹
Estimated total cost of ownership
Foundation model trained for assisting in patient diagnosis
Example use case
¹
Extract, transform, and load.
²
Model is trained on 65 billion to 1 trillion parameters and data set of 1.2 to 2.4 trillion tokens. The tool can be accessed via both text and audio
³
Model is optimized after each training run based on use of hyperparameters, data set, and model architecture. Model may be refreshed periodically when needed (eg, with fresh data).
⁴
Gilad Shaham, “Build or buy your MLOps platform: Main considerations,” LinkedIn, November 3, 2021.
Maker
Plug-in-layer building: ~$1.0 million to $3.0 million. Costs include a team of 6 to 8 working 6 to 12 months.
Model training²: ~$4.0 million to $200.0 million per training run.³ Costs include compute and labor cost of 4 to 6 data scientists working for 3 to 6 months.
Note: Through engineering optimizations, the economics of generative AI are evolving rapidly, and these are high-level estimates based on total cost of ownership (resources, model training, etc) as of mid-2023.
Note: Through engineering optimizations, the economics of generative AI are evolving rapidly, and these are high-level estimates based on total cost of ownership (resources, model training, etc) as of mid-2023.
Note: Through engineering optimizations, the economics of generative AI are evolving rapidly, and these are high-level estimates based on total cost of ownership (resources, model training, etc) as of mid-2023.