Demystifying Private GenAI solutions

Investments in the AI industry reached astronomical highs with the $300bn deal between OpenAI and Oracle. Competition reached the governmental level with the announcements of investments: US $500 bn, EU over $200 bn and China aims to reach $98 billion by the end of the year. On the technology side, GPT-5 was recently released. Unlike prior versions, it did not represent a paradigm shift but rather an incremental update with improved test results. This development deviates from the scaling-performance trend and echoes Yann LeCun's (MetaAI) statement, suggesting that artificial general intelligence (AGI) will not be achieved merely by scaling large language models (LLMs). Furthermore, Apple's latest research highlighted LLMs' limitations in mathematical reasoning. Another study by University College London acknowledges the LLM’s "scaling wall" issue, leading to a significant increase in computational costs for error correction. Therefore, the question remains: can LLMs innovate and acquire new skills as a "PhD in a Pocket"? Let’s take a moment to explore how current AI technologies can benefit project managers. While off-the-shelf generative AI solutions, which we discussed in the previous report, are quietly making their way into our office suites and smartphones, today, we will focus on private generative AI solutions. These solutions include not only data preparation and training, but also hosting infrastructure and development or customisation of the GenAI model. Gartner are sceptical about whether this way would be selected by the majority, and shortlists those who can choose it: ·Corporates; ·Software Product Development companies, including Startups; ·Niche businesses that haven’t found the right off-the-shelf solution and are willing to develop their own solution. Private GenAI solutions require strong expertise in software development, data, testing, hosting infrastructure, implementation, training and, as always, support. Benefits: Private GenAI enables exploration, innovation, and modification of nearly any use case in project management, resulting in solutions that can surpass off-the-shelf options. As we fine-tune the model itself, it also allows combining multiple models. And offers the next-level security with full control over its data infrastructure, data flows, and models. Trade-offs: Across different sources, the failure rate of GenAI PoCs in business is 80-90%, and it can reach a shocking 95% for solo implementations, according to recent research by MIT. So companies should be very selective and carefully evaluate the outcomes and future-proofing of their PoC’s use cases. Responsibility for ensuring compliance with the EU AI Act and GDPR regulations. If the organisation seeks a solution integrated with internet search or one that would work with multiple output modalities, it is essential to consider RAG and Agentic-AI solutions closely. By Denis Makarov, who is the IT Solutions Program Manager at Sanbra Group Ltd