Hacker Noon July 12, 2023
Big Data processing, BI analysis, and AI involve heavy usage of ML, including neural networks. This requires tremendous computational power: hundreds of gigabytes of RAM, tens of CPU cores, as well as graphics cards and/or special chips to speed up calculations.
Problems
When working in a big heterogeneous team (data analysts and researchers, development engineers, system engineers, etc.) on a large system, employees from different teams use various tools to solve problems. This applies both to development and runtime environments (Python, Matlab, Java, C/C++, etc.) and databases used (RDBMS, NoSQL, files, etc.).
Retraining can be time-consuming, expensive, and even impossible in the short run.
The introduction of new technologies can make it undesirable or impossible to...