Huawei Technologies has unveiled a software tool designed to accelerate inference in large artificial intelligence models, an advancement that could help China reduce its reliance on expensive high-bandwidth memory (HBM) chips. Unified Cache Manager (UCM) is an algorithm that allocates data according to varying latency requirements across different types of memories – including ultra-fast HBM, standard dynamic random access memory and solid-state drive – thereby enhancing inference efficiency,...