ASCII.jp: SambaNova AI SC40L processor fully focused on large language models Interesting CPU that attracted attention at Hot Chips 2024 (1/4)

Understand with the roadmap! Modern processor conditions
Chapter 792

For Part 5 of Hot Chips, I'd like to showcase the SC40L from SambaNova. SambaNova and the company's first SC10 processor were explained in Part 595 of the series, and this is an update from there.

The last time was in December 2020, right after they released a chip called the Cardinal SN10. Then, in 2021, Sambanova released the Cardinal SN20, and in 2022, the Cardinal SN30. However, SN20 and SN30 are the same thing, and the configuration is that SN20 x 2 is SN30. The specifications are summarized below.

In short, the SN20 is a slight upgrade of the SN10 version, with the only difference being a slight increase in SRAM capacity and perhaps a slight increase in operating frequency (more on this later).

Products and services using SN20 may have existed, but were quickly replaced by SN30. The reason you're saying “I wonder if it exists” is because of the software tuning tool the company provides.SambaTune release notesThis is because there is a statement “Version 1.17 (2023-10-20) GPT13B examples added for SN20 and SN30.” However, it is somewhat questionable how long services have been provided using the SN20.

Another mystery is the type of DRAM. According to the slides when the SN10 was announced in 2020, it was said that TB class memory could be connected externally. Regarding this, it is mentioned in Hot Chips slide 33 that the SN10-8R (a system with 8 SN10 chips) has a capacity of 12TB by connecting 48 channels of DDR4-2677, which means that one SN10 has DDR4-2667 that can be connected to 6 channels, Each has a capacity of 256MB, with a total capacity of 1.5TB.

In April 2021, there is talk of DDR5 for the SN20/SN30.Technology blog published by Samsung ElectronicsAccording to the company, the SambaNova platform uses DDR5. In terms of bandwidth, as of the end of 2021, DDR5-4800 has just been introduced, so it will not be able to double the bandwidth compared to DDR4, but in terms of capacity, it will support up to twice that and for this reason, it will probably It is possible to achieve the same amount of capacity (up to 1.5 TB) even if the number of channels is reduced by half.

Given the timing issue, there's not much point in sticking with DDR4 here. However, it bothers me that contradictory material appears (even from official SambaNova sources). The image below is an explanation of the SN30 based SambaNova platform that was delivered to the ALCF (Argonne Leadership Computing Facility), and you can see it says “TBs of DDR4 off-chip Memory”.

Platform based on SN30. This is itALCF videoYou can check out “High-level communication” in this chart is also key.

This “external chip” may have a profound meaning, but frankly, it is difficult to imagine that the product released in 2022 will be DDR4. I think this may be a typo in DDR5. Therefore, the above table is written as “DDR5?”