Hi Dietmar, Nice to hear from you.
Each cpu socket has two DIEs(two NUMA). It seems you are placing all memories in one DIE of one socket.
You might want to place one memory on odd slots like 010, 030. 0 EVEN 0 -> one numa 0 ODD 0 -> the other numa.
BTW, you might want to put one memory in each memory slot or place half-full memories(one memory in each memory channel). Otherwise, the bandwidth would be limited and the performance might behave differently with a system whose memories are fully or half-fully installed.
Thanks Barry
-----Original Message----- From: Linaro-open-discussions [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of Dietmar Eggemann via Linaro-open-discussions Sent: Thursday, September 9, 2021 8:34 PM To: linaro-open-discussions@op-lists.linaro.org Subject: [Linaro-open-discussions] Memory distribution between NUMA nodes on Taishan 2280 v2
Hi,
We are carrying out some NUMA related test activities (some of which we discussed on this forum) on the Taishan 2280 v2 server.
For those tests we need memory attached to all 4 NUMA nodes equally. That's why we added 2 extra 32GB memory modules to the existing 2.
The manual says that they should be placed into slot:
000 (CPU A) 100 (CPU A) 020 (CPU B) and 120 (CPU B).
BMC console as well as startup logs show that the memory modules get properly detected in these slots.
But after Linux booted we don't have the expected 32 GB per NUMA node. There is node 1 & 3 with 64GB and node 0 & 2 with no memory instead.
We didn't find any BIOS options which could explain this memory distribution.
Do you have any hints why this is happening? Any help with this issue is highly appreciated.
Thanks!
-- Dietmar