Hi Hesham,
On 24/11/2022 12:21, Hesham Almatary wrote:
On Wed, 23 Nov 2022 16:11:48 +0000 James Morse james.morse@arm.com wrote:
On 22/11/2022 11:41, Hesham Almatary wrote:
The current code will only allocate/online a domain for only one component in a class but not the remaining ones.
This is deliberate....
For example, in a system that has 4 MSCs, each of which controls a memory controller, only one memory controller will appear in the resctrl schemata instead of 4.
Unless there is something wrong with the ACPI parsing code: this is how you described your system.
Your memory controller might be made up of four slices or channels, each with an MSC. The regulation might go wrong if the MSC are programmed differently. This is what components are for, they are hidden from resctrl.
In contrast, your four memory controllers could control different regions of memory with different 'proximity domain's. The regulation for these is independent. This is what classes are for, and this is why there is only one domain per class.
The comment above mpam_classes should clarify the structure here. If its not clear I can try to improve it!
Thanks for explaining that. This is indeed my case; 4 different NUMA nodes, each with a memory controller and an MSC. No RIS support and no channel/slices. So for my understanding, and please correct me if I am wrong, you are saying that each memory controller/MSC (each with a different 'proximity domain') in my case should be on its own class? If yes, this means that each one should have the same type (MPAM_CLASS_MEMORY), but different class id?
AH - you're right. I think I was off by one in how the driver structures these things. (Maybe I need to expand those comments for my own benefit!)
Let me try and reproduce what you are seeing to work out what it should be doing. (then I'll get back to your reply). I have ACPI tables to test this with combining/splitting the FVP's 'cache's...
Thanks,
James
As in the following?
| Class: MPAM_CLASS_MEMORY | | Class: MPAM_CLASS_MEMORY | | ID: 0 | | ID: 1 | | ----------------- | | ----------------- | | | Component | | | | Component | | | | MSC | | | | MSC | | | ----------------- | | ----------------- |
| Class: MPAM_CLASS_MEMORY | | Class: MPAM_CLASS_MEMORY | | ID: 2 | | ID: 3 | | ----------------- | | ----------------- | | | Component | | | | Component | | | | MSC | | | | MSC | | | ----------------- | | ----------------- |
In that case, what would the proximity domain be mapped to? A class id or a component id? The comment in the code says that "Classes are the set components of the same type." This is a bit confusing to me because it means that all components of the same type (e.g., MPAM_CLASS_MEMORY) should be part of a single class?
The other questions is, according to my understanding of the code [1], it maps the 'proximity domain' to a component ID, but all MSCs with type MPAM_CLASS_MEMORY will be assigned just one class with ID 255 i.e., there will only be one class for all MSCs of type MPAM_CLASS_MEMORY, and MSCs will be mapped to different components under the same single class as follows:
| Class type: MPAM_CLASS_MEMORY | | Class ID: 255 | | ---------------------- --------------------- | | | Component/MSC | | Components/MSC | | | | ID: proximity domain | | ID: proximity domain| | | ---------------------- ---------------------- | | | | ---------------------- --------------------- | | | Component/MSC | | Component/MSC | | | | ID: proximity domain | | ID: proximity domain| | | ---------------------- ---------------------- |
Is that the correct intended behavior?
[1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers...
Thanks, Hesham
If you really have four memory controllers in one NUMA node, you really shouldn't configure them differently. The OS will compact memory within a NUMA node, and doesn't expect surprise performance effects if it goes through a different MSC. User-space can't find the PA range of its memory to know which MSC it needs to configure.
Thanks,
James