Hi Salil,
(replying to an off-list email from Salil...)
You suggested proposing a talk on the hotplug stuff for kvm forum, yes we can co-present it, but we'll need to discuss it here. Let me know if you plan to submit this today, or I need to do it. I finish work at roughly 6pm UK time.
For a title I suggest: "Supporting hotplug on architectures that don't".
I had a stab at the abstract before you wrote yours: vCPU hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardise without hardware. This talk covers the use-cases, difficulties, and a proposed way of supporting this on architectures that lack physical CPU hotplug.
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
We'll need to come up with slides, my suggested outline is something like: * The x86 history: CPU hotplug being supported before virtualisation. * Why vCPU hotplug is used for on x86 (kata containers, kubernetes, cloud orchestration) * Why that ACPI mechanism wouldn't work for arm64: we can't support hotplug virtualisation without real hardware. (not future proof) * What the actual requirements are. * Changes to PSCI and arm64-linux to support all this in a future proof way. * How physical CPU hotplug would work at the same time
[* Slides on the Qemu details and complexities.]
Something to be aware is of is that arm want a clear split in the presentation between the kernel side of it, and your Qemu work.
Thanks,
James
Hi James, Thanks for reverting back.
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 2:12 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
Hi Salil,
(replying to an off-list email from Salil...)
You suggested proposing a talk on the hotplug stuff for kvm forum, yes we can co-present it, but we'll need to discuss it here.
Thanks! should I assume that you have got the permission from your legal department?
Also, It is a noise for others so thought to avoid LOD. Plus, I was not sure if this is a technical stuff we are discussing rather a proposal to a presentation. But anyways...we can discuss here. No problem.
Let me know if you plan to submit this today, or I need to do it. I finish work at roughly 6pm UK time.
I think I will submit it.
For a title I suggest: "Supporting hotplug on architectures that don't".
Sure, that sounds good, just that I would add below as well:
"Challenges Revisited in Supporting Virtual CPU Hotplug on architectures that don't".
I had a stab at the abstract before you wrote yours: vCPU hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardise without hardware. This talk covers the use-cases, difficulties, and a proposed way of supporting this on architectures that lack physical CPU hotplug.
No problem, mostly sounds good to me. Changed a bit:
"Virtual CPU Hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardize without hardware.
This talk revisits the use-cases, challenges faced, and proposes a way of supporting virtual CPU Hotplug on architectures that lack physical CPU Hotplug with a thoughtful consideration to co-exist if later becomes a reality in future."
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
Agreed, A rough summary might help organizers weigh in terms of relevance when considering whether to accept the talk in case there are many talks submitted. Whole idea is to convince them that this talk is relevant which we think it is. (I was given feedback about the same in the past)
This is just a rough highlight of what we intend to present in principle. But will also need to check if all of this fits in that 1500 word limit.
We'll need to come up with slides, my suggested outline is something like:
- The x86 history: CPU hotplug being supported before virtualisation.
- Why vCPU hotplug is used for on x86 (kata containers, kubernetes, cloud orchestration)
- Why that ACPI mechanism wouldn't work for arm64: we can't support hotplug virtualization without real hardware. (not future proof)
- What the actual requirements are.
- Changes to PSCI and arm64-linux to support all this in a future proof way.
- How physical CPU hotplug would work at the same time
[* Slides on the Qemu details and complexities.]
Sounds good. All of above are useful content and we can organize them later when we prepare slides. We can discuss details of these later.
Something to be aware is of is that arm want a clear split in the presentation between the kernel side of it, and your Qemu work.
That is how it should considering the legal aspects and otherwise as well. (New kernel approach is proposed by ARM)
We will make sure this how slides are implemented. We can talk about all those details later.
Thanks!
Hi Salil,
On 31/03/2023 15:18, Salil Mehta wrote:
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 2:12 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
You suggested proposing a talk on the hotplug stuff for kvm forum, yes we can co-present it, but we'll need to discuss it here.
Thanks! should I assume that you have got the permission from your legal department?
Yes I got permission for this. This discussion being public was one of the terms.
Also, It is a noise for others so thought to avoid LOD. Plus, I was not sure if this is a technical stuff we are discussing rather a proposal to a presentation. But anyways...we can discuss here. No problem.
I agree its annoying - sorry to everyone else.
Let me know if you plan to submit this today, or I need to do it. I finish work at roughly 6pm UK time.
I think I will submit it.
Great, thanks.
For a title I suggest: "Supporting hotplug on architectures that don't".
Sure, that sounds good, just that I would add below as well:
"Challenges Revisited in Supporting Virtual CPU Hotplug on architectures that don't".
I don't think we are revisiting the challenges, we're proposing this as the solution for any architecture with the same problem. Adding 'virtual', confuses what the 'that don't' was supposed to be about, but I agree its too terse.
How about "Supporting vCPU hotplug on architectures that don't have CPU hotplug (volume 2)"
I had a stab at the abstract before you wrote yours: vCPU hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardise without hardware. This talk covers the use-cases, difficulties, and a proposed way of supporting this on architectures that lack physical CPU hotplug.
No problem, mostly sounds good to me. Changed a bit:
"Virtual CPU Hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardize without hardware.
This talk revisits the use-cases, challenges faced, and proposes a way of supporting virtual CPU Hotplug on architectures that lack physical CPU Hotplug with a thoughtful consideration to co-exist if later becomes a reality in future."
"co-exist if later becomes" is hard to parse, its also too long a sentence!:
| This talk revisits the use-cases, challenges faced, and proposes a way of supporting | virtual CPU Hotplug on architectures that lack physical CPU Hotplug. How these two | mechanisms can co-exist on future systems is discussed.
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
Agreed, A rough summary might help organizers weigh in terms of relevance when considering whether to accept the talk in case there are many talks submitted. Whole idea is to convince them that this talk is relevant which we think it is. (I was given feedback about the same in the past)
We should put this sort of stuff in the notes section for the reviewers: | This is a follow up to where the problem was first presented at KVM forum, we've now got | a solution we think works for arm64 and probably others. | There are rough edges, so this would benefit from discussion with orchestration | developers.
The rough edge being the present mask in /sys. I think the orchestrator folk need to be aware that this adds a new 'enabled' state, and they can't just rely on 'present'. (they don't appear to today)
This is just a rough highlight of what we intend to present in principle. But will also need to check if all of this fits in that 1500 word limit.
We'll need to come up with slides, my suggested outline is something like:
- The x86 history: CPU hotplug being supported before virtualisation.
- Why vCPU hotplug is used for on x86 (kata containers, kubernetes, cloud orchestration)
- Why that ACPI mechanism wouldn't work for arm64: we can't support hotplug virtualization without real hardware. (not future proof)
- What the actual requirements are.
- Changes to PSCI and arm64-linux to support all this in a future proof way.
- How physical CPU hotplug would work at the same time
[* Slides on the Qemu details and complexities.]
Sounds good. All of above are useful content and we can organize them later when we prepare slides. We can discuss details of these later.
Yup, I had to write it for arm, so I figured I may was well include it.
Thanks,
James
Hi James,
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 4:40 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org; Jonathan Cameron jonathan.cameron@huawei.com; Lorenzo Pieralisi lorenzo.pieralisi@linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
Hi Salil,
On 31/03/2023 15:18, Salil Mehta wrote:
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 2:12 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
You suggested proposing a talk on the hotplug stuff for kvm forum, yes we can co-present it, but we'll need to discuss it here.
Thanks! should I assume that you have got the permission from your legal department?
Yes I got permission for this. This discussion being public was one of the terms.
Also, It is a noise for others so thought to avoid LOD. Plus, I was not sure if this is a technical stuff we are discussing rather a proposal to a presentation. But anyways...we can discuss here. No problem.
I agree its annoying - sorry to everyone else.
Let me know if you plan to submit this today, or I need to do it. I finish work at roughly 6pm UK time.
I think I will submit it.
Great, thanks.
For a title I suggest: "Supporting hotplug on architectures that don't".
Sure, that sounds good, just that I would add below as well:
"Challenges Revisited in Supporting Virtual CPU Hotplug on architectures that don't".
I don't think we are revisiting the challenges, we're proposing this as the solution for any architecture with the same problem.
I think we are. Qemu code is 95% same as presented part of RFC V1 and in KVM Forum2020 but this time with a support to a new kernel approach. There are still old design problems lingering which need open discussion and there are some new problems which needs mention. These could not be discussed earlier because Qemu community was not interested to discuss till they hear from ARM about the resolution of the kernel/ARM arch problems mentioned in the earlier presentation. This was the official feedback of the first presentation I gave in KVMForum2020. So I have to continue from there...
Please don’t take Qemu changes for granted. There are already known hard bits in Qemu. It is very important to present them clearly. Forum's like these are very important From Huawei's perspective for the reasons you already know. We don’t have many channels to discuss quickly.
Kernel approach of handling ACPI Hotplug events is new and we are disclosing it to wider public in this forum and asking for their feedback. But then this also opens the door for Qemu discussions.
Adding 'virtual', confuses what the 'that don't' was supposed to be about, but I agree its too terse.
How about "Supporting vCPU hotplug on architectures that don't have CPU hotplug (volume 2)"
I would stick with revisiting as I have to present Qemu updates here and then Jump to what is being done extra from the last presentation to support new kernel Changes. This is very important from Qemu perspective.
I had a stab at the abstract before you wrote yours: vCPU hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardise without hardware. This talk covers the use-cases, difficulties, and a proposed way of supporting this on architectures that lack physical CPU hotplug.
No problem, mostly sounds good to me. Changed a bit:
"Virtual CPU Hotplug is used by orchestration managers to pre-create VMs, and scale them correctly when a workload is deployed. Architecture support for adding CPUs to an existing system is complex, and not feasible to standardize without hardware.
This talk revisits the use-cases, challenges faced, and proposes a way of supporting virtual CPU Hotplug on architectures that lack physical CPU Hotplug with a thoughtful consideration to co-exist if later becomes a reality in future."
"co-exist if later becomes" is hard to parse, its also too long a sentence!:
| This talk revisits the use-cases, challenges faced, and proposes a way of supporting | virtual CPU Hotplug on architectures that lack physical CPU Hotplug. How these two | mechanisms can co-exist on future systems is discussed.
Sure. We can add this phrase if it conveys idea more clearly. No problem.
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
Agreed, A rough summary might help organizers weigh in terms of relevance when considering whether to accept the talk in case there are many talks submitted. Whole idea is to convince them that this talk is relevant which we think it is. (I was given feedback about the same in the past)
We should put this sort of stuff in the notes section for the reviewers: | This is a follow up to where the problem was first presented at KVM forum, we've now got | a solution we think works for arm64 and probably others. | There are rough edges, so this would benefit from discussion with orchestration | developers.
Sure, we will add few bits there as well. But lets stick with some point wise rough abstract in the mains as that generates the interest of the audiences. It is also important to attract the audiences on the basis of relevance :)
The rough edge being the present mask in /sys. I think the orchestrator folk need to be aware that this adds a new 'enabled' state, and they can't just rely on 'present'. (they don't appear to today)
You will get an answer only if any cloud vendors are present (I hope they are). I think you should raise this in the LKML (and distros related mailing-list first)
This is just a rough highlight of what we intend to present in principle. But will also need to check if all of this fits in that 1500 word limit.
We'll need to come up with slides, my suggested outline is something like:
- The x86 history: CPU hotplug being supported before virtualisation.
- Why vCPU hotplug is used for on x86 (kata containers, kubernetes, cloud orchestration)
- Why that ACPI mechanism wouldn't work for arm64: we can't support hotplug virtualization without real hardware. (not future proof)
- What the actual requirements are.
- Changes to PSCI and arm64-linux to support all this in a future proof way.
- How physical CPU hotplug would work at the same time
[* Slides on the Qemu details and complexities.]
Sounds good. All of above are useful content and we can organize them later when we prepare slides. We can discuss details of these later.
Yup, I had to write it for arm, so I figured I may was well include it.
Thanks Salil
Hi Salil,
On 31/03/2023 17:42, Salil Mehta wrote:
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 4:40 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org; Jonathan Cameron jonathan.cameron@huawei.com; Lorenzo Pieralisi lorenzo.pieralisi@linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
For a title I suggest: "Supporting hotplug on architectures that don't".
Sure, that sounds good, just that I would add below as well:
"Challenges Revisited in Supporting Virtual CPU Hotplug on architectures that don't".
I don't think we are revisiting the challenges, we're proposing this as the solution for any architecture with the same problem.
I think we are. Qemu code is 95% same as presented part of RFC V1 and in KVM Forum2020 but this time with a support to a new kernel approach. There are still old design problems lingering which need open discussion and there are some new problems which needs mention. These could not be discussed earlier because Qemu community was not interested to discuss till they hear from ARM about the resolution of the kernel/ARM arch problems mentioned in the earlier presentation. This was the official feedback of the first presentation I gave in KVMForum2020. So I have to continue from there...
Fair enough - I assumed the kernel support was the principle challenge!
Please don’t take Qemu changes for granted. There are already known hard bits in Qemu. It is very important to present them clearly. Forum's like these are very important From Huawei's perspective for the reasons you already know. We don’t have many channels to discuss quickly.
Kernel approach of handling ACPI Hotplug events is new and we are disclosing it to wider public in this forum and asking for their feedback. But then this also opens the door for Qemu discussions.
Adding 'virtual', confuses what the 'that don't' was supposed to be about, but I agree its too terse.
How about "Supporting vCPU hotplug on architectures that don't have CPU hotplug (volume 2)"
I would stick with revisiting as I have to present Qemu updates here and then Jump to what is being done extra from the last presentation to support new kernel Changes. This is very important from Qemu perspective.
Sure, fine by me. I wasn't aware of the Qemu discussions.
[...]
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
Agreed, A rough summary might help organizers weigh in terms of relevance when considering whether to accept the talk in case there are many talks submitted. Whole idea is to convince them that this talk is relevant which we think it is. (I was given feedback about the same in the past)
We should put this sort of stuff in the notes section for the reviewers: | This is a follow up to where the problem was first presented at KVM forum, we've now got | a solution we think works for arm64 and probably others. | There are rough edges, so this would benefit from discussion with orchestration | developers.
Sure, we will add few bits there as well. But lets stick with some point wise rough abstract in the mains as that generates the interest of the audiences. It is also important to attract the audiences on the basis of relevance :)
The rough edge being the present mask in /sys. I think the orchestrator folk need to be aware that this adds a new 'enabled' state, and they can't just rely on 'present'. (they don't appear to today)
You will get an answer only if any cloud vendors are present (I hope they are). I think you should raise this in the LKML (and distros related mailing-list first)
I'm pretty sure its mentioned in the cover-letter, but I doubt any user-space developers read that. I think KVM forum is a better way to reach them.
Thanks,
James
Hi James,
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 6:06 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org; Jonathan Cameron jonathan.cameron@huawei.com; Lorenzo Pieralisi lorenzo.pieralisi@linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
Hi Salil,
On 31/03/2023 17:42, Salil Mehta wrote:
From: James Morse james.morse@arm.com Sent: Friday, March 31, 2023 4:40 PM To: Salil Mehta salil.mehta@huawei.com; salil.mehta@opnsrc.net Cc: linaro-open-discussions@op-lists.linaro.org; Jonathan Cameron jonathan.cameron@huawei.com; Lorenzo Pieralisi lorenzo.pieralisi@linaro.org Subject: Re: FYI, Regarding the talk at KVM Forum 2023 in June (CFP 2nd April)
For a title I suggest: "Supporting hotplug on architectures that don't".
Sure, that sounds good, just that I would add below as well:
"Challenges Revisited in Supporting Virtual CPU Hotplug on architectures that don't".
I don't think we are revisiting the challenges, we're proposing this as the solution for any architecture with the same problem.
I think we are. Qemu code is 95% same as presented part of RFC V1 and in KVM Forum2020 but this time with a support to a new kernel approach. There are still old design problems lingering which need open discussion and there are some new problems which needs mention. These could not be discussed earlier because Qemu community was not interested to discuss till they hear from ARM about the resolution of the kernel/ARM arch problems mentioned in the earlier presentation. This was the official feedback of the first presentation I gave in KVMForum2020. So I have to continue from there...
Fair enough - I assumed the kernel support was the principle challenge!
It was the foremost challenge without which Qemu changes were irrelevant as you have seen from the feedback of the last presentation. Now, that the ARM has proactively come up with a feasible solution to support virtual CPU Hotplug without having to mess with physical bits; there is a strong reason to persuade community to start a Qemu discussion as well.
Please don’t take Qemu changes for granted. There are already known hard bits in Qemu. It is very important to present them clearly. Forum's like these are very important From Huawei's perspective for the reasons you already know. We don’t have many channels to discuss quickly.
Kernel approach of handling ACPI Hotplug events is new and we are disclosing it to wider public in this forum and asking for their feedback. But then this also opens the door for Qemu discussions.
Adding 'virtual', confuses what the 'that don't' was supposed to be about, but I agree its too terse.
How about "Supporting vCPU hotplug on architectures that don't have CPU hotplug (volume 2)"
I would stick with revisiting as I have to present Qemu updates here and then Jump to what is being done extra from the last presentation to support new kernel Changes. This is very important from Qemu perspective.
Sure, fine by me. I wasn't aware of the Qemu discussions.
[...]
Comparing it to what you've written. I'm not sure the abstract needs any technical details. We can mention the history in the talk, but I don't think its useful to dwell on it.
Agreed, A rough summary might help organizers weigh in terms of relevance when considering whether to accept the talk in case there are many talks submitted. Whole idea is to convince them that this talk is relevant which we think it is. (I was given feedback about the same in the past)
We should put this sort of stuff in the notes section for the reviewers: | This is a follow up to where the problem was first presented at KVM forum, we've now got | a solution we think works for arm64 and probably others. | There are rough edges, so this would benefit from discussion with orchestration | developers.
Sure, we will add few bits there as well. But lets stick with some point wise rough abstract in the mains as that generates the interest of the audiences. It is also important to attract the audiences on the basis of relevance :)
The rough edge being the present mask in /sys. I think the orchestrator folk need to be aware that this adds a new 'enabled' state, and they can't just rely on 'present'. (they don't appear to today)
You will get an answer only if any cloud vendors are present (I hope they are). I think you should raise this in the LKML (and distros related mailing-list first)
I'm pretty sure its mentioned in the cover-letter, but I doubt any user-space developers read that. I think KVM forum is a better way to reach them.
Sure.
On this note will submit the abstract to KVMForum2023!
Thanks Salil
linaro-open-discussions@op-lists.linaro.org
-
James Morse -
Salil Mehta