Hi Anmar,
Sorry for top posting -- it's outlook.
lkp-tests does have the compare scripts
https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/compare https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/ncompare
However they work on data from local files, which does not scale. So we created
https://gitee.com/openeuler/compass-ci/blob/master/sbin/compare
to work on data from ElasticSearch database.
Some compare functions can be reused as base for detect/bisecting performance regression. However we still need add more functions to actually detect regression and bisect good/bad state. That is the current state.
Thanks, Fengguang ---
Hello Fengguang,
I hope this email finds you well. We dug into the regression detection tool and discovered things like lib/compare and reports such as [1] clearly showing that LKP is able to do some regression analysis.
TBH, I'm a little confused as to what the regression detection tool we discussed is meant to do given that the functionality is already present in LKP. I recall you mentioning that CompassCI doesn't do regression detection so I might have misunderstood your statement or something got lost in translation. Our intention is to create a benchmark data analysis and regression tool that can be used by both CompassCI and LKFT.
Let me know what you think please. We can always jump on a call to sort this out if need be.
-- anmar
[1] https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/V725VQJIASSE2NY...
Il giorno 9 feb 2021, alle ore 12:50, wufengguang wufengguang@huawei.com ha scritto:
Hi Anmar,
Sorry for top posting -- it's outlook.
lkp-tests does have the compare scripts
https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/compare https://gitee.com/wu_fengguang/lkp-tests/blob/master/sbin/ncompare
However they work on data from local files, which does not scale. So we created
https://gitee.com/openeuler/compass-ci/blob/master/sbin/compare
to work on data from ElasticSearch database.
Some compare functions can be reused as base for detect/bisecting performance regression. However we still need add more functions to actually detect regression and bisect good/bad state. That is the current state.
Hi Fengguang, how are you?
We are about to start implementing our tool for detecting regressions. In this respect, I'd like to make sure I've got the following important point right. So let me sum up the situation with my words and ask you whether I'm right.
Functionalities for detecting regression are not mature yet in Compass-CI, whereas 01.org does report performance regressions. So, compass-ci and 01.org are both based on LKP, but 01.org guys have already implemented their own code for regression detection and reporting. This code has not to do with the compare functionalities your are developing.
Am I correct?
Thanks, Paolo
Thanks, Fengguang
Hello Fengguang,
I hope this email finds you well. We dug into the regression detection tool and discovered things like lib/compare and reports such as [1] clearly showing that LKP is able to do some regression analysis.
TBH, I'm a little confused as to what the regression detection tool we discussed is meant to do given that the functionality is already present in LKP. I recall you mentioning that CompassCI doesn't do regression detection so I might have misunderstood your statement or something got lost in translation. Our intention is to create a benchmark data analysis and regression tool that can be used by both CompassCI and LKFT.
Let me know what you think please. We can always jump on a call to sort this out if need be.
-- anmar
[1] https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/V725VQJIASSE2NY...
-
Paolo Valente -
wufengguang