工作量感知软件缺陷预测中偏斜分布的影响及测试评估方法

郭育晨; 朱晓燕

doi:10.7652/xjtuxb202407019

您当前的位置：

首页 >

文章列表页 >

工作量感知软件缺陷预测中偏斜分布的影响及测试评估方法

—————————————————————— | 更新时间：2025-07-28

- 工作量感知软件缺陷预测中偏斜分布的影响及测试评估方法
- Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques
- 西安交通大学学报 2024年58卷第7期页码：203-213
- 作者机构：
  
  西安交通大学计算机科学与技术学院,西安,710049
- 作者简介：
  
  : 2024-01-31。作者简介: 郭育晨(1990—),男,博士生
  朱晓燕(通信作者),女,副教授,博士生导师。基金项目: 国家自然科学基金资助项目(92046009)
- 基金信息：
  
  : 2024-01-31。作者简介: 郭育晨(1990—),男,博士生;朱晓燕(通信作者),女,副教授,博士生导师。基金项目: 国家自然科学基金资助项目(92046009)
- DOI：10.7652/xjtuxb202407019
  中图分类号： TP311.5
- 网络首发：2024-07-10，
  
  纸质出版：2024
- 稿件说明：
移动端阅览
郭育晨, 朱晓燕. 工作量感知软件缺陷预测中偏斜分布的影响及测试评估方法[J]. 西安交通大学学报, 2024,58(7):203-213.

GUO Yuchen, ZHU Xiaoyan. Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques[J]. 2024, 58(7): 203-213.
郭育晨, 朱晓燕. 工作量感知软件缺陷预测中偏斜分布的影响及测试评估方法[J]. 西安交通大学学报, 2024,58(7):203-213. DOI： 10.7652/xjtuxb202407019.

GUO Yuchen, ZHU Xiaoyan. Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques[J]. 2024, 58(7): 203-213. DOI： 10.7652/xjtuxb202407019.

摘要

针对工作量感知软件缺陷预测中传统模型测试评估方法存在偏差这一问题

采用偏斜分布的偏度作为数值特征

研究了3种主要测试评估方法的测试集在工作量偏度的偏差和与其对应的估计误差

并基于偏度偏差较小的采样余量方法

提出一种改进方法——后采样方法

所提后采样方法能够保持测试集的类标签比例以避免生成无效测试集。研究结果表明:最常用的十折交叉验证方法偏度偏差最大

其估计误差也最大; 与十折交叉验证相比

改进方法性能估计误差减少约4.9%～26.9%; 与采样余量方法相比

改进方法不会产生无效测试集

并证明了减小测试集偏度偏差以减少估计误差的有效性。所提后采样方法为工作量感知软件缺陷预测提供了一种更可靠的测试评估方法

能够更准确地评估模型性能。

Abstract

To handle the bias in traditional model validation methods for effort-aware software defect prediction

the skewness of skewed distribution is used as numeric feature to investigate the bias of effort skewness and estimation bias in the datasets of three major validation techniques. Based on out-of-sample method which is less biased in effort skewness

a refined hold-to-sample method is proposed

which maintains the class label proportion of the testing data to avoid generating invalid testing data. Results show that the commonly used 10-fold cross-validation method has the greatest bias in effort skewness and the greatest estimation bias. In comparison with this method

our refined method reduces the evaluation bias by 4.9%—26.9%; in comparison with the out-of-sample

it does not generate invalid testing data. Experiments confirm the effectiveness of reducing the evaluation bias by decreasing bias of skewness in testing data. The hold-to-sample method provides a more reliable validation method for effort-aware software defect prediction and can evaluate model performance more accurately.

关键词

Keywords

references

HAMILL M, GOSEVA-POPSTOJANOVA K. Analyzing and predicting effort associated with finding and fixing software faults [J]. Information and Software Technology, 2017, 87: 1-18.

LESSMANN S, BAESENS B, MUES C, et al. Benchmarking classification models for software defect prediction: a proposed framework and novel findings [J]. IEEE Transactions on Software Engineering, 2008, 34(4): 485-496.

YU Xiao, RAO Jiqing, LIU Lei, et al. Improving effort-aware defect prediction by directly learning to rank software modules [J]. Information and Software Technology, 2024, 165: 107250.

KAMEI Y, SHIHAB E, ADAMS B, et al. A large-scale empirical study of just-in-time quality assurance [J]. IEEE Transactions on Software Engineering, 2013, 39(6): 757-773.

YANG Xinli, LO D, XIA Xin, et al. TLEL: a two-layer ensemble learning approach for just-in-time defect prediction [J]. Information and Software Technology, 2017, 87: 206-220.

刘旭同, 郭肇强, 刘释然, 等. 软件缺陷预测模型间的比较实验:问题、进展与挑战 [J]. 软件学报, 2023, 34(2): 582-624.

LIU Xutong, GUO Zhaoqiang, LIU Shiran, et al. Comparing software defect prediction models: research problem, progress, and challenges [J]. Journal of Software, 2023, 34(2): 582-624.

MENDE T, KOSCHKE R. Effort-aware defect prediction models [C]//2010 14th European Conference on Software Maintenance and Reengineering. Piscataway, NJ, USA: IEEE, 2010: 107-116.

YANG Yibiao, ZHOU Yuming, LIU Jinping, et al. Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models [C]//Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York, USA: Association for Computing Machinery, 2016: 157-168.

ZHOU Yuming, YANG Yibiao, LU Hongmin, et al. How far we have progressed in the journey?: an examination of cross-project defect prediction [J]. ACM Transactions on Software Engineering and Methodology, 2018, 27(1): 1-51.

NI Chao, XIA Xin, LO D, et al. Revisiting supervised and unsupervised methods for effort-aware cross-project defect prediction [J]. IEEE Transactions on Software Engineering, 2022, 48(3): 786-802.

HUANG Qiao, XIA Xin, LO D. Revisiting supervised and unsupervised models for effort-aware just-in-time defect prediction [J]. Empirical Software Engineering, 2019, 24(5): 2823-2862.

蔡亮, 范元瑞, 鄢萌, 等. 即时软件缺陷预测研究进展 [J]. 软件学报, 2019, 30(5): 1288-1307.

CAI Liang, FAN Yuanrui, YAN Meng, et al. Just-in-time software defect prediction: literature review [J]. Journal of Software, 2019, 30(5): 1288-1307.

TANTITHAMTHAVORN C, MCINTOSH S, HASSAN A E, et al. An empirical comparison of model validation techniques for defect prediction models [J]. IEEE Transactions on Software Engineering, 2017, 43(1): 1-18.

GUO Yuchen, SHEPPERD M, LI Ning. Poster: bridging effort-aware prediction and strong classification-a just-in-time software defect prediction study[C]//2018 IEEE/ACM40th International Conference on Software Engineering: Companion(ICSE-Companion). Piscataway, NJ, USA: IEEE, 2018: 325-326.

SHIHAB E, KAMEI Y, ADAMS B, et al. Is lines of code a good measure of effort in effort-aware models? [J]. Information and Software Technology, 2013, 55(11): 1981-1993.

NI Chao, XIA Xin, LO D, et al. Just-in-time defect prediction on JavaScript projects: a replication study [J]. ACM Transactions on Software Engineering and Methodology, 2022, 31(4): 76.

LI Fuyang, YANG Peixin, KEUNG J W, et al.Revisiting ‘revisiting supervised methods for effort-aware cross-project defect prediction' [J]. IET Software, 2023, 17(4): 472-495.

KING R S, JULSTROM B. Applied statistics using the computer [M]. Palo Alto, CA, USA: Distributed by Mayfield Pub. Co., 1982.

GONG Lina, ZHANG Haoxiang, ZHANG Jingxuan, et al. A comprehensive investigation of the impact of class overlap on software defect prediction [J]. IEEE Transactions on Software Engineering, 2023, 49(4): 2440-2458.

ZHANG Tanghaoan, YU Yue, MAO Xinjun, et al. FENSE: a feature-based ensemble modeling approach to cross-project just-in-time defect prediction [J]. Empirical Software Engineering, 2022, 27(7): 162.

PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al. Scikit-learn: machine learning in python [J]. The Journal of Machine Learning Research, 2011, 12: 2825-2830.

SONG Qinbao, GUO Yuchen, SHEPPERD M. A comprehensive investigation of the role of imbalanced learning for software defect prediction [J]. IEEE Transactions on Software Engineering, 2019, 45(12): 1253-1269.

BOETTICHER G. The PROMISE repository of empirical software engineering data [EB/OL]. [2024-01-01]. http://promisedata.org/repository.

D'AMBROS M, LANZA M, ROBBES R. Evaluating defect prediction approaches: a benchmark and an extensive comparison [J]. Empirical Software Engineering, 2012, 17(4): 531-577.[25] TIAN Yuli, LI Ning, TIAN J, et al. How well just-in-time defect prediction techniques enhance software reliability?[C]//2020 IEEE 20th International Conference on Software Quality, Reliability and Security(QRS). Piscataway, NJ, USA: IEEE, 2020: 212-221.

HOLLANDER M, WOLFE D A. Nonparametric statistical methods [M]. New York: Wiley, 1973: 503.

BENJAMINI Y, HOCHBERG Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing [J]. Journal of the Royal Statistical Society: Series B Methodological, 1995, 57(1): 289-300.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

冷库水平空气幕热湿阻隔性能数值模拟研究

电子级超纯氮深冷空分流程节能优化研究

融合物理信息与深度算子网络的变压器绕组热-流场快速计算方法

离子液体改性燃料的电流体动力学效应研究

采用试验设计方法的第三代CO2气体箔轴承箔片几何参数敏感性分析