Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques

GUO Yuchen; ZHU Xiaoyan

doi:10.7652/xjtuxb202407019

您当前的位置：

首页 >

文章列表页 >

Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques

更新时间：2025-07-28

- Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques
- Vol. 58, Issue 7, Pages: 203-213(2024)
- 作者机构：
  
  西安交通大学计算机科学与技术学院,西安,710049
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202407019
  CLC： TP311.5
- Online First：10 July 2024，
  
  Published：2024
- 稿件说明：
移动端阅览
GUO Yuchen, ZHU Xiaoyan. Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques[J]. 2024, 58(7): 203-213.
DOI：

GUO Yuchen, ZHU Xiaoyan. Impact of Skewed Effort Distribution in Effort-Aware Software Defect Prediction and Its Validation Techniques[J]. 2024, 58(7): 203-213. DOI： 10.7652/xjtuxb202407019.

摘要

针对工作量感知软件缺陷预测中传统模型测试评估方法存在偏差这一问题

采用偏斜分布的偏度作为数值特征

研究了3种主要测试评估方法的测试集在工作量偏度的偏差和与其对应的估计误差

并基于偏度偏差较小的采样余量方法

提出一种改进方法——后采样方法

所提后采样方法能够保持测试集的类标签比例以避免生成无效测试集。研究结果表明:最常用的十折交叉验证方法偏度偏差最大

其估计误差也最大; 与十折交叉验证相比

改进方法性能估计误差减少约4.9%～26.9%; 与采样余量方法相比

改进方法不会产生无效测试集

并证明了减小测试集偏度偏差以减少估计误差的有效性。所提后采样方法为工作量感知软件缺陷预测提供了一种更可靠的测试评估方法

能够更准确地评估模型性能。

Abstract

To handle the bias in traditional model validation methods for effort-aware software defect prediction

the skewness of skewed distribution is used as numeric feature to investigate the bias of effort skewness and estimation bias in the datasets of three major validation techniques. Based on out-of-sample method which is less biased in effort skewness

a refined hold-to-sample method is proposed

which maintains the class label proportion of the testing data to avoid generating invalid testing data. Results show that the commonly used 10-fold cross-validation method has the greatest bias in effort skewness and the greatest estimation bias. In comparison with this method

our refined method reduces the evaluation bias by 4.9%—26.9%; in comparison with the out-of-sample

it does not generate invalid testing data. Experiments confirm the effectiveness of reducing the evaluation bias by decreasing bias of skewness in testing data. The hold-to-sample method provides a more reliable validation method for effort-aware software defect prediction and can evaluate model performance more accurately.

关键词

Keywords

references

HAMILL M, GOSEVA-POPSTOJANOVA K. Analyzing and predicting effort associated with finding and fixing software faults [J]. Information and Software Technology, 2017, 87: 1-18.

LESSMANN S, BAESENS B, MUES C, et al. Benchmarking classification models for software defect prediction: a proposed framework and novel findings [J]. IEEE Transactions on Software Engineering, 2008, 34(4): 485-496.

YU Xiao, RAO Jiqing, LIU Lei, et al. Improving effort-aware defect prediction by directly learning to rank software modules [J]. Information and Software Technology, 2024, 165: 107250.

KAMEI Y, SHIHAB E, ADAMS B, et al. A large-scale empirical study of just-in-time quality assurance [J]. IEEE Transactions on Software Engineering, 2013, 39(6): 757-773.

YANG Xinli, LO D, XIA Xin, et al. TLEL: a two-layer ensemble learning approach for just-in-time defect prediction [J]. Information and Software Technology, 2017, 87: 206-220.

刘旭同, 郭肇强, 刘释然, 等. 软件缺陷预测模型间的比较实验:问题、进展与挑战 [J]. 软件学报, 2023, 34(2): 582-624.

LIU Xutong, GUO Zhaoqiang, LIU Shiran, et al. Comparing software defect prediction models: research problem, progress, and challenges [J]. Journal of Software, 2023, 34(2): 582-624.

MENDE T, KOSCHKE R. Effort-aware defect prediction models [C]//2010 14th European Conference on Software Maintenance and Reengineering. Piscataway, NJ, USA: IEEE, 2010: 107-116.

YANG Yibiao, ZHOU Yuming, LIU Jinping, et al. Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models [C]//Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York, USA: Association for Computing Machinery, 2016: 157-168.

ZHOU Yuming, YANG Yibiao, LU Hongmin, et al. How far we have progressed in the journey?: an examination of cross-project defect prediction [J]. ACM Transactions on Software Engineering and Methodology, 2018, 27(1): 1-51.

NI Chao, XIA Xin, LO D, et al. Revisiting supervised and unsupervised methods for effort-aware cross-project defect prediction [J]. IEEE Transactions on Software Engineering, 2022, 48(3): 786-802.

HUANG Qiao, XIA Xin, LO D. Revisiting supervised and unsupervised models for effort-aware just-in-time defect prediction [J]. Empirical Software Engineering, 2019, 24(5): 2823-2862.

蔡亮, 范元瑞, 鄢萌, 等. 即时软件缺陷预测研究进展 [J]. 软件学报, 2019, 30(5): 1288-1307.

CAI Liang, FAN Yuanrui, YAN Meng, et al. Just-in-time software defect prediction: literature review [J]. Journal of Software, 2019, 30(5): 1288-1307.

TANTITHAMTHAVORN C, MCINTOSH S, HASSAN A E, et al. An empirical comparison of model validation techniques for defect prediction models [J]. IEEE Transactions on Software Engineering, 2017, 43(1): 1-18.

GUO Yuchen, SHEPPERD M, LI Ning. Poster: bridging effort-aware prediction and strong classification-a just-in-time software defect prediction study[C]//2018 IEEE/ACM40th International Conference on Software Engineering: Companion(ICSE-Companion). Piscataway, NJ, USA: IEEE, 2018: 325-326.

SHIHAB E, KAMEI Y, ADAMS B, et al. Is lines of code a good measure of effort in effort-aware models? [J]. Information and Software Technology, 2013, 55(11): 1981-1993.

NI Chao, XIA Xin, LO D, et al. Just-in-time defect prediction on JavaScript projects: a replication study [J]. ACM Transactions on Software Engineering and Methodology, 2022, 31(4): 76.

LI Fuyang, YANG Peixin, KEUNG J W, et al.Revisiting ‘revisiting supervised methods for effort-aware cross-project defect prediction' [J]. IET Software, 2023, 17(4): 472-495.

KING R S, JULSTROM B. Applied statistics using the computer [M]. Palo Alto, CA, USA: Distributed by Mayfield Pub. Co., 1982.

GONG Lina, ZHANG Haoxiang, ZHANG Jingxuan, et al. A comprehensive investigation of the impact of class overlap on software defect prediction [J]. IEEE Transactions on Software Engineering, 2023, 49(4): 2440-2458.

ZHANG Tanghaoan, YU Yue, MAO Xinjun, et al. FENSE: a feature-based ensemble modeling approach to cross-project just-in-time defect prediction [J]. Empirical Software Engineering, 2022, 27(7): 162.

PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al. Scikit-learn: machine learning in python [J]. The Journal of Machine Learning Research, 2011, 12: 2825-2830.

SONG Qinbao, GUO Yuchen, SHEPPERD M. A comprehensive investigation of the role of imbalanced learning for software defect prediction [J]. IEEE Transactions on Software Engineering, 2019, 45(12): 1253-1269.

BOETTICHER G. The PROMISE repository of empirical software engineering data [EB/OL]. [2024-01-01]. http://promisedata.org/repository.

D'AMBROS M, LANZA M, ROBBES R. Evaluating defect prediction approaches: a benchmark and an extensive comparison [J]. Empirical Software Engineering, 2012, 17(4): 531-577.[25] TIAN Yuli, LI Ning, TIAN J, et al. How well just-in-time defect prediction techniques enhance software reliability?[C]//2020 IEEE 20th International Conference on Software Quality, Reliability and Security(QRS). Piscataway, NJ, USA: IEEE, 2020: 212-221.

HOLLANDER M, WOLFE D A. Nonparametric statistical methods [M]. New York: Wiley, 1973: 503.

BENJAMINI Y, HOCHBERG Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing [J]. Journal of the Royal Statistical Society: Series B Methodological, 1995, 57(1): 289-300.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Numerical Simulation of the Energy Conversion Process in Thermoacoustic Engines with Different Stack Materials

Windage Loss and Flow Characteristics in Impeller Back Gaps of sCO2 Radial Inflow Turbines

I/O Performance Modeling and Prediction Method for Large-Scale Parallel Applications Using Machine Learning

Effects of Expansion Angle Crater Film Hole on the Film Cooling Characteristics of a Double-Wall Structure

Study on the Thermodynamic Properties in the Anomalous Region of Supercritical Water

Related Author

No data

Related Institution

No data

⁰