Analysis of ChatGPT's capabilities for solving problems of reverse-engineering of software


  • Volodymyr Sokolov Institute of special communication and information protection of National technical university of Ukraine “Igor Sikorsky Kyiv polytechnic institute”, Kyiv, Ukraine



ChatGPT, reverse engineering, software


The article presents the results of the analysis and evaluation of the capabilities of the open version of the chatbot with artificial intelligence ChatGPT 3.5 for solving typical problems of software reverse engineering. Three classes of reverse engineering tasks were selected for analysis: source code analysis, binary code analysis, and data models analysis. In each class of tasks, the most typical tasks were selected taking into account the limitations of ChatGPT regarding the processing of graphical models and the amount of input and output data, and sets of test tasks were developed for each task. As an approach to assessment, an approach similar to the assessment of competencies of higher education applicants after studying the relevant discipline was chosen. The following were considered as criteria for evaluating answers to test tasks: correctness (rightness, coincidence with expectations); completeness (obtaining the final result); accuracy (solving the task without additional questions); reasonableness (availability of explanations and answers to questions). The following scale was used and described for evaluations: excellent, very good, good, satisfactory, sufficient, unsatisfactory. During the testing for each test task, tasks statement and all necessary data were entered through the ChatGPT interface. As a result of the analysis, it was determined that ChatGPT better solves the problems of source code analysis (excellent and very good grades were obtained for semantic and structural analysis, restoration of mathematical support, quality assessment, security audit and refactoring, as well as for conversion to another programming language), tasks of decompilation of IDA pseudocode into complete C source code, tasks of reverse engineering of relational databases, and tasks of generating YARA rules for recognizing file formats. Unsatisfactory grades were obtained for dynamic analysis of assembly code and determination of binary file structures of non-standard formats. ChatGPT solves the rest of the problems well and satisfactorily, but requires checking the results, clarifying queries and prompts, as well as manual error correction in some cases. ChatGPT errors were observed when analyzing binary data represented by hexadecimal symbols, as well as errors in generated scripts for programming in IDA. On the basis of the set grades, conclusions were made regarding the expediency, possibility, or impracticality of using ChatGPT to solve each type of software reverse engineering problems, and appropriate recommendations were also provided. Prospects for further research include testing new versions of ChatGPT and other similar artificial intelligence systems regarding the capabilities of analyzing and synthesizing graphical models of software.

Author Biography

Volodymyr Sokolov, Institute of special communication and information protection of National technical university of Ukraine “Igor Sikorsky Kyiv polytechnic institute”, Kyiv

candidate of technical sciences, associate professor, associate professor at the cybersecurity and application of information systems and technologies academic department


X. Xu et al., “LmPa: Improving Decompilation by Synergy of Large Language Model and Program Analysis”, arXiv preprint arXiv:2306.02546, 2023. doi:

S. Frieder et al. “Mathematical capabilities of chatgpt”, arXiv preprint arXiv:2301.13867, 2023. doi:

Z. Wang, Z. Lan, and L. Peng, “ChatGPT for Software Security: Exploring the Strengths and Limitations of ChatGPT in the Security Applications”, arXiv preprint arXiv:2307.12488, 2023. doi:

W. Tann et al., “Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions”, arXiv preprint arXiv:2308.10443, 2023. doi:

M. Gupta, C. kiri, K. Aryal, E. Parker, and L. Praharaj, “From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy”, arXiv preprint arXiv:2307.00691, 2023. doi:

A. Qammar et al., “Chatbots to ChatGPT in a Cybersecurity Space: Evolution, Vulnerabilities, Attacks, Challenges, and Future Recommendations”, arXiv preprint arXiv:2306.09255, 2023. doi:

B. Yetiştiren et al., “Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT”, arXiv preprint arXiv:2304.10778, 2023. doi:

A. Sadik et al., “Analysis of ChatGPT on Source Code”, arXiv preprint arXiv:2306.00597, 2023. doi:

J. White et al., “Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design”, arXiv preprint arXiv:2303.07839, 2023. doi:

A. Buscemi, “A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages”, arXiv preprint arXiv:2308.04477, 2023. doi:

Md. M. Rahman, and W. Yutaka, “ChatGPT for Education and Research: Opportunities, Threats, and Strategies”, Applied Sciences, vol. 13 (9), art. 5783, 2023. doi:



How to Cite

Sokolov, V. (2023). Analysis of ChatGPT’s capabilities for solving problems of reverse-engineering of software. Collection "Information Technology and Security", 11(2), 193–205.