Detection Tools in Smart Contracts: Reevaluating the Effectiveness of Reentrant Detection Methods Through Replication

Abstract

Smart contracts are programs that facilitate financial exchanges through cryptocurrency, making it crucial that they are secure and resistant to exploitation. One prominent vulnerability in smart contracts is reentrancy, which has led to the theft of millions of dollars in cryptocurrency. To address this issue, many reentrancy detection tools have been developed and tested over the years. This study aims to validate the findings of a previous study by Zheng, et. al, which concluded that five widely used detection tools were largely ineffective at identifying reentrancy vulnerabilities. The tools examined were Oyente, Mythril, Sailfish, Smartian, and Securify. To replicate the original study, we set up and ran each detection tool within Docker containers. While three of the detection tools have Docker images to pull straight from Docker hub, the other two requires us to build our own images which comes with some issues in version compatibility. We then analyzed 139,424 smart contracts pulled from the original study's git repository of test data by creating python scripts to feed the data into each detection tool. The original study found that 99.8% of the detected vulnerabilities were false positives. We anticipate that our results will align with Zheng et. al's findings, although there may be some improvement in detection accuracy due to updates to Mythril since the initial study. Our findings will highlight the ineffectiveness of modern tools in detecting reentrancy vulnerabilities in smart contracts.