Flattening the Anvil
In order to flatten an anvil, you must first identify its imperfections, and determine the amount of heat and pressure needed in order to perfectly flatten its surface. With a flattened anvil, other tools and armours can be flattened and corrected as well. But even an expert Blacksmith can have a hard time working with an imperfect anvil.
Analogously, we think of ink! integration and end-to-end (e2e for short) testing environments as an anvil on which we can resolve imperfections of ink! smart contracts. We have discovered integration tests for ink! smart contracts lack some of the functionalities, or present differences, when compared with e2e tests. On the other hand, integration tests run significantly faster than e2e tests. Hence, the full availability of integration test functionalities shortens and improves development time. As a result, if a full range of functionalities were provided, it could reduce testing and QA times.
Spotting the Differences
We noticed these missing functionalities in integration tests while developing fuzzing detectors for Scout Audit, a tool for detecting vulnerabilities in ink! smart contracts developed by our R&D team at CoinFabrik. During this work, we identified differences between integration tests and end-to-end tests in two functions, default_account() and set_contract_storage(), and we noticed that in some cases we were able to perform e2e tests but the integration tests failed due to three test-related functions not being implemented. These findings were the groundwork for a project we wanted to work on. We decided to apply for a grant at the Web3 Foundation program, named “On Ink Integration Tests” filling this grant application for the grant #1875 On Ink Integration Tests.
This post summarizes the results of the first of a two-milestones project in which we focused on identifying functions that might be used in integration tests which are either missing or result in differences when compared with their analogous in e2e tests. During a space of two weeks, we worked on prominent test-related functions where there was evidence of missing implementations or differences, with the aim of developing ink! smart contracts that work as test cases in order to assess the implementation in both test environments. Our goal was to understand the missing functions or differing properties, and to provide an implementation idea and a time estimation of its development.
The main result of this work is a deep analysis of 11 functions that showed problems, of which 9 had missing implementation in integration tests as indicated with the unimplemented!() macro, 1 had an implementation difference, and 1 had a missing restriction on the integration test when compared to e2e. The function with the implementation difference was default_accounts(), which provided different account hashes in integration tests vs e2e tests. The missing restriction was found on the function set_contract_storage(), which had a missing restriction on the integration test when compared to e2e.
Moreover, we established a list of 13 additional functions which need further analysis. These functions are accessible through integration and e2e tests, they show implementations for both environments, but should be analyzed to determine if they have implementation differences, or work exactly as expected.
Looking Forward
Just as a blacksmith needs a flattened anvil before working on a new weapon or armour, implementing these missing features in integration testing, and correcting these differences or deficiencies is the first step toward building a tool or set of tools that assist developers in their testing and QA of ink! smart contract code.
We envision using smart contract classification together with AI to build automated or semi automated tools for Testing and QA of ink! smart contracts.
