Visual Studio
A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.
5,044 questions
How do I get coverage data from a fuzz test?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 61%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIB%3C/text%3E%3C/svg%3E)
Ian Brockbank
0
Reputation points
I have implemented a fuzz test in Visual Studio using /fsanitize=fuzzer /fsanitize=address.
Since the LLVM fuzzing engine uses coverage to direct the fuzz-testing, it should be able to output this as an artefact of the test run.
When I did this in a previous project using a clang build, I was able to get coverage data out using llvm-cov. This doesn't seem to work with this project built with Visual Studio/msvc++.
Since this is for automated testing, I really need a method that works from the command line, which I can wrap in a script.
How can I generate a coverage report from the fuzzer in Visual Studio?
Visual Studio
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 78%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETS%3C/text%3E%3C/svg%3E)
Tianyu Sun-MSFT 30,351 Reputation points Microsoft Vendor2024-08-20T02:48:05.8933333+00:00 Hello @Ian Brockbank ,
Welcome to Microsoft Q&A forum.
When I did this in a previous project using a clang build, I was able to get coverage data out using llvm-cov. This doesn’t seem to work with this project built with Visual Studio/msvc++.
Looks like the /fsanitize=fuzzer compiler option enables experimental support for LibFuzzer, and the code coverage information for libFuzzer is provided by LLVM’s SanitizerCoverage instrumentation. And, LibFuzzer requires a matching version of Clang.
Did you also try to install the
C++ Clang Compiler for WindowsandMSBuild support for LLVM(clang-cl) toolsetcomponents from VS Installer > Modify > Individual components and set the platform toolset toLLVM(clang-cl)to test and check? -
Ian Brockbank 0 Reputation points
2024-08-20T08:39:07.94+00:00 @suntianyu sun The fuzz testing is working fine as expected. The only part missing is extracting the coverage data.
Previously I got the raw coverage data by setting environment variable LLVM_PROFILE_FILE to my_test.profraw, and I could then use llvm-cov to parse the captured data and generate the coverage reports. When running the version built using MSVC, this doesn't give me the my_test.profraw file.
-
Tianyu Sun-MSFT 30,351 Reputation points Microsoft Vendor
2024-08-21T09:36:44.62+00:00 @Ian Brockbank I doubt that some settings are missing.
Did you also try to use flags like
-fprofile-instr-generate,-fcoverage-mappingwhen building the test by using MSVC(maybe add the flags to the linker)?
Sign in to comment
Sign in to answer