When I first came across the issue described in the previous post, I was baffled. I tried to reduce the problem, but whenever I tried to manually copy out bits I thought could be the problem, the problem went away. The “building up” solution not working, I started to try at the “breaking down” solution, starting from the full code that demonstrated the problem and deleting irrelevant code. But this was in a file more than a thousand lines long, and the broken code had many dependencies on other parts of this file, making stripping it down difficult.
But I had a tool in my toolbox that could help. I’ve been a fan of delta debugging for a long time. With delta debugging, you define an “interestingness” criterion, and provide this, along with a large but interesting input to a reducer tool, and it will attempt to reduce the code to the smallest example that is still interesting per the provided criterion.
The core of my criteria was:
- The file continues to compile without error on Clang.
- The file continues to produce errors on GCC.
- The GCC errors continue to mention a deleted lambda destructor.
Without the last one, I was worried the reducer would instead find other functionality that differed between GCC and Clang that was different from the original bug I was trying to troubleshoot.
Aside from this, the C-Reduce documentation suggests preprocessing input files before running the reducer on it. I opted not to do this, since GCC and Clang in this case were using different standard libraries (macOS headers in the Clang case and GNU/Linux headers in the GCC case), and if the problem came down to different definitions in the standard libraries, I didn’t want to preprocess that out and then not find a difference any more, or discover other unrelated compiler differences. To make this work, I opted to remove all #include directives from the file to be reduced, and instead have my interestingness script prepend all #includes, so that even fully-minimized files would continue to have all #includes.
I did some quick tests on my interestingness check and determined it was working as intended.
Actually installing C-Reduce was not trivial. Attempting a standard autotools-style install from source, it informed me that it had a dependency on LLVM 9, which is quite old, and compiling LLVM from source is not a small endeavor. It’s not hard, but it can take a long time and use a lot of disk space. Besides, I didn’t know if the Clang in LLVM 9 supported all the newer C++ constructs I was using, which could render the whole thing moot.
Fortunately, I was able to work around this. Noticing the only part of C-Reduce that used LLVM was clang_reduce, I figured I might be able to excise just this component and see if it could run without it. Using the CMake-based build instead (as it does not require as much tooling to regenerate the build files), I removed the LLVM dependency and commented out the add_subdirectory for clang_reduce.
LLVM is not the only dependency. C-Reduce also uses a lot of Perl. I’d previously built and installed a Perl interpreter on this machine and installed cpanminus, so installing its Perl module dependencies was a simple matter:
cpanm Exporter::Lite File::Which Getopt::Tabular Regexp::Common Term::ReadKeyWith its Perl dependencies installed, I was able to build and install C-Reduce successfully. Hooray! Running C-Reduce looks like this:
creduce --timeout 10 ./interesting.sh repro.cppBut without clang_reduce installed, there’s an error about it not being able to find clang_reduce. Very well. Digging inside creduce (which is a Perl script), it has a list of “methods” it applies. creduce --help helpfully explains that passes can be removed with --remove-pass, but I wasn’t able to get this to work. Instead, I hacked up the creduce Perl script directly. 2/3rds of the way through the script, there’s an array of @all_methods. There are two pass modules that use clang_reduce: pass_clang and pass_clang_binsrch. I crudely commented out all elements mentioning these.
With that in place, C-Reduce successfully reduced my file down to only a few lines. It seems the Clang passes were not needed for it to be effective. It did take a long time for it to run, though – I have a feeling the Clang passes could have provided substantial speedups, allowing the other passes to do less work. Also, the resulting “reduced” file still had some unnecessary namespace declarations. These were trivial to identify as being irrelevant to the reproduction, but it’s possible the Clang passes would have been able to remove them.
All-in-all, C-Reduce is a very handy tool for tracking down compiler behaviors you can’t explain. Delta debugging in general can also be used on other types of programs too, as long as you can form an interestingness check and provide input to the program as a single textual file.
Tags: C-Reduce, Debugging, Delta Debugging