Property-Based Testing of Browser Rendering Engines with a Consensus Oracle

Verifying that a browser rendering engine correctly renders all valid web pages is a challenging problem due to the size of the input space (valid web pages) and the challenge of knowing whether the rendering for any given page is correct (the test Oracle problem). We propose a property-based testing (PBT) approach that uses controllable grammar-based generators for creating the inputs (web pages containing HTML5 and CSS3) and that uses a (lack of) consensus among multiple rendering engines to identify failing test cases. To assist with root-cause analysis the system uses test shrinking to report much smaller versions of failing test cases. This technique may also be used to test other software systems where there are multiple preexisting implementations of the System under Test (SUT).