Hey Paul,

As ever, I completely agree.

Underneath this work, there is a question in my mind that looks at knowing where critical parts of files are. Some file types will naturally localise errors into chucks, others will spread them evenly throughout the file object, and some will not tolerate any errors at all (depending on how the bitstream is organised in the file object, and now the file object is organised on the storage medium).

If a single bit in a txt file is errored, the error is always only localised to the affected byte – any burst errors would be dispersed at byte level throughout the file, and any cluster errors the same.

If a single bit in a mp3 file is errored, as long as the bit is not in the critical setup/declaration parts of the header, then errors are confined to the frame that contains the errored bit. Any burst errors would be dispersed at a frame level throughout the file – (without doing an impact study in mp3, I’m not sure what size error mp3 frames are tolerant of).

It’s not hard to imagine there are some files in which any damage to any bit results in a complete render failure.

What we can see from these jpgs is that some errors will cause critical failure (it follows that if a one bit error is capable of preventing the object from being rendered, at some offsets an error of any size can do the same)

Quick count of the failed render files: 1930 failed to render in script. That means we can assume that a jpg of a comparable size encountering any number of distributed bit errors is 1.37% per bit. Without accounting for a location bias, a full byte flip error has a ~10% chance of disabling a file.

This asks questions (1) what is the location bias – I can see from my data that the first 6 bytes of jpg are critical – any error of any bit results in a 91.6% chance the whole file fail to render. (2) If I throw big enough errors at the file, when does it behave differently? (3) What if I spread those errors around the file (transmission or storage cluster errors) or in a single block (transmission or write errors)

So yupe, totally take your point! it is a hypothetical attack, but it has helped to start some interesting discussions already, so I’m very happy with that!

Good spot,

Fixed now, thanks.

Indeed. When I get round to it, I’m going to loop them all into a movie. Even at full frame rate its going to be a very long and dull movie!

Especially:

“I’m not expecting a direct correlation between the bit and the damage (though it’d be neat if flipping bit 17 always resulted in a cyan swathe across the image for example) but rather that images that are broken may all produce similar artefacts/shapes?”

In my head, there IS a few bytes that are critical to a file – from my brief foray into bit mashing I saw that the first few bytes of jpeg are critical. Given how inefficient my code was, I’ve not been able to really run as many tests as I wanted – however, I’d love to see the aggregated results of a decent number of different jpgs (a few hundred, covering frame size and compression aggression) and see there are any relative or absolute patterns to errors and error percentages.

For example, we know that jpeg is built of 8×8 blocks, each block having undergone a DCT, and the result is stored as one of the main parts of the jpeg file. This means that any changes to the MSB side of a byte will result in a larger ‘per DCT block’ error than a change on the LSB end – this means that we are more ‘tolerant’ (visually and arithmetically) of LSB skewed errors in most of the jpeg file than MSB errors.

This offers some questions: At a base level, do these sorts of patterns affect how we should be clustering different file types to reduce the likelihood of errors / damage in the long term (or do simply not care because modern error correction and concealment and distributed bit writing methods remove this issue?…)

What other patterns can we see? Are there critical portions of files that would benefit from a higher bit budget?