Blog Posts

This note exists mainly because I couldn’t find much about this online. Information From Probability When compressing data with arithmetic coding, the length of the produced code is equal to the information content of the data (rounded to the next integer) with respect to the probabilistic model used. It takes...

Arithmetic codes are pretty useful for compression. You can find a number of implementations of the algorithm online, some work with static models, some adaptive. Often, an abstract interface is defined so users can call the algorithm with their own model implementation. It makes sense for the algorithm to interface...

I find that information theory, a core discipline of computer science, is given relatively little attention in a computer science degree. Although Hamming and Shannon are likely to be mentioned at some point, the only attribution to the field I remember having to study were Huffman trees, which are nice...