GB Mal 7z
Komga runs on the Java Virtual Machine (JVM). The JVM works differently than other native programs in regard to memory consumption.On startup, the JVM will reserve some memory from the OS, but that doesn't mean this memory is used by the application. If the OS needs to reclaim that memory, the JVM will try to release it.
GB Mal 7z
The double pages feature of the webreader requires image sizes to be available. This feature was added in v0.51.0. If your books have been analyzed before that version, you will need to re-analyze them in order for the double pages feature to work properly.
Add func.getattr=newest to the options in your /etc/fstab entry for the mergerfs volume. By default, mergerfs doesn't update the modified times for everything for performance reasons. This forces it to. In most cases the performance impact is negligible.
To try to give me more insight, I checked Windows Reliability Monitor History for that date. I found that the time it was created coincides with the time I tried to mod GTA IV with ICE Enhancer and texture packs.
Guess I posted too soon. Just figured it out. I had an idea to try to compress it with 7-Zip to see what the compression ratio was. I figured that if the compression ratio was extremely high, it's probably some kind of failed download that Chrome allocated space for.
It's most likely a backup or a leftover temporary file for a download. Also found huge files (1GB) in the exact same spot with no file extension and named 00000001 and 00000002. Opened the files using VLC (free video player capable of playing pretty much any video file) and found out that the files were actually videos I downloaded ages ago, but for some reason Chrome didn't remove these temporary files after the download had finished.
TL;DR I'm pretty sure these files are safe to delete without affecting the functionality of the program.(I've been a Windows power-user for 12 years and I'm a University student of Information Processing.)
Compression bombs that use the zip formatmust cope with the fact that DEFLATE,the compression algorithm most commonly supported by zip parsers,cannot achieve a compression ratio greater than 1032.For this reason, zip bombs typically rely on recursive decompression,nesting zip files within zip files to get an extra factor of 1032 with each layer.But the trick only works on implementations thatunzip recursively, and most do not.The best-known zip bomb,42.zip,expands to a formidable 4.5 PBif all six of its layers are recursively unzipped,but a trifling 0.6 MB at the top layer.Zip quines,like those of Ellingsenand Cox,which contain a copy of themselvesand thus expand infinitely if recursively unzipped,are likewise perfectly safe to unzip once.
The central directory is at the end of the zip file.It is a list of central directory headers.Each central directory header contains metadata for a single file,like its filename and CRC-32 checksum,and a backwards pointer to a local file header.A central directory header is 46 bytes long,plus the length of the filename.
This description of the zip format omits many details thatare not needed for understanding the zip bomb.For full information,refer tosection 4.3 of APPNOTE.TXTor The structure of a PKZip file by Florian Buchholz,or see the source code.
By compressing a long string of repeated bytes,we can produce a kernelof highly compressed data.By itself, the kernel's compression ratio cannotexceed the DEFLATE limit of 1032,so we want a way to reuse the kernel in many files,without making a separate copy of it in each file.We can do it by overlapping files:making many central directory headers point toa single file, whose data is the kernel.
We could make every central directory header have the same filenameas the local file header, but that too is unsatisfyingbecause it means that if extracted to disk,all the files will just overwrite each other and not take up more spacethan a single file.
We need to separate the local file headers for each file,while still reusing a single kernel.Simply concatenating all the local file headers does not work,because the zip parser will find a local file headerwhere it expects to find the beginning of a DEFLATE stream.But the idea will work, with a minor modification.We'll use a feature of DEFLATE, non-compressed blocks,to "quote" local file headersso that they appear to be part of the same DEFLATE streamthat terminates in the kernel.Every local file header(except the first)will be interpreted in two ways:as code (part of the structure of the zip file)and as data (part of the contents of a file).
A DEFLATE stream is a sequence ofblocks,where each block may be compressed or non-compressed.Compressed blocks are what we usually think of;for example the kernel is one big compressed block.But there are also non-compressed blocks,which start with a5-byte headerwith a length field that means simply, "output the next n bytes verbatim."Decompressing a non-compressed block means only stripping the 5-byte header.Compressed and non-compressed blocks may be intermixed freelyin a DEFLATE stream.The output is the concatenation ofdecompressing all the blocks in order.The "non-compressed" notion only has meaning at the DEFLATE layer;the file data still counts as "compressed" at the zip layer,no matter what kind of blocks are used.
The giant-steps feature only pays when you are not constrained by maximum output file size.In zblg.zip, we actually want to slow file growth as much as possibleso that the smallest file, containing the kernel, can be as large as possible.Using giant steps in zblg.zip actually decreases the compression ratio.
This quoted-overlap construction has better compatibilitythan the full-overlap construction of the previous section,but the compatibility comes at the expense of the compression ratio.There, each added file cost only a central directory header;here, it costs a central directory header,a local file header,and another 5 bytes for the quoting header.
It pays to compress the kernel as densely as possible,because every decompressed byte gets magnified by a factor of N.To that end, we use a custom DEFLATE compressorcalled bulk_deflate,specialized for compressinga string of repeated bytes.
All decent DEFLATE compressors will approach a compression ratio of 1032when given an infinite stream of repeating bytes,but we care more about specific finite sizesthan asymptotics.bulk_deflate compresses more datainto the same space than the general-purpose compressors:about 26 kB more than zlib and Info-ZIP,and about 15 kB more thanZopfli,a compressor that trades speed for density.
For our purposes, filenames are mostly dead weight.While filenames do contribute something to the output sizeby virtue of being part of quoted local file headers,a byte in a filename does not contribute nearly as muchas a byte in the kernel.We want filenames to be as short as possible,while keeping them all distinct,and subject to compatibility considerations.
The first compatibility consideration is character encoding.The zip format specification states that filenamesare to be interpreted as CP 437,or UTF-8 if a certain flag bit is set(APPNOTE.TXT Appendix D).But this is a major point of incompatibilityacross zip parsers,which may interpret filenames as being insome fixed or locale-specific encoding.So for compatibility, we must limit ourselves to charactersthat have the same encoding in bothCP 437 and UTF-8;namely, the 95 printable characters of US-ASCII.
We are further restricted by filesystem naming limitations.Some filesystems are case-insensitive, so "a" and "A" do not count as distinct names.Common filesystems like FAT32prohibit certain characterslike '*' and '?'.
As a safe but not necessarily optimal compromise,our zip bomb will use filenames consisting of charactersdrawn from a 36-character alphabetthat does notrely on case distinctionsor use special characters:
Given that the N filenames in the zip fileare generally not all of the same length,which way should we order them,shortest to longest or longest to shortest?A little reflection shows that it is better toput the longest names last, because those names are the most quoted.Ordering filenames longest lastadds over 900 MB of outputto zblg.zip,compared to ordering them longest first.It is a minor optimization, though,as those 900 MBcomprise only 0.0003%of the total output size.
The quoted-overlap constructionallows us to place a compressed kernel of data,and then cheaply copy it many times.For a given zip file size X,how much space should we devote to storing the kernel,and how much to making copies?
To find the optimum balance,we only have to optimize the single variable N,the number of files in the zip file.Every value of N requiresa certain amount of overhead forcentral directory headers,local file headers,quoting block headers, and filenames.All the remaining space can be taken up by the kernel.Because N has to be an integer,and you can only fit so many filesbefore the kernel size drops to zero,it suffices to test every possible value of Nand select the one that yields the most output.
It's a little more complicated,because the precise limits depend on the implementation.Python zipfileignoresthe number of files.Go archive/zipallowslarger file counts, as long as they are equal in the lower 16 bits.But for broad compatibility,we have to stick to the limits as stated.
Furthermore, crc32_update_1 is justcrc32_update_0 plus (XOR) a constant.That makes crc32_update_1 anaffine transformation:a matrix multiplication followed by a translation (i.e., vector addition).We can represent both the matrix multiplication and the translationin a single stepif we enlarge the dimensions of the transformation matrix to 3333and append an extra element to the state vector that is always 1.(This representation is calledhomogeneous coordinates.)
Both operations crc32_update_0 and crc32_update_1can be represented by a 3333 transformation matrix.The matrices M0 and M1 are shown.The benefit of a matrix representation is that matrices compose.Suppose we want to represent the state change effected by processingthe ASCII character 'a', whose binary representation is011000012.We can represent the cumulative CRC-32 state change of those 8 bitsin a single transformation matrix: 041b061a72