Document the current behavior and add a changelog

CHANGELOG.md

@@ -0,0 +1,103 @@
+# Changelog
+
+## 0.4.0 (unreleased)
+
+The first release since 2019. It fixes two defects that could lose or expose data, so anything
+encrypted with 0.3.x is worth revisiting.
+
+### Fixed: ciphertexts could be silently wrong
+
+**`--length-checksum` corrupted plaintexts.** In randomized testing, 34 of 120 round-trips returned
+the right number of bytes with the wrong contents — and because the length header truncates the
+output, the damage was invisible. It also affected equal-length plaintexts whenever an odd number of
+single-nibble fallbacks left a nibble pending.
+
+The cause: a nibble-gram reader could report "no data" both at end of stream and when it merely held
+fewer nibbles than requested. The encrypter treated both as "pad", writing padding over live
+plaintext and emitting a block without consuming anything, which shifted everything after it.
+
+**If you have ciphertexts written with `--length-checksum`, verify they decrypt to what you
+expect.** `--dictionary` (the default) and `--same-length` were not affected.
+
+### Fixed: ciphertexts leaked their own filename
+
+The gzip envelope recorded the output path in cleartext, because gzip derives the header FNAME field
+from the file object it is given:
+
+```
+$ lenticrypt -o my-secret-plans.enc -e key1 p1 -e key2 p2
+$ dd if=my-secret-plans.enc bs=1 skip=10 count=19
+my-secret-plans.enc
+```
+
+For a tool whose purpose is plausible deniability, that is recoverable metadata in the first
+30 bytes. New ciphertexts carry no filename and no timestamp.
+
+### Fixed: other correctness defects
+
+- **The default encryption mode hung forever** when the keys lacked entropy for some byte — exactly
+  the case `-f/--force-encrypt` documents itself as handling.
+- **`-v/--version` printed nothing at all**, and on Python 3.14 the same bug broke the test suite:
+  `main()` closed stdout, discarding output still buffered above it.
+- **`-5/--best` was silently identical to `-4`**, so 16-nibble-grams were unreachable.
+- **`-t` reported every byte combination as missing** on failure, and printed them as control
+  characters.
+- **`--seed` did not actually produce reproducible output.** Dictionary indices depended on set
+  iteration order, which varies with `PYTHONHASHSEED`.
+- **An empty plaintext decrypted to one byte**, and a ciphertext declaring more bytes than it
+  delivered returned a truncated plaintext silently. Both are now correct or refused.
+- **`IOWrapper` indexed the file path instead of the file**, because `str` is itself a `Sequence`.
+  `IOWrapper('-')` also closed `sys.stdin`.
+- Malformed ciphertexts raised `struct.error`, `TypeError`, or `EOFError`. They now raise
+  `MalformedCiphertextError`, and the CLI reports it as a message rather than a traceback.
+- With color disabled, every log level printed bare, so redirecting stderr discarded the fact that
+  a message was an ERROR.
+- `ProgressBar` divided by zero for short keys, and drew against the first phase's scale forever.
+
+### Changed
+
+- **Certificate offsets are chosen with a CSPRNG** rather than Mersenne Twister, whose state is
+  recoverable from enough output. Those offsets are the ciphertext. `--seed` still gives a
+  reproducible generator, and now warns that reproducible means predictable.
+- **Unknown ciphertext format versions are refused** instead of decoded as an older version, which
+  produced garbage.
+- **Nibble-gram lengths are chosen adaptively** from key size and secret count; `-1`..`-5` cap that
+  choice rather than making it. Lengths that cannot be hit are no longer indexed:
+
+  ```
+  2 x 512 KiB keys   2.85s / 1109 MB  ->  0.24s / 62 MB
+  2 x   2 MiB keys  12.40s / 4334 MB  ->  0.95s / 145 MB
+  ```
+
+  Ciphertext sizes are unchanged; the dropped lengths were never used.
+- **Encryption and decryption stream**, so memory no longer scales with file size.
+- Non-gzipped ciphertexts now decrypt, and `-d` accepts `-` for standard input.
+
+### Removed and renamed
+
+- `lenticrypt.lenticrypt` is now `lenticrypt.core`. The package re-exports the public names, so
+  `from lenticrypt import ...` is unaffected; `from lenticrypt.lenticrypt import ...` is not.
+- The package previously leaked `array`, `itertools`, `random`, `struct` and every imported `typing`
+  name into its namespace via a bare star import. It now exports a deliberate `__all__`.
+- `Encrypter`'s hooks changed shape: `process_nibble` and `process_nibbles` are replaced by
+  `pad_nibble_gram`, `can_encode` and `encode_block`; `get_max_length` by `total_nibbles`;
+  `get_tuple` and `are_valid_nibbles` are gone. Subclasses outside this package will need updating —
+  deliberately breaking rather than silently changing behavior.
+- `decrypt`'s `cert=` and `file_length=` parameters are no longer positional; they existed for an
+  internal recursion that no longer happens.
+- `AutoUnzippingStream` and `GzipIOWrapper` are replaced by `auto_unzip`.
+- The version number no longer encodes the ciphertext format version as its minor component.
+  `ENCRYPTION_VERSION` is a separate constant, still 3.
+
+### Project
+
+- Python 3.10 through 3.14, tested on Linux and Windows. The package previously could not be
+  imported on Windows at all.
+- Packaging moved from `setup.py` to `pyproject.toml` with hatchling; `py.typed` is shipped and the
+  package is fully annotated.
+- The test suite went from 7 non-deterministic tests to 256, including fixtures that pin the
+  on-disk format so old ciphertexts keep decrypting.
+
+## 0.3.1 (2019-02-15)
+
+Python 3 port and color logging.

README.md

@@ -9,16 +9,18 @@ Unlike alternative plausibly deniable cryptosystems like discontinued [TrueCrypt
 
 In fact, Lenticrypt has the theoretical property that, under reasonable assumptions, there is always a near 100% probability that there exists a key in the public domain that will decrypt a given ciphertext to _any_ desired plaintext, even if that key is not known.  Therefore, even if an incriminating plaintext is revealed, the author of the ciphertext can plausibly deny having created it because there is a non-zero probability that the plaintext was legitimately decrypted by random chance. Creating the legal precedent for this theoretical property is left as an exercise for the reader.
 
-Lenticrypt _can_ provide secrecy, but it does not _guarantee_ it. _**Do not**_ rely on Lenticrypt alone if you care about the secrecy of your plaintexts! 
+Lenticrypt _can_ provide secrecy, but it does not _guarantee_ it. _**Do not**_ rely on Lenticrypt alone if you care about the secrecy of your plaintexts!
 
 More technical details on the cryptosystem as well as additional use-cases are described in [Issue 0x04](https://www.sultanik.com/pocorgtfo/#0x04) of [The International Journal of PoC||GTFO](https://www.sultanik.com/pocorgtfo/).
 
 ## Installation
 
 ```shell
-$ pip3 install lenticrypt
+$ uv tool install lenticrypt      # or: pip install lenticrypt
 ```
 
+Requires Python 3.10 or newer, and has no runtime dependencies.
+
 ## Usage
 
 ```shell
@@ -31,8 +33,66 @@ $ lenticrypt -d key2 output.enc | diff - plaintext2 -s
 Files - and plaintext2 are identical
 ```
 
+Both gzipped and plain ciphertexts are accepted on decryption, and `-` reads from standard input.
 Additional instructions are available by running with the `-h` option.
 
+### Choosing keys
+
+The keys must jointly contain every byte combination the plaintexts use, or those bytes cannot be
+encoded. `-t` checks a set of keys before you rely on them, and exits non-zero if they fall short:
+
+```shell
+$ lenticrypt -t key1 key2
+This set of secrets looks good!
+```
+
+Large, high-entropy files make good keys. `-f` forces encryption with insufficient keys, but the
+ciphertext will not decrypt back to the original plaintexts.
+
+### Reproducible output
+
+`--seed` makes the ciphertext reproducible, which also makes it predictable. It exists for testing;
+do not use it for anything you need kept secret. Without it, certificate offsets are drawn from the
+operating system's CSPRNG.
+
+## Performance
+
+Encryption indexes the keys first, and that index dominates both time and memory. The nibble-gram
+lengths worth indexing are chosen from the key size and the number of secrets, because a length-L
+match needs every key to agree at the same offset — so the longer lengths are unreachable unless
+the keys are enormous. The `-1` through `-5` options cap that choice rather than making it.
+
+For two 512 KiB keys the index takes about 0.25s and 62 MB. Encryption itself then runs at roughly
+6 seconds per MiB of plaintext, which is the dominant cost for anything sizeable.
+
+Ciphertexts are larger than the plaintext, by a factor that improves with size as the dictionary
+header is amortized — measured with two secrets, at 5.5x for 512 B, 4.4x at 4 KiB, 3.5x at 64 KiB,
+and 2.7x at 512 KiB. Size tracks the *largest* plaintext, not the number of them.
+
+Both encryption and decryption stream, so memory does not scale with file size.
+
+## Development
+
+```shell
+$ uv sync
+$ uv run pytest -q
+$ uv run ruff check && uv run ruff format --check
+$ uv run ty check
+```
+
+If your environment sets `exclude-newer` in a uv config file, regenerate the lockfile with
+`uv --no-config lock`; a plain `uv lock` bakes that policy into `uv.lock`.
+
+## File format
+
+Ciphertexts are gzip-wrapped. Three format versions exist and all three still decrypt:
+
+| version | option | notes |
+|---|---|---|
+| 1 | `--same-length` | No length header, so plaintexts of unequal length are truncated or zero-padded |
+| 2 | `--length-checksum` | Encrypted length header, so unequal lengths round-trip |
+| 3 | `--dictionary` | Adds an index dictionary. The default. Shrinks ciphertexts above roughly 8 KiB of plaintext; below that its header costs more than it saves |
+
 ## Author
 
 Evan A. Sultanik, Ph.D.<br />

lenticrypt/core.py

@@ -407,7 +407,7 @@ def consume_grams(
         """Walks the plaintexts in lockstep, yielding each `(length, grams)` that can be encoded.
 
         This is the only place nibbles are consumed, so it is structurally impossible to emit a
-        block for nibbles that were not consumed -- the desynchronisation that corrupted output.
+        block for nibbles that were not consumed -- the desynchronization that corrupted output.
         Longest gram lengths are tried first; a length that cannot be represented falls back to a
         shorter one, and at length 1 the nibbles are consumed with a warning so that the walk always
         makes progress.
@@ -451,7 +451,7 @@ def consume_grams(
                     )
                     raise LenticryptError(message)
                 break
-        # Only summarise when there was more than the individually reported grams to report.
+        # Only summarize when there was more than the individually reported grams to report.
         if self._unencodable_count > len(self._unencodable):
             logger.warning(
                 f"{self._unencodable_count} nibble-gram(s) in total could not be encoded during "
@@ -695,7 +695,7 @@ def get_header(self) -> Iterator[bytes]:
 
 
 class _NibbleAssembler:
-    """Reassembles a plaintext from nibbles, honouring the declared length.
+    """Reassembles a plaintext from nibbles, honoring the declared length.
 
     A single place that owns the half-byte carry and the `file_length` bound. There were four
     near-identical copies of this carry logic, and the one in `_decrypt_dictionary` checked the
@@ -926,6 +926,6 @@ def decrypt(
             # A damaged compression envelope is still a damaged ciphertext, and should read as one
             # rather than as an `EOFError` traceback from deep inside gzip. Caught narrowly on
             # purpose: a generic OSError here could be a real disk failure, which must not be
-            # relabelled as malformed input.
+            # relabeled as malformed input.
             message = f"The ciphertext is not readable: {error}"
             raise MalformedCiphertextError(message) from error

lenticrypt/logger.py

@@ -15,7 +15,7 @@
 
 DEFAULT_FORMAT = "$RESET$LEVELCOLOR$BOLD%(levelname)-8s$RESET %(message)s"
 
-# Placeholder substituted with the colour of the record's own level, rather than a fixed colour.
+# Placeholder substituted with the color of the record's own level, rather than a fixed color.
 LEVEL_COLOR_PLACEHOLDER = "$LEVELCOLOR"
 
 
@@ -57,14 +57,14 @@ def __init__(
 
     @staticmethod
     def expand(fmt: str) -> str:
-        """Replaces colour placeholders with ANSI escapes, leaving `$LEVELCOLOR` for `format`."""
+        """Replaces color placeholders with ANSI escapes, leaving `$LEVELCOLOR` for `format`."""
         for color in CGAColors:
             fmt = fmt.replace(f"${color.name}", ansi_color(color))
         return fmt.replace("$RESET", ANSI_RESET).replace("$BOLD", ANSI_BOLD)
 
     @staticmethod
     def strip(fmt: str) -> str:
-        """Removes every colour placeholder, for non-tty output."""
+        """Removes every color placeholder, for non-tty output."""
         for color in CGAColors:
             fmt = fmt.replace(f"${color.name}", "")
         for placeholder in ("$RESET", "$BOLD", LEVEL_COLOR_PLACEHOLDER):
@@ -76,7 +76,7 @@ def format(self, record: logging.LogRecord) -> str:
         if record.levelno == logging.INFO:
             return record.getMessage()
         if not self._use_color:
-            # Colour off still means *labelled*. Previously this returned the bare message for every
+            # Color off still means *labeled*. Previously this returned the bare message for every
             # level, so redirecting stderr to a file silently discarded the fact that a message was
             # a WARNING or an ERROR. The placeholders are already stripped from the format string.
             return super().format(record)

lenticrypt/progress.py

@@ -75,13 +75,13 @@ def _render(self, percent: float, rounded: int, status: str | None) -> str:
         start = max((width - len(label)) // 2, 0)
         # Spaces inside the label that fall within the filled region become bar characters, so the
         # label stays legible without punching a hole in the bar.
-        labelled = "".join(
+        labeled = "".join(
             "=" if character == " " and start + offset < filled else character
             for offset, character in enumerate(label)
         )
         left = "".join("=" if i < filled else "-" for i in range(start))
         right = "".join("=" if i < filled else "-" for i in range(start + len(label), width))
-        return f"[{left}{labelled}{right}]"
+        return f"[{left}{labeled}{right}]"
 
 
 class ProgressBarCallback:

tests/test_logger.py

@@ -1,4 +1,4 @@
-"""ColorFormatter output, with and without colour.
+"""ColorFormatter output, with and without color.
 
 This module had no tests, so the placeholder expansion and the INFO special case were unverified.
 """
@@ -24,7 +24,7 @@ def test_info_is_printed_bare():
 
 
 @pytest.mark.parametrize("level", [logging.WARNING, logging.ERROR, logging.CRITICAL, logging.DEBUG])
-def test_other_levels_are_labelled_and_coloured(level):
+def test_other_levels_are_labeled_and_colored(level):
     formatter = ColorFormatter(DEFAULT_FORMAT, use_color=True)
     output = formatter.format(record(level))
     assert logging.getLevelName(level) in output
@@ -35,7 +35,7 @@ def test_other_levels_are_labelled_and_coloured(level):
 @pytest.mark.parametrize(
     "level", [logging.WARNING, logging.ERROR, logging.CRITICAL, logging.DEBUG, logging.INFO]
 )
-def test_no_escapes_when_colour_is_disabled(level):
+def test_no_escapes_when_color_is_disabled(level):
     """Redirected output must stay free of escape sequences."""
     formatter = ColorFormatter(DEFAULT_FORMAT, use_color=False)
     output = formatter.format(record(level))
@@ -45,20 +45,20 @@ def test_no_escapes_when_colour_is_disabled(level):
 
 
 @pytest.mark.parametrize("level", [logging.WARNING, logging.ERROR, logging.CRITICAL])
-def test_level_is_still_named_when_colour_is_disabled(level):
+def test_level_is_still_named_when_color_is_disabled(level):
     """Redirecting stderr used to discard the level entirely, so an ERROR read as ordinary text."""
     formatter = ColorFormatter(DEFAULT_FORMAT, use_color=False)
     assert logging.getLevelName(level) in formatter.format(record(level))
 
 
-def test_info_stays_bare_without_colour():
-    """INFO is conversational at any colour setting."""
+def test_info_stays_bare_without_color():
+    """INFO is conversational at any color setting."""
     formatter = ColorFormatter(DEFAULT_FORMAT, use_color=False)
     assert formatter.format(record(logging.INFO)) == "hello"
 
 
-def test_level_colour_placeholder_is_resolved_per_record():
-    """`$LEVELCOLOR` must become the colour of *this* record's level, not a fixed one."""
+def test_level_color_placeholder_is_resolved_per_record():
+    """`$LEVELCOLOR` must become the color of *this* record's level, not a fixed one."""
     formatter = ColorFormatter(DEFAULT_FORMAT, use_color=True)
     warning = formatter.format(record(logging.WARNING))
     error = formatter.format(record(logging.ERROR))
@@ -76,7 +76,7 @@ def test_multiline_messages_get_continuation_markers():
     assert output.count("\n") >= 2
 
 
-def test_named_colour_placeholders_expand():
+def test_named_color_placeholders_expand():
     assert ansi_color(CGAColors.BLUE) in ColorFormatter.expand("$BLUE")
     assert ColorFormatter.expand("$RESET") == ANSI_RESET