Skip to content

core: classify one-cell characters outside the known ranges as :other - #2399

Open
theangelperalta wants to merge 1 commit into
lem-project:sdl3-nextfrom
theangelperalta:fix/char-type-other
Open

theangelperalta wants to merge 1 commit into
lem-project:sdl3-nextfrom
theangelperalta:fix/char-type-other

Conversation

@theangelperalta

Copy link
Copy Markdown
Collaborator

Targets sdl3-next, the integration branch for SDL3 and text-rendering work. main, and so the nightly releases, get it when sdl3-next is merged in.

Problem

char-type returns :cjk for every character outside its known ranges, so Greek, Cyrillic, arrows, math symbols and box drawing are all "CJK". The SDL2 and SDL3 frontends draw :cjk with the CJK font, which draws many of these full-width in a one-cell slot. To work around that, both frontends re-check the character's width themselves (#2376, #2395, #2396).

Change

  • char-type returns :cjk only for wide characters (wide-char-p), and the new type :other for the rest.
  • char-type is exported from lem-core/display, with docstrings for the type and the function, so frontends and tests no longer need lem-core::char-type.
  • SDL3 draws :other with the latin fonts, which fall back to the CJK font and the others. narrow-non-cjk-char-p is removed.
  • SDL2 draws :other with the latin font when it has the glyph, and with the CJK font otherwise, as before.

ncurses, webview and the server don't use the type, so they're unaffected. A frontend that sends unknown types to its default font gets the old behaviour.

Testing

  • New tests/char-type.lisp covers Latin, wide (:cjk), one-cell (:other) and the unchanged types, and checks that every result is a char-type.
  • The SDL font tests now take the type from char-type instead of hard-coding :cjk.
  • lem-tests, lem-sdl2/tests and lem-sdl3/tests pass, and Lem, SDL2 and SDL3 compile with no warnings.
  • An SDL3 frame of Greek, Cyrillic, math, CJK and box drawing is pixel-identical to before, so nothing changes on screen.

char-type returned :cjk for every character outside its known ranges,
so Greek, Cyrillic, arrows, math symbols and box drawing were all
"CJK". The SDL2 and SDL3 frontends draw :cjk with the CJK font, which
draws many of these full-width in a one-cell slot, so both frontends
had to re-check the width themselves (lem-project#2395, lem-project#2396).

char-type now returns :cjk only for wide characters (wide-char-p) and
the new type :other for the rest. Frontends that pick fonts by type
can draw :other with their monospace font; ones that ignore the type,
or send unknown types to a default, behave as before.

- char-type is exported from lem-core/display, with docstrings for the
  type and the function, so frontends and tests no longer need
  lem-core::char-type.
- SDL3 draws :other with the latin fonts (which fall back to the CJK
  font and the others) and drops narrow-non-cjk-char-p.
- SDL2 draws :other with the latin font when it has the glyph, else
  with the CJK font, as before.

Tests: new tests/char-type.lisp covers latin, wide (:cjk), one-cell
(:other) and the unchanged types, and that every result is a
char-type. The SDL font tests now take the type from char-type.
lem-tests, lem-sdl2/tests and lem-sdl3/tests pass, and an SDL3 frame
of Greek, Cyrillic, math, CJK and box drawing is pixel-identical to
before.
@code-contractor-app

code-contractor-app Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ Code Contractor Validation: PASSED

📌 Result for commit 4e661de

=== Contract: contract ===

✓ Code Contractor Validation Result
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📋 Contract Source: Repository

📊 Statistics:
  Files Changed:    8
  Lines Added:      72
  Lines Deleted:    45
  Total Changed:    117
  Delete Ratio:     0.38 (38%)

Status: PASSED ✅

🤖 AI Providers:
  - codex — profile: bedrock-gpt61-terra

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🎉 No violations detected. Great job!
📋 Contract Configuration: contract (Source: Repository)
version: 2

trigger:
  paths:
    - "extensions/**"
    - "frontends/**/*.lisp"
    - "src/**"
    - "tests/**"
    - "contrib/**"
    - "**/*.asd"
  head_branches:
    exclude:
      - 'revert-*'

validation:
  limits:
    max_total_changed_lines: 400
    max_delete_ratio: 0.5
    max_files_changed: 10
    severity: warning

  ai:
    system_prompt: |
      You are a senior Common Lisp engineer reviewing code for Lem editor.
      Lem is a text editor with multiple frontends (ncurses, SDL2, webview).
      Focus on maintainability, consistency with existing code, and Lem-specific conventions.
    rules:
      # === File Structure ===
      - name: defpackage_rule
        prompt: |
          First form must be `defpackage` or `uiop:define-package`.
          Package name should match filename (e.g., `foo.lisp` → `:lem-ext/foo` or `:lem-foo`).
          Extensions must use `lem-` prefix (e.g., `:lem-python-mode`).

      - name: file_structure_rule
        prompt: |
          File organization (top to bottom):
          1. defpackage
          2. defvar/defparameter declarations
          3. Key bindings (define-key, define-keys)
          4. Class/struct definitions
          5. Functions and commands

          Only flag violations when elements appear OUT OF ORDER (e.g., key bindings AFTER functions, or defvar AFTER functions).
          Key bindings appearing BEFORE functions is correct and expected.

          Exception: In major-mode files (e.g., *-mode.lisp), flexible ordering is acceptable when it follows established patterns from similar modes (e.g., clojure-mode, python-mode).

      # === Style ===
      - name: loop_keywords_rule
        prompt: |
          Loop keywords must use colons: `(loop :for x :in list :do ...)`
          NOT: `(loop for x in list do ...)`

      - name: naming_conventions_rule
        prompt: |
          Naming conventions:
          - Functions/variables: kebab-case (e.g., `find-buffer`)
          - Special variables: *earmuffs* (e.g., `*global-keymap*`)
          - Constants: +plus-signs+ (e.g., `+default-tab-size+`)
          - Predicates: -p suffix for functions (e.g., `buffer-modified-p`)
          - Do NOT use -p suffix for user-configurable variables

      # === Documentation ===
      - name: docstring_rule
        prompt: |
          Required docstrings for:
          - Exported functions, methods, classes
          - `define-command` (explain what the command does)
          - Generic functions (`:documentation` option)
          Important functions should explain "why", not just "what".
        severity: warning

      # === Lem-Specific ===
      - name: internal_symbol_rule
        prompt: |
          Use exported symbols from `lem` or `lem-core` package.
          Avoid `lem::internal-symbol` access.
          If internal access is necessary, document why.

      - name: error_handling_rule
        prompt: |
          - `error`: Internal/programming errors
          - `editor-error`: User-facing errors (displayed in echo area)
          Always use `editor-error` for messages shown to users.

      - name: frontend_interface_rule
        prompt: |
          Frontend-specific code must use `lem-if:*` protocol.
          Do not call frontend implementation directly from core.
        severity: warning

      # === Functional Style ===
      - name: functional_style_rule
        prompt: |
          Prefer explicit function arguments over dynamic variables.
          Avoid using `defvar` for state passed between functions.
          Exception: Well-documented cases like `*current-buffer*`, constants, and configuration variables (not meant to be modified at runtime).

      - name: dynamic_symbol_call_rule
        prompt: |
          Avoid `uiop:symbol-call`. Rethink architecture instead.
          Exception: Acceptable in extensions when following patterns established in other major-mode extensions (e.g., for optional dependency loading or plugin systems).
          If used, document the reason.

      # === Libraries ===
      - name: alexandria_usage_rule
        prompt: |
          Alexandria utilities allowed: `if-let`, `when-let`, `with-gensyms`, etc.
          Avoid: `alexandria:curry` (use explicit lambdas)
          Avoid: `alexandria-2:*` functions not yet used in codebase

      # === Macros ===
      - name: macro_style_rule
        prompt: |
          Keep macros small. For complex logic, use `call-with-*` pattern:
          ```lisp
          (defmacro with-foo (() &body body)
            `(call-with-foo (lambda () ,@body)))
          ```
          Prefer `list` over backquote outside macros.

          Exception: In major-mode files, macro patterns that are standard in other major-modes (e.g., clojure-mode, python-mode) are acceptable.
📚 About Code Contractor

Declarative Code Standards That Learn and Improve

Define domain-specific validation rules in YAML.
Your contracts document team knowledge and evolve into more accurate AI enforcement.

Want this for your repo?
→ Install Code Contractor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant