Skip to content

Fix unconverted \textit{} macros in BNF grammar, math, and code blocks #83

@lefticus

Description

@lefticus

Problem

Unconverted \textit{} macros were appearing in the generated markdown output:

  • n4950: 42 instances before fix
  • trunk: 45 instances before fix

Root Causes

  1. BNF Grammar (75% of issues): \fmtnontermdef{} macro is preprocessed to \textit{} by simplified_macros.tex, but cpp-grammar.lua wasn't stripping it
  2. Math Mode (20% of issues): Complex math expressions that can't be fully converted revert to LaTeX but weren't cleaning up text macros
  3. Code Blocks (4% of issues): Nested \textit{\textrm{}} patterns in code comments weren't fully processed

Solution

Changes Made

  1. cpp-grammar.lua: Added \textit{} stripping after BNF content processing using process_macro_with_replacement()
  2. cpp-math.lua: Added strip_text_macros_in_math() call before reverting complex math to LaTeX
  3. cpp-code-blocks.lua: Added \textrm{} detection and processing for nested macro patterns

Tests Added

  1. test_fmtnontermdef_no_textit() - Verifies BNF nonterminal definitions don't leave \textit{}
  2. test_textit_in_simple_math() - Verifies math mode strips text macros
  3. test_nested_textit_textrm_in_code() - Verifies deeply nested text macros are stripped from code

Results

  • n4950: Reduced from 42 to 9 instances (79% reduction)
  • trunk: Reduced from 45 to 14 instances (69% reduction)
  • All 646 tests passing

Remaining Edge Cases (9 instances in n4950)

  1. 1 code block: Triple-nested \textit{\textrm{C++{}}} with extra braces
  2. 8 math expressions: Complex patterns like $\texttt{\textit{map_}...}$ intentionally preserved for MathJax

These edge cases would require more complex handling and are acceptable given the 79% overall reduction.

Testing

./setup-and-build.sh  # Runs all tests and regenerates output

Files Changed

  • src/cpp_std_converter/filters/cpp-grammar.lua
  • src/cpp_std_converter/filters/cpp-math.lua
  • src/cpp_std_converter/filters/cpp-code-blocks.lua
  • tests/test_filters/test_grammar.py
  • tests/test_filters/test_math.py
  • tests/test_filters/test_code_blocks_fixes.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions