Refactor: Integrate Citation Processing into Filesystem Observer

Summary

Integrated citation processing directly into the main FileSystemObserver within the tidyverse/observers submodule. This removes the need for separate citation processing scripts and unifies file handling logic.

Why Care

This refactoring simplifies the content processing workflow by handling both frontmatter validation/updates and citation processing within a single, coordinated system. It improves consistency, reduces potential conflicts between operations, and makes the observer system more robust and maintainable. It also centralizes citation logic using a CitationService and ensures interaction with the citation-registry.json.

Implementation

The core changes involved modifying the FileSystemObserver to incorporate citation handling logic previously specified in content/lost-in-public/prompts/data-integrity/Refactored-Citations-Observer.md.
Key implementation steps included:
  1. Unified Processing: Integrated citation detection (numeric/hex), conversion (numeric to hex), and registry updates directly into the observer's file processing sequence (add/change events).
  2. Centralized Logic: Introduced a CitationService singleton to manage the citation registry (site/src/content/citations/citation-registry.json), ensuring consistent loading, checking, updating, and persistence.
  3. Hex ID Consistency: Implemented logic to maintain a consistent mapping between numeric citation IDs (e.g., [123]) and their generated hex IDs (e.g., [^abcdef]) during processing.
  4. Footnote Handling: Ensured footnote definitions ([^abcdef]: Footnote text) are correctly added or updated in the processed markdown files.
  5. Conflict Resolution: Addressed potential race conditions or conflicts between frontmatter updates and citation processing to prevent file corruption or duplicate frontmatter sections.
  6. Dependency Refinement: Continued refactoring efforts to minimize reliance on external YAML parsing libraries where possible, using manual construction for better control over output formatting (related to commit c503704).

Files Modified (Likely Candidates)

  • tidyverse/observers/src/FileSystemObserver.ts
  • tidyverse/observers/src/services/CitationService.ts (Created/Modified)
  • tidyverse/observers/src/types/ (Updated interfaces/types)
  • tidyverse/observers/src/utils/ (Potentially added citation-specific utilities)
  • tidyverse/observers/index.ts
  • tidyverse/observers/package.json (If dependencies changed)

Future Work

  • Further optimize citation processing performance for large numbers of files.
  • Enhance reporting to include detailed citation processing statistics.