fix: add file size validation to prevent memory exhaustion

[roomote]

Summary

This PR implements file size validation across all file reading operations in Roo-Code to prevent memory exhaustion and performance issues, as identified in #6155.

Changes

1. Created shared file size constants (src/services/constants/file-limits.ts)

• MAX_FILE_SIZE_BYTES: 1MB for general files
• MAX_CONFIG_FILE_SIZE_BYTES: 100KB for configuration files
• MAX_GITIGNORE_FILE_SIZE_BYTES: 50KB for .gitignore files
• MAX_CHECKPOINT_FILE_SIZE_BYTES: 5MB for checkpoint files
• Added formatBytes() helper and FileSizeLimitError class

2. Added file size validation to:

• roo-config service: readFileIfExists() now checks file size before reading
• glob/list-files service: .gitignore file reading now validates size
• ShadowCheckpointService: checkpoint file reading validates size
• McpHub: all config file reading operations (10+ locations) validate size

3. Added comprehensive unit tests

• Updated existing tests to mock fs.stat() calls
• Added new tests for file size validation scenarios
• Tests verify files exceeding limits are rejected with appropriate warnings

Testing

• ✅ All unit tests pass for modified services
• ✅ Linting passes
• ✅ Type checking passes
• ✅ Files at size limits are accepted
• ✅ Files exceeding limits are rejected with console warnings

Notes

• When files exceed size limits, operations return null or skip the file rather than throwing errors, maintaining backward compatibility
• Different file types have appropriate size limits based on their typical usage
• The implementation follows the existing pattern from code indexing service which already had file size validation

Fixes #6155

---

Important

Add file size validation to prevent memory exhaustion by setting limits for different file types and updating services to enforce these limits.

• Behavior:
  • Implement file size validation across file reading operations to prevent memory exhaustion.
  • Return null or skip files exceeding size limits to maintain backward compatibility.
• Constants:
  • Add MAX_FILE_SIZE_BYTES, MAX_CONFIG_FILE_SIZE_BYTES, MAX_GITIGNORE_FILE_SIZE_BYTES, MAX_CHECKPOINT_FILE_SIZE_BYTES in file-limits.ts.
  • Introduce FileSizeLimitError class for handling size violations.
• Services:
  • Update readFileIfExists() in roo-config to check file size.
  • Add size validation in list-files for .gitignore files.
  • Validate checkpoint file size in ShadowCheckpointService.
  • Validate config file size in McpHub.
• Testing:
  • Add unit tests for file size validation in list-files-gitignore-size.spec.ts and index.spec.ts.
  • Mock fs.stat() in tests to simulate file size scenarios.

^{This description was created by for 40a52ae. You can customize this summary. It will automatically update as commits are pushed.}

[daniel-lxs]

Closing in favor of #6174