Add LineCache to optimize newlines lookup

[L1quidH2O]

Scrolling is incredibly slow with large files because newlines aren't cached.
This PR introduces a simplified newline cache that reduces the need for repeated large scans during scrolling.

• Added LineCache struct that stores (offset, line) pairs every 64*1024 lines.
• Very simple, doesn't build a cache unless a document is opened.
• Changes in document are tracked.
• Implemented nearest_offset() to retrieve the closest cached line.

Ideally in the future, newline tracking would be built into a rope tree. But since this project uses a gap buffer, this is a fast enough shortcut.

Slow Demo:

slow.mp4

Fast Demo:

fast.mp4

[L1quidH2O]

@microsoft-github-policy-service agree

[lhecker]

I explicitly decided against the inclusion of a line cache because forgetting to invalidate caches can be moderately dangerous (if you forget it / do it incorrectly) and can be slow in edge cases if it's not written well. The editor was not meant to be used for very large files as specialized editors are better suited for this IMO. We should focus on providing an excellent editor for regular files first.

We do need a line cache eventually for syntax highlighting though to cache the highlighter state very 1000 lines or so. So, despite everything I said, long-term this PR is very welcome.

But I'm somewhat skeptical of this PR in the current state for two reasons:

• For syntax highlighting we need to store a snapshot every ~1000 lines or so, because the regex matching is very expensive (relatively speaking; runs at <100MB/s)
• Having 2 line caches for both syntax highlighting and also scrolling may be prohibitive (binary size)
• ...but if we store snapshots every 1000 lines, storing snapshots for a 140M line file would result in a 140k item array

So, I wonder if we should not repurpose this PR for a syntax highlighter.

[lhecker]

Instead of storing indices/lines, ideally, we would store index/line deltas between items. During updates, we then only need to update the items that are affected by the change, but not all items past it.

[lhecker]

This is O(n^2) worst case if the user deletes the contents between the start of the file and the middle of it, if I'm reading this right. It would be better for us to first find the offset of the range.start item, then find the offset of the range.end item, and then delete all items in that range in a single .drain() call.

[lhecker]

This also needs to insert new items, basically in reverse to what delete did.

[lhecker]

I believe this should ideally be a binary search, right?
If we use deltas as I proposed above, we could note the last deletion/insertion/nearest_offset position and its absolute index and cache it in a member. Then we can start iterating from there. Modifications and searches should occur near each other, after all.

[lhecker]

I forgot to mention, I had a solution to this performance issue in mind and if you're interested in SIMD you could implement it. (I was planning to do that myself at some point.) I'm quite certain that it would solve this performance issue as well, even without a line cache:

edit/src/unicode/measurement.rs

Lines 532 to 538 in a82483d

	// TODO: This code could be optimized by replacing memchr with manual line counting.
	//
	// If `line_stop` is very far away, we could accumulate newline counts horizontally
	// in a AVX2 register (= 32 u8 slots). Then, every 256 bytes we compute the horizontal
	// sum via `_mm256_sad_epu8` yielding us the newline count in the last block.
	//
	// We could also just use `_mm256_sad_epu8` on each fetch as-is.

Since I wrote that, I realized that we could also just always use HADD unconditionally.

Edit: I started working on this because it made me very curious. 😅

[lhecker]

I finished implementing a first prototype: https://github.com/microsoft/edit/tree/dev/lhecker/optimized-line-seeking
It's roughly 20x faster for me, which is neat, regardless of other design choices. 🙂 I'm quite certain it can be further optimized.

FYI you can check the latency with the debug-latency feature. E.g.:

cargo build --config .cargo/release.toml --release --features debug-latency

[L1quidH2O]

Wow the SIMD runs just as fast! that's seriously impressive work
Repurposing could work to keep it simple, i don't think the performance wins of other approaches would justify their added complexity and potential binary size

[lhecker]

Since we now have the SIMD performance improvements (they're now another 5x faster than what you tested!), I've updated your PR to not compile the line cache. We will still have great use for it in the future, however, for syntax highlighting. Thank you for your work!