Add --max-syntax-highlighting-length, set to 400

[th1000s]

• Add --max-syntax-highlighting-length, set to 400

  --max-line-length increased to 3000, highlighting now stops after 400
  characters. In that case the highlighting may be incorrect until it
  is reset for the next hunk.

• Add truncate_str_short() which always returns a prefix of the input

  Document that the input of truncate_str() may be cut on a double width
  char ("fullwidth") - then the output is no longer a prefix of the input.

  truncate_str_short() could be faster by not constructing the result and
  instead only returning an index, but it will be used in rare edge cases so
  reusing existing logic is good enough.

  Common code moved to truncate_str_impl().

  Assume (with fallback) that graphemes are at most 2 wide.
  Fewer allocations.
  Tests.

---

See #290

Some rough benchmarking shows it is faster when truncating lines. But note that this has a good chance of ruining the highlighting of the following lines.

[dandavison]

Suggested change

	/// To always highlighting entire lines set to zero - but note that delta will be slow on very
	/// To always highlight entire lines set to zero - but note that delta will be slow on very

[dandavison]

(Ignore, I made this change and force-pushed to your last commit)

[dandavison]

Great, thanks! I tried it out a bit manually and I see (as you said somewhere else) that incorrect highlighting after a truncated line is quite common both on this branch and released main. (Is it feasible to fix that by resetting the highlighter manually in such cases?)

[th1000s]

HighlightLines itself doesn't provide a reset option, so the between-hunks resetting has to be made available in the relevant function. And it could still be wrong if an opening quote is never closed. Other options would be to tolerate one or two overly long lines here and there, and/or to not even start highlighting those lines. I'd wait and see for now, I think these lines are quite rare :)