Today I've been looking into fixing undo for the text editing (for 2.8x release) and I've found to correctly fix this would require some very strange logic.
- T67045: Crash undo/redo in the text editor
- T66695: Text editor becomes out of sync in object mode
- T65909: Python API calling: bpy.ops.text.cut +bpy.ops.text.paste will fail when undo.
The root of the issue if what all undo steps that use an ID's need to apply on top of the last memfile step (since ID data blocks can be added and removed, we can never step into an operation on an ID data-block which doesn't exist).
This isn't a problem for undo systems that store their state on each step, however the text editor accumulates changes.
Using undo which works this way doesn't work well when mixing it with memfile-undo.
A fix for the current system could be to either make an exception where text is allowed to use the memfile-undo state from the future
Or loading the last memfile-undo state, then replaying all steps up until the current state, requiring is to keep all text editor undo steps after a memfile-step, also replaying the steps could become slow.
While both could be made to work they're going to introduce some quite awkward logic into the undo system just to support text editing undo.
This patch simply stores the entire text buffer for each state, de-duplicating data via BLI_array_store.
I was concerned performance could be an issue, however testing with a 40,000 line file (1.88mb), editing isn't noticeably slow. At 200,000 lines (9.5mb), I could notice a difference although it's still usable.
Note that the initial patch freed all lines and recreated them, it's since been been updated to use existing lines when available which improves performance in common cases such as entering & removing single characters.
Room for improvement
This patch could be improved in a few ways.
- Currently a single BArrayStore is used for all undo steps, it would be better that each Text gets it's own.
- Simple, easy to maintain undo logic.
Current undo logic can fail at times, testing 2.79b I managed to get it to crash 3 times in a row (performing many random edits, undoing & redoing ~ thrashing).
Not a great argument, since these bugs could be investigated, reported & fixed, nevertheless - after years of having a text editor, we didn't manage to make undo work all that reliably.
- Extensions/Python operators.
Currently any Python text operations don't get properly handled by undo, with this patch we wouldn't need to undo system to be aware of different text manipulation actions.
- Performance (in practice I don't think it's so bad, nevertheless, it is worse in some cases).
- Undo isn't aware of the location of changes. Knowing this information could be useful in the future, although currently we don't make use of it, noting it as a 'con'.
Since Blender isn't primarily a text editor, and this patch works acceptably, I'm proposing to use simpler logic to avoid having to maintain the current system.