Keep prev spaces

[tompng]

Re-imlement prev_space feature #605 (temporarily dropped in #500)

Code pasted to irb sometimes have base indent. This pull request make IRB indent correctly with base indent

# no base indent
irb(main):001:1* if 1
irb(main):002:1*   2
irb(main):003:0> end

# base indent = 4
irb(main):001:1*     if 1
irb(main):002:1*       2
irb(main):003:0>     end

Example of base indent and heredoc

irb(main):001:1*         if 1
irb(main):002:0"           <<A
irb(main):003:0" hello
irb(main):004:1* A
irb(main):005:2"           <<~A
irb(main):006:2"             world
irb(main):007:1*           A
irb(main):008:0>         end

TODO: wait for #515, #608, write tests for heredoc done

Basic algorithm

I defined base_indent to satisfy actual_indent == base_indent + 2 * indent_level. base_indent can differ on line.

Pasted code

      if 1 # base_indent is 6, indent_level is 0
          if 2 # base_indent is 8, indent_level is 1
            3
            4.tap do # base_indent is 8, indent_level is 2, indent is 12 spaces
              5

Reference code

if 1
  if 2
    3
    4.tap do # indent is 4 spaces (= 2 * indent_level)
      5

Line 5 is just inside token do on line 4. We need to calculate base_indent of line 4 where do is located.
base_indent of line 4 can be calculated by 12 - 2 * indent_level.
12 is the indent of pasted code on line 4 and indent_level is 2

Exceptional case

      if 1
          if 2
            "helloworld
helloworld".tap do
              5

If the beginning of line 4 is inside string literal, we cannot calculate base_indent of line 4.
In that case, base_indent of line 3 is used instead.

Exceptional case 2

      :"ruby
ruby
ruby" + "helloworld
helloworld".tap do
        5

Cannot calculate line 4, fallback to line 3 (beginning of tstring_beg)
Cannot calculate line 3, fallback to line 1 (beginning of symbeg)
base_indent of line 1 is 6

[tompng]

I forgot to add :on_backtick and the test test_pasted_code_keep_base_indent_spaces failed

          ]+[["a
b" + `c
d` + /e ← failed here
f/ + :"g
h".tap do

[st0012]

Does this imply the result of indent_difference(lines, line_results, prev_open_token.pos[0] - 1) could be negative?

[tompng]

Yes. It could be negative.

if 1 # diff = 0
  if 2 # diff = 0
  end
end

    if 1 # diff = 4
      if 2 # diff = 4
      end
    end

    if 1 # diff = 4
    if 2 # diff = 2
    end
    end

if 1 # diff = 0
if 2 # diff = -2 ← negative
end
end