[C++/Python] Support necessary functionality to have an Arrow-string type in pandas

[asfimport]

This should serve as an umbrella issue for the needed functionality to have an Apache Arrow backed string type in pandas. In addition to the string kernels (ARROW-555), we probably need to implement some more support functionality to efficiently the pandas interfaces.

Some of these functions are already present in fletcher but a native string type in pandas should not have a hard dependency on numba.

Reporter: Uwe Korn / @xhochy

Related issues:

• [C++] String algorithm library for StringArray/BinaryArray (depends upon)
• [C++/Python] Kernel for SetItem(IntegerArray, values) (depends upon)
• [C++/Python] Add option to Take kernel to interpret negative indices as indexing from the right (depends upon)
• [C++/Python] Add option to Take kernel to interpret negative indices as NULL (depends upon)
• [Python] Accept pd.NA as missing value in array constructor (depends upon)
• [Python] ChunkedArray.to_numpy (depends upon)
• [C++/Python] Kernel for SetItem(BooleanArray, values) (depends upon)
• [C++] Add fill_null kernel implementation for (array[string], scalar[string]) (depends upon)

_{Note: This issue was originally created as ARROW-9401. Please see the migration documentation for further details.}

[asfimport]

Josh Ring:
General question from the other thread about mutability. If we can pad each string to the UTF-8 max string length, can we better support mutability without forcing a complete reallocation of the array?

[asfimport]

Uwe Korn / @xhochy:
[~Ring]  Please stop posting the same question in multiple places, I'll answer in the first pandas issue.

[asfimport]

F. H.:
So there is now the String[pyarrow] type in pandas.

Can we use this automatically when converting a pyarrow table to pandas?

Currently I still get str back when converting pyarrow -> pandas...

[asfimport]

Apache Arrow JIRA Bot:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[thisisnic]

This issue hasn't had activity in a long time. If it's still being worked on, please leave a comment. Otherwise, it will be closed on 23rd June.

Labelled Status: Stale-Warning for tracking.