[21] generate a better optimized query for path prefix search filters

[icewind1991]

Applies the optimizations from #28476 and #28608 to the nc21 jail search logic

[juliusknorr]

@icewind1991 Is there any way to verify from the query that it is using an indexed search? The query still seems to use just the fs_mtime index.

MariaDB [nextcloud]> EXPLAIN SELECT `file`.`fileid`, `storage`, `path`, `path_hash`, `file`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`,  `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `file` LEFT JOIN `oc_filecache_extended` `fe` ON `file`.`fileid` = `fe`.`fileid`  WHERE (`storage` = 2) AND (LOWER(`name`) LIKE LOWER('%animals%') ESCAPE '\\') AND ((`path` LIKE '__groupfolders/1/%' ESCAPE '\\') OR (`path` = '__groupfolders/1')) ORDER BY `mtime` desc LIMIT 5;
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+----------+---------+-----------------------+------+-------------+
| id   | select_type | table | type   | possible_keys                                                                                       | key      | key_len | ref                   | rows | Extra       |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+----------+---------+-----------------------+------+-------------+
|    1 | SIMPLE      | file  | index  | fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,fs_storage_path_prefix | fs_mtime | 8       | NULL                  | 10   | Using where |
|    1 | SIMPLE      | fe    | eq_ref | PRIMARY                                                                                             | PRIMARY  | 8       | nextcloud.file.fileid | 1    | Using where |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+----------+---------+-----------------------+------+-------------+
2 rows in set (0.001 sec)

Might be overlooking something here, but maybe we'd need to have a combined index on storage/mtime/path for that? create index fs_storage_mtime on oc_filecache(storage, mtime, path(64));

MariaDB [nextcloud]> create index fs_storage_mtime_path_prefix on oc_filecache(storage, mtime, path(64));
Query OK, 0 rows affected (0.057 sec)
Records: 0  Duplicates: 0  Warnings: 0

MariaDB [nextcloud]> EXPLAIN SELECT `file`.`fileid`, `storage`, `path`, `path_hash`, `file`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`,  `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `file` LEFT JOIN `oc_filecache_extended` `fe` ON `file`.`fileid` = `fe`.`fileid`  WHERE (`storage` = 2) AND (LOWER(`name`) LIKE LOWER('%animals%') ESCAPE '\\') AND ((`path` LIKE '__groupfolders/1/%' ESCAPE '\\') OR (`path` = '__groupfolders/1')) ORDER BY `mtime` desc LIMIT 5;
+------+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-----------------------+------+-------------+
| id   | select_type | table | type   | possible_keys                                                                                                                    | key                          | key_len | ref                   | rows | Extra       |
+------+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-----------------------+------+-------------+
|    1 | SIMPLE      | file  | range  | fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,fs_storage_path_prefix,fs_storage_mtime_path_prefix | fs_storage_mtime_path_prefix | 8       | NULL                  | 4431 | Using where |
|    1 | SIMPLE      | fe    | eq_ref | PRIMARY                                                                                                                          | PRIMARY                      | 8       | nextcloud.file.fileid | 1    | Using where |
+------+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+-----------------------+------+-------------+
2 rows in set (0.002 sec)

[icewind1991]

Do you have the index from #28541 ?

Also those queries look like the one that get generated for sqlite, LOWER(..) LIKE LOWER(..) shouldn't use used on mysql

[juliusknorr]

Oh sorry, indeed my stable21 setup was on sqlite for some reason, will retest. Sorry for the confusion.

[PVince81]

👍

[icewind1991]

/backport to stable20

[juliusknorr]

All right, seems to do the trick for me now with the right database picked as well 🎉