Skip to content

Conversation

@longshuicy
Copy link
Member

@longshuicy longshuicy commented Jan 31, 2024

This PR features:

  1. Create additional view that combines both file and folders
  2. Add indicator "object_type" to differentiate
  3. Additional endpoint /dataset/{id}/folders_and_files to list all the folders and files; with folders listed first and sorted by created date
  4. Frontend listing, pagination logic
  5. A few clean ups

To Test:

  1. Try create nested folder structures
  2. Upload files within each level see if it works
  3. Try delete folder, delete files, update files and etc to see everything works

To Do:
I haven't touch anything related to public datasets. Will address in another issue: #901

@longshuicy longshuicy marked this pull request as ready for review February 1, 2024 15:02
@longshuicy longshuicy requested a review from max-zilla as a code owner February 1, 2024 15:02
Copy link
Contributor

@tcnichol tcnichol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this for non-public datasets. It looks good and works, so I am marking approved.

@longshuicy longshuicy merged commit 4ea057c into 872-refactoring-pagination Feb 5, 2024
@longshuicy longshuicy deleted the 896-folder-plus-file-pagination branch February 5, 2024 18:16
longshuicy added a commit that referenced this pull request Feb 6, 2024
* might work

* working now

* model working now

* everything in dataset should work

* facet skip limit works

* wire pagination to explore page

* set model; but can't make id work

* user has quirks but kinda works

* files pagination

* revert dataset.tsx change

* file not working

* need to replace pyObjectId with pydanticObjectId otherwise the objectId can't be json serialized

* update to pydanticObjectId

* clean up files component

* file and datasets should all work now

* toggle user works

* fix everything with search user

* fix group pagination

* fix share group

* file pagination should only stay in file page

* apikey pagination works

* metadata defintion refactored

* listener jobs not quite working

* listener pagination finished

* rewrite sort

* replace all pyObjectId to pydanticObjectId

* extraction history hooked with backend

* fix pytest

* sorting works

* black

* black formatter

* fix bug

* more metadata bugs

* update pages

* moving to config

* metadata plural to singualar

* summarize page metadata into a function

* use metadata construction function in dataset

* upgrade black to latest version see if it fixes the linting issue

* Revert "upgrade black to latest version see if it fixes the linting issue"

This reverts commit aa660df.

* pin black action version to 23.3.0 to match pipfile

* black

* 896 folder plus file pagination (#903)

* fix folder pagination

* fix metadata pagination bug

* fix another one

* flipping logic clean up

* frontend pagination is impossible...

* Revert "frontend pagination is impossible..."

This reverts commit 41702ac.

* the view is working

* redux for folder and files

* listing works

* fix query bug

* nested folder correct now

* allow fancy sorting

* fix subfolder and file pagination issue

* remove outdated redux action and states

* black

* adjust file folder per page

* 901 pagination update with public datasets (#907)

* fix folder pagination

* fix metadata pagination bug

* fix another one

* flipping logic clean up

* frontend pagination is impossible...

* Revert "frontend pagination is impossible..."

This reverts commit 41702ac.

* the view is working

* redux for folder and files

* listing works

* fix query bug

* nested folder correct now

* allow fancy sorting

* fix subfolder and file pagination issue

* remove outdated redux action and states

* black

* public endpoint pagination conversion

* public folder paths not working

* fix public folder path

* black

* use utility function to construct page metadata

* small fix, call to a method that did not exist for public datasets

---------

Co-authored-by: toddn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants