Skip to content

Conversation

@ddey2
Copy link
Member

@ddey2 ddey2 commented Jan 17, 2023

  1. Metadata version updated in mogodb with every metadata update
  2. Show latest available metadata if metadata with specific file version is not available

@ddey2 ddey2 linked an issue Jan 17, 2023 that may be closed by this pull request
@ddey2 ddey2 requested review from lmarini and tcnichol January 17, 2023 20:30
@tcnichol
Copy link
Contributor

I think doing the black formatting should fix the build error, then I can mark this one approved.

@ddey2
Copy link
Member Author

ddey2 commented Jan 17, 2023

I think doing the black formatting should fix the build error, then I can mark this one approved.

No idea why it's failing for a file where I haven't made any changes. However, still trying to format the file.

@longshuicy
Copy link
Member

Sorry for not understanding the context, do you mean that if metadata has been changed, the file version need to be automatically bumped up? If so could you point me to the part that file_version has been updated?

@ddey2
Copy link
Member Author

ddey2 commented Jan 20, 2023

Sorry for not understanding the context, do you mean that if metadata has been changed, the file version need to be automatically bumped up? If so could you point me to the part that file_version has been updated?

There is 2 parts to this PR:

  1. I enabled the 'all_version' from UI end so that it captures the file version whenever there is a request to update metadata.
  2. We want to update the version in 'metadata' table in mongodb, current scenario we weren't updating the version. The reason we want to update it - If a file is updated, we first want to fetch metadata associated with that specific version, if it's not available, we want to show the latest version metadata.

I hope that answers your question. Lmk if you have any more.

public static getFileMetadataApiV2FilesFileIdMetadataGet(
fileId: string,
allVersions: boolean = false,
allVersions: boolean = true,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@longshuicy I enabled this here to send the version with the request to update metadata

Metadata document that was updated
"""

# check if metadata with file version exists, replace metadata if none exists
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace_metadata is the function which is already taking care of updating resource object etc.

If there is no specific version of metadata is available in mongodb, we want to update the version, hence called 'replace_metadata'

@ddey2 ddey2 changed the title File version updated with metadata update Metdata version updated with metadata update Jan 20, 2023
@lmarini
Copy link
Member

lmarini commented Jan 23, 2023

@ddey2 can you resolve conflicts? I am not seeing the black check. Does it only run it if there is no conflict?

@tcnichol
Copy link
Contributor

tcnichol commented Jan 26, 2023

I added a file and added metadata. I updated the file, and when i changed the metadata it updated the version to v2. Is this the expected behavior? Or should the file v1 metadata still be saved in the db?

@ddey2
Copy link
Member Author

ddey2 commented Jan 26, 2023

I added a file and added metadata. I updated the file, and when i changed the metadata it updated the version to v2. Is this the expected behavior? Or should the file v1 metadata still be saved in the db?

This should be the expected behavior for now (as per our discussion). We plan to keep all the metadata versions, but that's for future. I have created an issue for that #277

Copy link
Contributor

@tcnichol tcnichol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and this work. Marking approved.

{
"_id": ObjectId(metadata_in.metadata_id),
"resource.resource_id": ObjectId(file_id),
"resource.version": metadata_in.file_version,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the resource_id and version filters are unnecessary here since the _id of the metadata should be unique.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's necessary because I need to find out if metadata with the specific file id and version id exists. If it does, we can simple update the metadata here in this method, else I need to call 'replace_file_metadata' where it creates a new object with the file id and version. I could have updated the current method but chose to re-use what we already have.

@ddey2 ddey2 requested a review from max-zilla February 3, 2023 20:24
@max-zilla max-zilla merged commit 5e0dacc into main Feb 3, 2023
@max-zilla max-zilla deleted the 266-extractor-metadata-and-file-versioning branch February 3, 2023 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extractor metadata and file versioning

6 participants