Skip to content

Conversation

@benoit74
Copy link
Collaborator

@benoit74 benoit74 commented Jun 28, 2024

Fix #164
Fix #156

Changes:

  • New utility function to compute tags from scraper defaults and user provided values
  • Also raise error in tags validation when duplicates are found, either in string with semi-colon separator or in

This does not deduplicate automatically tags, it is still the scraper responsibility to properly pass tags (ideally with the new utility function) since config_metadata is more a wrapper passing directly the values as-is, I did not wanted to make an exception for tags

Edit: I had to also modify URL used for yt-dlp tests because former Vimeo URL is now returning a 403 on Github runners. Hopefully the new Jeena Peertube instance will be a little bit more permissive for few years ...

@benoit74 benoit74 self-assigned this Jun 28, 2024
@benoit74 benoit74 requested a review from rgaudin June 28, 2024 10:08
@benoit74
Copy link
Collaborator Author

Oups, CI is failing, I've requested review too early, sorry

@benoit74
Copy link
Collaborator Author

Arf, this has nothing to do with my change, it is Vimeo returning 403 on yt_dlp tests ...

@codecov
Copy link

codecov bot commented Jun 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (07cb331) to head (4402698).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #175   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           32        32           
  Lines         1439      1448    +9     
  Branches       248       251    +3     
=========================================
+ Hits          1439      1448    +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@benoit74 benoit74 marked this pull request as ready for review June 28, 2024 10:22
@benoit74
Copy link
Collaborator Author

Moved to Jeena Peertube instance instead of Vimeo ... this will work the time it will ...

Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM although I find it odd to specify default tags as iterable and user ones as string but I understand it's because default is immutable from constant and user ones from CLI

@benoit74
Copy link
Collaborator Author

benoit74 commented Jul 4, 2024

Yep, maybe we could support both types for both variables, but there is no use-case as far as I can tell and we've agreed to not implement something "should it become a need somewhere in the future".

@benoit74 benoit74 merged commit 7ab3fcd into main Jul 4, 2024
@benoit74 benoit74 deleted the tags_utilities branch July 4, 2024 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add utility function to compute ZIM Tags Deduplicate ZIM tag values

3 participants