Updated API endpoints, return payload, reducers and actions #330
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…xtractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend
arunapa
added
frontend
longshuicy
changed the base branch from
main
to
261-read-and-display-extractor-logs-on-frontend
longshuicy
approved these changes
Feb 16, 2023
…61-extractor-logs-backend
max-zilla
pushed a commit
that referenced
this pull request
Feb 28, 2023
* Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]>
longshuicy
added a commit
that referenced
this pull request
Feb 28, 2023
* Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]>
longshuicy
added a commit
that referenced
this pull request
Feb 28, 2023
* various fixes to message listener * black formatting * Added new component for file actions (#352) * Added new component for file actions * Tested file download and file delete from File page * Verified after deleting file, page navigates back to main dataset page * Fixed version-not-updating error * Updated onSave sequence to first synchronously update the file and then subsequently call listVersions API to display on the frontend * Fixed lint issues by running eslint * UI for displaying logs on extractors (#317) * Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> * Millisecond (#360) * it's seconds * add more icons * default to open tab --------- Co-authored-by: Aruna Parameswaran <[email protected]> Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]>
longshuicy
added a commit
that referenced
this pull request
Mar 2, 2023
* Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]>
longshuicy
added a commit
that referenced
this pull request
Mar 2, 2023
* various fixes to message listener * black formatting * Added new component for file actions (#352) * Added new component for file actions * Tested file download and file delete from File page * Verified after deleting file, page navigates back to main dataset page * Fixed version-not-updating error * Updated onSave sequence to first synchronously update the file and then subsequently call listVersions API to display on the frontend * Fixed lint issues by running eslint * UI for displaying logs on extractors (#317) * Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> * Millisecond (#360) * it's seconds * add more icons * default to open tab --------- Co-authored-by: Aruna Parameswaran <[email protected]> Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]>
max-zilla
added a commit
that referenced
this pull request
Mar 6, 2023
* Added group_id to authorization * Added list of group_ids instead of one group_id per entry * Addressing comments * fixing lint failure * Added new component for file actions (#352) * Added new component for file actions * Tested file download and file delete from File page * Verified after deleting file, page navigates back to main dataset page * Fixed version-not-updating error * Updated onSave sequence to first synchronously update the file and then subsequently call listVersions API to display on the frontend * Fixed lint issues by running eslint * UI for displaying logs on extractors (#317) * Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> * Properly interpret extractor statuses (#356) * various fixes to message listener * black formatting * Added new component for file actions (#352) * Added new component for file actions * Tested file download and file delete from File page * Verified after deleting file, page navigates back to main dataset page * Fixed version-not-updating error * Updated onSave sequence to first synchronously update the file and then subsequently call listVersions API to display on the frontend * Fixed lint issues by running eslint * UI for displaying logs on extractors (#317) * Static UI for displaying logs on extractors * Created 2 React custom components: * ExtractorStatus -- UI table to display summary of job/progress so far * ExtractorLogs -- terminal-like UI to display logs in real-time as they are being fetched * [WIP] Enable extractor job id retrieval from backend * Updated redux store value when job id is returned * Added actions for job update, job summary API endpoints * [WIP DO NOT MERGE] Testing frontend functionality with dynamic log rendering * Updated API endpoints, return payload, reducers and actions (#330) * Updated API endpoints, return payload, reducers and actions for the extractor jobs * Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data * Tested submit job process flow, able to retrieve extractor job id from the backend * Removed redundant code --------- Co-authored-by: Chen Wang <[email protected]> * Integrated UI with backend * Tested with sample extractor logs, able to view both extractor status and summary from the UI * Removed hardcoding of job id * Fixed indentation * Adding download button on UI to download correct verison file (#334) * Adding download button on UI to download correct verison file * fix filename issue * Update FileVersion.ts --------- Co-authored-by: Chen Wang <[email protected]> * fix member typing (#336) * fix member typing * adjust the test so it can pass * fixing error of blank page on submit file to extractor (#338) files.py - renaming resubmit method codegen fixing error in listeners.py * Modified extractor job summary duration calculation * Moving fetchJobSummary, fetchJobSummary API calls to ExtractorStatus component to fix interval issue --------- Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> * Millisecond (#360) * it's seconds * add more icons * default to open tab --------- Co-authored-by: Aruna Parameswaran <[email protected]> Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Dipannita <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> * Context matches v1 type. (#328) * context is now changed to list union of AnyUrl or dict. This should make v2 compatible with some extractors from extractors-core which dynamically build the context. * formatting fix test * formatting * still getting pytest errors changed context from optional[list] to list with default value empty, but still is not fixing the issues. * should fix the tests * formatting * getting rid of class 'context element' not needed * getting rid of class 'context element' not needed * removing console log --------- Co-authored-by: Chen Wang <[email protected]> * Mongo views (#353) * add a readme * add listener job update view * temp * add init * add to production docker-compose as well * 343 list resources that user has access (#355) * add logic to filter user * add filters to file * list files working too * add same filter for folder * include executions * exempt the current resource owner * add test database * rewrite createView part * rewrite init script but it's still not working yet * fix typo * fix linting * Replaced EmbeddedSearch with custom search box component to fix session refresh issue (#365) * Tested dataset search by clicking button and pressing enter, able to view search results * match user_ids which is a list now (#368) * update to lookup query * update authorization_deps queries * simplify group logic in auth check * formatting * Formatting --------- Co-authored-by: Aruna Parameswaran <[email protected]> Co-authored-by: Chen Wang <[email protected]> Co-authored-by: Todd Nicholson <[email protected]> Co-authored-by: Max Burnette <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updated API endpoints, return payload, reducers and actions for the extractor jobs
Tested API calls to fetchJobSummary, fetchJobUpdates from frontend, able to receive extractor job data
Tested submit job process flow, able to retrieve extractor job id from the backend